Three.js: transforming textures into data textures

Currently, my goal is to create a delayed webcam viewer using javascript, utilizing Three.js for its WebGL capabilities.

At the moment, I am able to capture frames from the webcam and display them after a specified time interval using canvas and getImageData(). An example of this implementation can be found here.

However, my aim is to achieve this without relying on canvas, but rather utilizing the Texture or DataTexture object from Three.js. You can see my attempt at this here. The challenge I am facing is figuring out how to transfer the image from one Texture (where the image is of type HTMLVideoElement) to another.

Within the rotateFrames() function, I need the older frame to be replaced by the newer one in a FIFO manner. However, the line

frames[i].image = frames[i + 1].image;

simply copies the reference, not the actual texture data. I suspect that utilizing DataTexture might be the solution, but I am struggling to extract a DataTexture from a Texture or HTMLVideoElement.

Any thoughts or suggestions on how to approach this?

Important Note: To run the examples successfully, ensure you have access to a camera and have granted permission for the browser to use it. Additionally, make sure you are using an updated browser.

javascripthtml5-canvaswebglthree.js

Answer №1

Are you searching for this information?

http://fiddle.jshell.net/m4Bh7/10/

Edit:

function onFrame(dt) {
    if (video.readyState === video.HAVE_ENOUGH_DATA) { /* new frame available from webcam */
        context.drawImage(video, 0, 0,videoWidth,videoHeight);
        frames[framesNum - 1].image.data = new Uint8Array(context.getImageData(0,0,videoWidth, videoHeight).data.buffer);
        frames[framesNum - 1].needsUpdate = true;
    }
}

This part is crucial: It duplicates the frame and stores it as data in a dataTexture.

function rotateFrames() {
    for (var i = 0; i != framesNum - 1; ++i) {
/*
         * FIXME: this does not work!
         */
        frames[i].image.data = frames[i + 1].image.data;
        frames[i].needsUpdate = true;
    }
}

It transfers the data from one texture to another within the frames.

New version: http://fiddle.jshell.net/hWL2E/4/

function onFrame(dt) {
    if (video.readyState === video.HAVE_ENOUGH_DATA) { /* new frame available from webcam */
        context.drawImage(video, 0, 0,videoWidth,videoHeight);
         var frame = new THREE.DataTexture(new Uint8Array(context.getImageData(0,0,videoWidth, videoHeight).data.buffer) ,videoWidth,videoHeight);
        frames[framesNum - 1] = frame;
        frames[framesNum - 1].needsUpdate = true;
        sprites[framesNum - 1].map = frames[framesNum - 1];
    }
}

It creates a new texture for every video frame.

function rotateFrames() {
    for (var i = 0; i != framesNum - 1; ++i) {
/*
         * FIXME: this does not work!
         */
        frames[i] = frames[i + 1];
        sprites[i].map = frames[i];

    }
}

This improvement consists of reusing the textures, eliminating the need for re-sending to the GPU.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

what is the mechanism behind __dirname in Node.js?

Node.js is a new technology for me and I recently discovered the "__dirname" feature which is really useful for obtaining the absolute path of the script. However, I am intrigued by how it works and how it manages to interpret the directory structure. De ...

javascriptnode.js

Utilizing Azure Mobile Services with HTML

Is there a way to integrate my Windows Azure Mobile Services into an HTML page? I attempted to utilize the code snippets provided in the Azure portal, but unfortunately, they did not work for me. The prescribed code snippets that need to be included in m ...

javascripthtmlazureazure-mobile-services

Tips for running a function at regular intervals in NodeJS

I've experimented with the setInterval() method before. While it seemed ideal, the problem I encountered was that it didn't start the first call immediately; instead, it waited for X seconds before producing the desired value. Is there an alterna ...

javascriptnode.js

The useEffect hook is failing to trigger

Currently, I am attempting to retrieve data from an API using Redux for state management. Despite trying to dispatch the action within the useEffect hook, it does not seem to be functioning properly. I suspect that the issue lies with the dependencies, but ...

javascriptreactjsredux

What is the process for setting up both an open and closed status for my accordion?

After browsing through multiple threads on accordions, none seem to match my current structure which I believed was effective. In an attempt to learn and build elements from scratch, I created the following accordion with some help from Google and experi ...

javascriptjquerycss

Unable to access an element using jquery

This is an example of an HTML file: <div id ="main"> </div> Here is the JavaScript code: //creating a new div element var divElem = $('<div class="divText"></div>'); //creating an input element inside the div var i ...

javascriptjquery

Retrieving the selected date from JqueryUI Datepicker after form submission

I am currently using an inline datepicker that fills in an input textbox. $("#left-calendar").datepicker({ altField: "#left-date-text" }); The left-date-text is located within a form, and upon submission with a submit button, it sends the data to a p ...

phpjavascriptjqueryjquery-uidatepicker

Using the onClick function to set attributes in React JS

I am currently working with 2 functions: addBookmark() and removeBookmark(). There is a variable called IsBookmarked that can be either true or false. In my JSX code related to bookmarks, it looks like this: {props.singleCompany.IsBookmarked ? ( ...

javascripthtmlreactjsjsx

Vue.js computed property experiencing a minor setback

I'm currently working on developing a test-taking system using Vue and Laravel. When a user inputs the test code and email address, they are directed to the test page. To display all the test questions based on the entered code, I implemented a naviga ...

javascriptfunctionvue.js

Achieving Optimal Performance with Node.js Cluster in High-Traffic Production Settings

Currently, my web service is managing http requests to redirect users to specific URLs. The CPU is currently handling around 5 million hits per day, but I am looking to scale it up to handle over 20 million. However, I am hesitant about using the new Nod ...

javascriptnode.jscluster-computingnode-cluster

How can I transfer the data from a file to upload it in Angular 9 without manually typing it out?

In my Angular application, I have a functionality where users can upload two files for processing on the server. However, I am looking to add a feature that allows users to simply copy and paste the contents of the files into two textboxes instead of going ...

javascriptangulartypescriptangular9

Error: Document's _id field cannot be modified

I am new to both MongoDB and Backbone, and I find it challenging to grasp the concepts. My main issue revolves around manipulating attributes in Backbone.Model to efficiently use only the necessary data in Views. Specifically, I have a model: window.User ...

javascriptmongodbbackbone.jsexpress

Using Nuxt.js to import custom NPM packages on a global scale

The installation process for Nuxt's plugins/modules system can be quite intricate. Despite attempting to follow various suggestions, I have struggled to accomplish a seemingly simple task. After installing the NPM package csv-parse (which can be found ...

javascriptvue.jsnpmnuxt.js

Having trouble with the initial tap not being recognized on your mobile browser?

When using mobile web browsers (specifically chrome and firefox on iOS), I am experiencing an issue where the hamburger menu does not trigger when tapped for the first time. For a simplified version of the HTML/CSS/JS code, you can check it out at: https ...

javascripthtmlcssmobile

Creating interactive <td> elements in HTML with JavaScript editor capabilities

Currently, I am working on creating an editable table feature and managed to find a good example to follow. However, I have encountered some issues along the way. <td contenteditable="true" class="hover" onBlur="saveToDatabase(this,'question' ...

javascriptphpjqueryhtml

Creating a dynamic background color that pulses with animation

I recently created a script to animate the menu li element when hovering over the corresponding a element. Everything is functioning as expected, but now I'm looking to enhance the effect by having it continue as long as the mouse remains over the a e ...

javascriptjqueryhtmlcssjquery-animate

Transfer an array via Ajax to a Python server script

I need to send the names, values, and labels of my form elements when a button is clicked. Since the submit button messes up the order, I decided to handle it with JavaScript: $('#mybutton').click(function() { m.modal('show'); ...

javascriptpythonajax

What methods are available for managing model fields within a for loop in Django?

As a newcomer to Django, I am embarking on the journey of creating a Quiz web application that presents one question at a time. Upon selecting an answer using radio buttons, the application should promptly display whether the response is correct or incorre ...

javascripthtmldjangofor-loopdjango-quiz

What is the most effective way to determine which radio button is currently selected in a group with the same name using JQuery?

<form class="responses"> <input type="radio" name="option" value="0">False</input> <input type="radio" name="option" value="1">True</input> </form> Just tested: $('[name="option"]').is(':checked ...

javascriptjqueryhtml

Efficiently pinpointing the <div> element with precision using jQuery

I am building an interactive quiz for users of my app. The questions are table based, and can be answered "yes," "no," or "maybe." If the user answers "no" or "maybe," I would to slide open an informational panel that lives beneath the table. Here is the ...

javascriptjqueryhtml