Extracting video frames in three.js for future reference

Question

Extracting video frames in three.js for future reference

Currently, I am utilizing three.js version r73 and have implemented a fragment shader that showcases video frames with code like this:

    gl_FragColor = texture2D( iChannel0,  uv);

My objective is to introduce another uniform (let's call it iChannel1) that stores the video frame displayed at an earlier stage. The plan is to use this for motion capture, edge detection, and similar functions.

Despite my efforts, I have not been successful in achieving this goal.

I believe I need to duplicate the video texture somehow using JavaScript and then assign it to iChannel1, but I am unsure of how to grab a frame.

One option could be capturing the canvas content, but there might be unwanted noise present on the canvas. I specifically require the video frame, not the surrounding canvas. Additionally, going through the canvas capture process seems inefficient. It feels like there should be a straightforward API call to capture the current video frame effectively for creating a texture used by iChannel1.

I explored UniformsUtils as well, but it didn't provide the solution I was seeking either.

javascript three.js

Answer 1

Answer №1

After posting my question, I quickly discovered the solution myself. Oops!

For future reference, here is the method I used:

if (timeToCaptureFrame) {
    that.uniforms.iChannel1.value.image = that.uniforms.iChannel0.value.image;
    that.uniforms.iChannel1.value.needsUpdate = true;
}

It's important to note that iChannel1 must be initialized correctly for this approach to work. An example initialization would look like this:

var pathToSubtractionTexture = 'aStillImage.jpg';
(new THREE.TextureLoader()).load(pathToSubtractionTexture, function ( texture ) {
    that.uniforms.iChannel1.value = texture;
});

Answer 2

After posting my question, I quickly discovered the solution myself. Oops!

For future reference, here is the method I used:

if (timeToCaptureFrame) {
    that.uniforms.iChannel1.value.image = that.uniforms.iChannel0.value.image;
    that.uniforms.iChannel1.value.needsUpdate = true;
}

It's important to note that iChannel1 must be initialized correctly for this approach to work. An example initialization would look like this:

var pathToSubtractionTexture = 'aStillImage.jpg';
(new THREE.TextureLoader()).load(pathToSubtractionTexture, function ( texture ) {
    that.uniforms.iChannel1.value = texture;
});

Extracting video frames in three.js for future reference

Answer №1

Similar questions

What is the most efficient method for assigning a JSON array?

Update in Angular version 1.5.9 regarding $location.hashPrefix causing changes

AngularJS date formatting fails to properly format dates

AngularJS confirmation directive for deleting items

Angular 6: TypeError - The function you are trying to use is not recognized as a valid function, even though it should be

When utilizing the built-in filter in Angular 2 ag-grid, the Clear Filter button efficiently removes any text from the filter box without needing to refresh the

The MaterialTable component is indicating that there is no property called 'tableData' on the IPerson type

Increasing the Efficiency of Styled Components

Join the nested object using the delimiter ';'

Learn the process of uploading an image to Firebase storage from the server side

Execute javascript whenever the page is being loaded

Error encountered when trying to access Proxy Object or Object[] in MobX with TypeScript

Leveraging await with jsmediatags

Ways to decrease the size of this item while maintaining its child components?

My three.js program isn't displaying anything on the screen, even though I'm confident that my

Refreshing Yii Dropdown List After Adding a New Item

Is it possible to generate a post automatically every second using an AJAX call, with no user input required?

Parsing error: Unforeseen token encountered. Consider adding a supplementary loader to manage the output of these loaders

Determine the originating component in React by identifying the caller

Guide on transforming a JSON string into an array of custom objects using the json2typescript NPM module within a TypeScript environment