Using recycled frame buffers in a threejs fragment shader

I'm currently working on a project to develop an app that emulates the effect of long exposure photography. The concept involves capturing the current frame from the webcam and overlaying it onto a canvas. As time progresses, the image will gradually 'expose', becoming brighter and more vivid. (refer to )

My shader functions flawlessly, mimicking the 'add' blend mode in Photoshop perfectly. However, I am facing difficulty in getting it to recycle the previous frame.

Initially, I assumed that a simple solution like renderer.autoClear = false; would resolve this issue but unfortunately, this option seems ineffective in the current context.

Below is the code snippet utilizing THREE.EffectComposer to apply the shader.

        onWebcamInit: function () {    
            var $stream = $("#user-stream"),
                width = $stream.width(),
                height = $stream.height(),
                near = .1,
                far = 10000;

            this.renderer = new THREE.WebGLRenderer();
            this.renderer.setSize(width, height);
            this.renderer.autoClear = false;
            this.scene = new THREE.Scene();

            this.camera = new THREE.OrthographicCamera(width / -2, width / 2, height / 2, height / -2, near, far);
            this.scene.add(this.camera);

            this.$el.append(this.renderer.domElement);

            this.frameTexture = new THREE.Texture(document.querySelector("#webcam"));
            this.compositeTexture = new THREE.Texture(this.renderer.domElement);

            this.composer = new THREE.EffectComposer(this.renderer);

            // Same effect with or without this line
            // this.composer.addPass(new THREE.RenderPass(this.scene, this.camera));

            var addEffect = new THREE.ShaderPass(addShader);
            addEffect.uniforms[ 'exposure' ].value = .5;
            addEffect.uniforms[ 'frameTexture' ].value = this.frameTexture;
            addEffect.renderToScreen = true;
            this.composer.addPass(addEffect);

            this.plane = new THREE.Mesh(new THREE.PlaneGeometry(width, height, 1, 1), new THREE.MeshBasicMaterial({map: this.compositeTexture}));
            this.scene.add(this.plane);

            this.frameTexture.needsUpdate = true;
            this.compositeTexture.needsUpdate = true;

            new FrameImpulse(this.renderFrame);

        },
        renderFrame: function () {
            this.frameTexture.needsUpdate = true;
            this.compositeTexture.needsUpdate = true;
            this.composer.render();
        }

Here is the shader implementation. It is straightforward and plain.

        uniforms: {
            "tDiffuse": { type: "t", value: null },
            "frameTexture": { type: "t", value: null },
            "exposure": { type: "f", value: 1.0 }
        },

        vertexShader: [
            "varying vec2 vUv;",
            "void main() {",
            "vUv = uv;",
            "gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );",

            "}"
        ].join("\n"),

        fragmentShader: [

            "uniform sampler2D frameTexture;",
            "uniform sampler2D tDiffuse;",
            "uniform float exposure;",
            "varying vec2 vUv;",

            "void main() {",
            "vec4 n = texture2D(frameTexture, vUv);",
            "vec4 o = texture2D(tDiffuse, vUv);",
            "vec3 sum = n.rgb + o.rgb;",
            "gl_FragColor = vec4(mix(o.rgb, sum.rgb, exposure), 1.0);",
            "}"

        ].join("\n")

Answer №1

My approach is similar to the one offered by posit labs, but I have found success with a more efficient solution. I create an EffectComposer containing only the ShaderPass that needs to be reused, and then alternate between renderTargets for that composer during each render.

To initialize:

THREE.EffectComposer.prototype.swapTargets = function() {
    var tmp = this.renderTarget2;
    this.renderTarget2 = this.renderTarget1;
    this.renderTarget1 = tmp;
};

...

composer = new THREE.EffectComposer(renderer,  
    new THREE.WebGLRenderTarget(512, 512, { minFilter: THREE.LinearFilter, magFilter: THREE.NearestFilter, format: THREE.RGBFormat })
);

var addEffect = new THREE.ShaderPass(addShader, 'frameTexture');
addEffect.renderToScreen = true;
this.composer.addPass(addEffect);

For rendering:

composer.render();
composer.swapTargets();

A secondary EffectComposer can then take either of the two renderTargets and display it on screen or apply further transformations.

I also specify "frameTexture" as textureID when initializing the ShaderPass. This informs ShaderPass to update the frameTexture uniform using the outcome of the previous pass.

Answer №2

In order to create a feedback effect like this, you must switch between writing to different instances of WebGLRenderTarget. If you don't do this, the frame buffer will be overwritten. The reason behind this behavior is not entirely clear, but here's how you can solve it.

Initialization:

    this.rt1 = new THREE.WebGLRenderTarget(512, 512, { minFilter: THREE.LinearFilter, magFilter: THREE.NearestFilter, format: THREE.RGBFormat });
    this.rt2 = new THREE.WebGLRenderTarget(512, 512, { minFilter: THREE.LinearFilter, magFilter: THREE.NearestFilter, format: THREE.RGBFormat });

Rendering:

    this.renderer.render(this.scene, this.camera);
    this.renderer.render(this.scene, this.camera, this.rt1, false);

    // swap buffers
    var temp = this.rt2;
    this.rt2 = this.rt1;
    this.rt1 = temp;
    this.shaders.add.uniforms.tDiffuse.value = this.rt2;

Answer №3

Experiment with the following option:

this.canvas = new THREE.WebGLCanvas( { retainDrawingBuffer: true } );

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Can you please explain the concept of straightforward UI routes to me?

I recently discovered that ui-routes are more powerful and advantageous than ngRoutes. In an effort to learn more about the ui-routing feature, I started studying from http://www.ng-newsletter.com/posts/angular-ui-router.html. However, I found it challengi ...

Discovering and editing a specific line in Sheets: An in-depth look at the Message Counter

Currently, the bot checks if your ID already exists in the sheet list and adds you if it doesn't when someone writes a message in the chat. Now, I want the bot to implement a message counter in the sheet list. What would be the most effective way to ...

Rails 7 is missing the Toast element from Bootstrap 5

Having some trouble with implementing Bootstrap 5 Toast in Rails 7 for flash messages. Here is the code I am currently using: # application.html.erb <head> ... <%= javascript_importmap_tags %> <script> const toastElList = document.que ...

Is it possible to import files in Vue JavaScript?

I want to incorporate mathematical symbols from strings extracted from a JSON file. While it seems to work perfectly on this example, unfortunately, I encountered an issue when trying it on my own machine. The error message 'Uncaught (in promise) Refe ...

The values in my JavaScript don't correspond to the values in my CSS

Is it possible to retrieve the display value (display:none; or display:block;) of a div with the ID "navmenu" using JavaScript? I have encountered an issue where I can successfully read the style values when they are set within the same HTML file, but not ...

Ways to verify if a string is a number without using isNaN and with specified criteria

I am trying to determine if a string represents a valid number without relying on the isNaN function. The reason for this is that I need to be able to accept the ',' character, which isNaN ignores. Additionally, I do not want to allow negative nu ...

Unable to adjust layout when code is functioning alongside background-color

I'm looking to dynamically change the position of an item on my webpage when it is clicked. Is there a way I can achieve this without relying on id names? I currently have a code snippet that successfully changes the background color, but for some rea ...

An argument error in IE 8 caused by an invalid procedure call

Is there a way to access the opener's class in a child window created using window.open? This works smoothly in W3C browsers, but fails in IE 8. On the other hand, I tested using an iframe and it seems to work fine across all browsers. The main goal o ...

Information sent by the Firefox TCP socket using the socket.send() method cannot be retrieved until the socket is closed

I am experiencing an issue while trying to send data from Firefox to a Java desktop application. My Java class functions as a server, and the Firefox script acts as a client. When I test it using another Java class called client.java, the data is successfu ...

Sort with AngularJS: orderBy multiple fields, with just one in reverse

Currently, I am faced with the challenge of filtering data based on two variables: score and name (score first, followed by name). This task involves various games, some of which have scores in reverse order (like golf) while others maintain a normal scor ...

A guide on customizing the appearance of individual items in a vue v-for loop based on specific conditions

I am currently developing a multiple choice quiz game and I want the selected answer by the user to change color, either red or green, depending on its correctness. To achieve this, I have created a variable called selected that correctly updates when the ...

Deactivate Mongoose connection in Node.js after completing tasks

Here is a mongoose script I have been using to connect to the local database and perform some operations. However, I am facing an issue with disconnecting the connection after use. const mongoose = require('mongoose'); const db = mongoose.connec ...

Having trouble with Redux's mapStateToProps function?

I'm working on a React component that triggers an AJAX call in the componentWillMount lifecycle method and then stores the fetched data in a Redux store. Here's the code snippet: componentWillMount() { var self = this; var xmlhttp = new XMLH ...

Is there a way to incorporate a variable into a JSON URL?

I'm attempting to incorporate a variable I have defined into the JSON URL: var articleName = "test"; $.getJSON( "https://www.googleapis.com/customsearch/v1?key=API_MY&cx=CX_MY&q='+articleName+'&searchType=image&fileType= ...

Ways to implement material-ui button design on an HTML-native button

I am using pure-react-carousel which provides me an unstyled HTML button (ButtonBack). I would like to customize its style using material-ui. Trying to nest buttons within buttons is considered not allowed. An approach that works is manually assigning th ...

Saving information to a hidden div using jStorage

Currently, I am utilizing jStorage for storing data in local storage. When I store the data using console.log() and later retrieve it using $.jStorage.get(), I found that the values are not being assigned to the hidden div. Can someone provide guidance o ...

AngularJS Alert Timer Setting

Is there a way to display the timer value in AngularJS? I have included the following code for the timer tag: <timer interval="1000" countdown="100">{{countdown}}</timer> I have also added an alert function in the script.js file to display th ...

Issues with implementation of map and swiper carousel functionality in JavaScript

I am trying to populate a Swiper slider carousel with images from an array of objects. However, when I use map in the fetch to generate the HTML img tag with the image data, it is not behaving as expected. All the images are displaying in the first div a ...

Choose a random cell in Jquery every second

I am searching for a method to choose one div out of sixteen and change its CSS backgrounds. This selection needs to occur every second, so a new div is selected from the group each second. I am struggling with implementing the "every second" functionalit ...

What are the steps to showcase a multiplication chart based on user-inputted rows and columns using jQuery?

I'm currently facing some challenges with coding a multiplication table using jQuery. I already have the code to display the multiplication table based on inputted rows, but I need help in modifying it to allow for inputting both rows and columns. An ...