Calculating the screen-space coordinates for FBO value retrieval using a depth texture

EDIT: I have updated the JSFiddle link because it was not displaying correctly in Chrome on Windows 7.

Situation

I am experimenting with particles in THREE.JS and utilizing a frame buffer / render target (double buffered) to write positions to a texture. This texture is manipulated by its own ShaderMaterial, and then read by the PointCloud's ShaderMaterial to position the particles. Everything is working smoothly so far; all functions as anticipated.

My current objective is to utilize my scene's depth texture to determine if any particles are intersecting with my scene's geometry.

Initially, I referenced my depth texture in the PointCloud's fragment shader, utilizing

gl_FragCoord.xy / screenResolution.xy
to create the uv for the depth texture lookup.

Here is a JSFiddle demonstration. It functions well - when a particle is positioned behind an object in the scene, I change the particle's color to red instead of white.

However, I encounter issues when attempting to perform the same depth comparison in the position texture shader. In the draw fragment shader, I can use the value of gl_FragCoord to obtain the particle's position in screen space and utilize it for the depth uv lookup. This is achievable because in the draw vertex shader, I employ the modelViewMatrix and projectionMatrix to define the value of gl_Position.

Despite my efforts in the position fragment shader, I have been unsuccessful. To clarify, my objective is particle collision with the scene on the GPU.

The Question:

  • Given a texture where each pixel/texel represents a world-space 3D vector denoting a particle's position, how can I project this vector to screen-space in the fragment shader, ultimately using the .xy properties of this vector for a uv lookup in the depth texture?

My Attempts

  • In the position texture shader, I attempted to transform a particle's position to screen-space using model-view and projection matrices - which is similar to the process in the draw shader:

    // Position texture's fragment shader:
    void main() {
        vec2 uv = gl_FragCoord.xy / textureResolution.xy;
        vec4 particlePosition = texture2D( tPosition, uv );
    
        vec2 screenspacePosition = modelViewMatrix * projectionMatrix * vec4( particlePosition, 1.0 );
        vec2 depthUV = vec2( screenspacePosition.xy / screenResolution.xy );
        float depth = texture2D( tDepth, depthUV ).x;
    
        if( depth < screenspacePosition.z ) {
            // Particle is behind something in the scene,
            // so do something...
        }
    
        gl_FragColor = vec4( particlePosition.xyz, 1.0 );  
    }
    
  • Other variations I tried include:

    • Adjusting the depth's uv by 0.5 - depthUV
    • Utilizing the tPosition texture resolution instead of the screen resolution to scale the depthUV.
    • An alternate depth uv approach: using depthUV = (depthUV - 1.0) * 2.0;. This improved the situation slightly, but the scaling is incorrect.

Assistance would be greatly appreciated! Thank you in advance.

Answer №1

Through numerous trials and extensive research, I was able to pinpoint the root of the issue to the specific values assigned by THREE.js to modelViewMatrix and projectionMatrix when utilizing THREE.ShaderMaterial.

Initially, everything functioned correctly within my 'draw' shaders, where the value of modelViewMatrix was conveniently set to:

new THREE.Matrix4().multiplyMatrices( camera.matrixWorldInverse, object.matrixWorld)

Upon further investigation, it became apparent that when creating a ShaderMaterial to render textures independently (unattached to any object in the scene/world), the object.matrixWorld essentially amounted to an identity matrix. To align the values between my position texture shaders and draw shaders (linked to objects in the scene/world), I needed to ensure identical modelViewMatrix values.

Once that adjustment was made, the only remaining task was to accurately transform a particle's position to screen-space. To facilitate this process, I implemented several helper functions in GLSL:

    // Conversion of worldspace coordinate to clipspace
    // Note: `mvpMatrix` denotes: `projectionMatrix * modelViewMatrix`
    vec4 worldToClip( vec3 v, mat4 mvpMatrix ) {
        return ( mvpMatrix * vec4( v, 1.0 ) );
    }

    // Conversion of clipspace coordinate to screenspace
    vec3 clipToScreen( vec4 v ) {
        return ( vec3( v.xyz ) / ( v.w * 2.0 ) );
    }

    // Conversion of screenspace coordinate to a 2D vector for
    // utilization as a texture UV lookup
    vec2 screenToUV( vec2 v ) {
        return 0.5 - vec2( v.xy ) * -1.0;
    }

I've crafted a JSFiddle demonstration showcasing this concept here. The code is heavily commented to cater to those unfamiliar with such technicalities.

A brief clarification regarding the fiddle: its visual impact may seem modest, as I am simulating the effects of depthTest: true in the absence of that property being set on the PointCloud. Nevertheless, by altering the y position of particles colliding with scene elements to 70.0, a white band appears near the top of the screen. Subsequently, I aim to perform this computation within a velocity texture shader to enable valid collision responses.

I trust this explanation proves beneficial to others :)

EDIT: Here's an implementation of the above approach featuring a (possibly flawed) collision response.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Personalized ES6 Bootstrap module created for a toggle switch button in Vue

Utilizing the custom switch toggle in a Vue application is my current task. Check out this link for more on the custom switch toggle I found a button that suits my needs, but I am unsure how to properly integrate it into my component so it can handle the ...

uib-datepicker-popup allowing for changing minimum mode dynamically

Is there a way to dynamically set the minMode of an Angular Bootstrap Datepicker? I managed to achieve this using the following code: <input type="text" ng-model="myDate" uib-datepicker-popup="{{datepickerFormat}}" datepicker-options="{& ...

Uncover the secrets of HTML with JavaScript

I'm facing an issue with extracting data from a link's data attribute and trying to decode the HTML within it, but unfortunately, it's not functioning as expected. Check out my code on JSFiddle: http://jsfiddle.net/Kd25m/1/ Here's the ...

Next.js app experiencing issues with Chakra UI not transitioning to dark mode

After attempting to incorporate Chakra UI into my Next.js application, I carefully followed every step outlined in their documentation: Despite setting the initialColorMode to "dark" for the ColorModeScript prop, it seems that the dark mode is not being a ...

Searching for a substring within a larger string in JavaScript

I am trying to create a loop in JavaScript (Node.js) that checks if a string contains the "\n" character and then splits the string based on that character. <content>Hello </content><br /> <content>bob </content><br / ...

Is it possible to simultaneously wait for the completion of two methods instead of awaiting each one individually?

When dealing with 2 async methods, one may want to run them simultaneously but wait for both to finish before proceeding. Here is an example: exports.get = async id => { const part1 = await context.get(id); const part2 = await context.get2(id ...

Connecting UserIDs with Embedded Documents in Mongoose

My goal is to connect individuals with each other by embedding a Match document in the user's matches array. This is my User Model: const mongoose = require('mongoose'); const Schema = mongoose.Schema; const Match = new Schema({ with: ...

The output of jQuery('body').text() varies depending on the browser being used

Here is the setup of my HTML code: <html> <head> <title>Test</title> <script type="text/javascript" src="jQuery.js"></script> <script type="text/javascript"> function initialize() { var ...

What is the best way to make several revisions to a message on Discord?

Is there a way to edit the same message multiple times with a delay? Here is the approach I attempted: message.channel.send(`a`) .then(message => { setTimeout(() => { message.edit(`ab`) }, 1000); }) .then(message => { setT ...

Wait for the playwright to detect a specific and exact change in the inner text

There is a specific innerText that transitions from Loading to Play after 2-3 seconds. I want to wait for this change to happen before proceeding. Currently, I am using the following code snippet: let attempt = 0; let maxRetries = 4; let payerButtonStatus ...

Incorporate a pause into a JavaScript function

Consider the following function: $(document.body).ready(function() { var o = $(".hidden"); $(".about_us").click(function() { o.hasClass("visible") ? o.removeClass("visible") : o.addClass("visible"); }); }); I am looking to add a delay to this f ...

I am facing difficulty in getting React to recognize the third-party scripts I have added to the project

I am currently working my way through the React Tutorial and have encountered a stumbling block. Below is the HTML markup I am using: <html lang="en"> <head> <meta charset="utf-8" /> <meta name="viewport" content="width=devi ...

Creating a project that utilizes Bing Translate and the Node.js programming language

As I work on developing a web application that enables users to translate text using the Bing translator API, I am facing an issue. I attempted to execute a translator.js file via a script tag, but encountered a roadblock since Node.js code cannot be run t ...

Cookie settings are not functioning properly in Chrome when attempting to access two different websites

Having an issue with the Set-Cookie functionality not functioning properly in Chrome, although untested in other browsers. Previously, it was working fine but recently stopped working. I operate two websites with separate domain names, and it is essential ...

What steps should I take to adjust the price when multiple items are chosen?

I am working on a school project that requires JavaScript programming assistance. You can view the code here: https://jsfiddle.net/zvov1jpr/3/ HTML: <script src="java.js"></script> <div id="formular"> <div id=&qu ...

What is the process for creating a folder using Firebase Cloud Functions with Storage?

How do I create a folder named "posts"? Storage bucket path --> gs://app.appspot.com/posts Code to generate thumbnail from storage object exports.generateThumbnail = functions.storage.object() .onChange(event => { const object = event.data ...

Tips for implementing JS function in Angular for a Collapsible Sidebar in your component.ts file

I am attempting to collapse a pre-existing Sidebar within an Angular project. The Sidebar is currently set up in the app.component.html file, but I want to transform it into its own component. My goal is to incorporate the following JS function into the s ...

Node.js does not support running Bootstrap and CSS, resulting in style not being applied due to the MIME type ('text/html') not being a supported stylesheet MIME type

I attempted to test out the Start Bootstrap Bare template using Node.js and Express. (Link to Start Bootstrap Template) I opted not to make any alterations to the HTML, CSS, and JavaScript files provided with the template. Instead, I created a new index.j ...

What is the best way to integrate react-final-form with material-ui-chip-input in a seamless manner

Currently, I am utilizing Material UI Chip Input wrapped with Field from react-final-form. https://i.sstatic.net/vJKM1.jpg The main objective is to restrict the number of "CHIPS" to a maximum of 5. Chips serve as concise elements representing inputs, at ...

Retrieving child class prototype from superclass

In my current project, I am working on implementing a base class method that shares the same logic across all child classes. However, I need this method to utilize specific variables from each child class. function A() {} A.prototype.foo = 'bar' ...