Ways to incorporate lighting into the instancing shader

Is there a way to incorporate ambient and directional lighting into the shader when using InstancedBufferGeometry?

For example, I am looking to add lighting effects to a specific instance, like in this example:

Below is the vertex shader code:

precision highp float;
uniform mat4 modelViewMatrix;
uniform mat4 projectionMatrix;
uniform mat3 normalMatrix;
attribute vec3 position;
attribute vec3 offset;
attribute vec3 normal;
attribute vec2 uv;
attribute vec4 orientation;
varying vec2 vUv;

// lighting
struct DirectionalLight {
    vec3 direction;
    vec3 color;
    int shadow;
    float shadowBias;
    float shadowRadius;
    vec2 shadowMapSize;
    };
uniform DirectionalLight directionalLights[ NUM_DIR_LIGHTS ];
uniform vec3 ambientLightColor;
varying vec3 vLightFactor;
//

void main() {
    vec3 vPosition = position;
    vec3 vcV = cross(orientation.xyz, vPosition);
    vPosition = vcV * (2.0 * orientation.w) + (cross(orientation.xyz, vcV) * 2.0 + vPosition);
    vUv = uv;
    gl_Position = projectionMatrix * modelViewMatrix * vec4( offset + vPosition, 1.0 );

    // lighting
    vec4 ecPosition = modelViewMatrix*vec4(offset + vPosition,1.0);
    vec3 ecNormal= -(normalize(normalMatrix*normal));
    vec3 fromLight = normalize(directionalLights[0].direction);
    vec3 toLight = -fromLight;
    vec3 reflectLight = reflect(toLight,ecNormal);
    vec3 viewDir = normalize(-ecPosition.xyz);
    float ndots = dot(ecNormal, toLight);
    float vdotr = max(0.0,dot(viewDir,reflectLight));
    vec3 ambi = ambientLightColor;
    vec3 diff = directionalLights[0].color * ndots;
    vLightFactor = ambi + diff;     
    //

}

And here is the fragment shader code:

precision highp float;
uniform sampler2D map;
varying vec2 vUv;

// lighting
varying vec3 vLightFactor;
//

void main() {
    gl_FragColor = texture2D(map, vUv) * vec4(vLightFactor,1.0);
}

Lastly, the material setup:

var uniforms = Object.assign( 
    THREE.UniformsLib['lights'], 
    {
    map: { value: texture }
    }
);  

var material = new THREE.RawShaderMaterial({
    lights: true,
    uniforms: uniforms, 
    vertexShader: document.getElementById( 'vertexShader' ).textContent,
    fragmentShader: document.getElementById( 'fragmentShader' ).textContent,
});

Appreciate any guidance on this matter.

Answer №1

Within a rendering, every mesh in the scene typically undergoes transformation by the model matrix, view matrix, and projection matrix.

The model matrix dictates the position, orientation, and relative scale of a mesh within the scene. It transforms the vertex positions of the mesh into world space.

https://i.sstatic.net/LG29O.png

The view matrix specifies the viewpoint's direction and position in relation to the scene. It transforms from world space to view (eye) space.

https://i.sstatic.net/FnOCk.png

It's worth noting that the model view matrix modelViewMatrix combines the view matrix and model matrix. However, in some cases, the model matrix may be an identity matrix, making the modelViewMatrix essentially equal to the view matrix. This assumption is made when model transformations are done by vectors orientation and offset, rather than a model matrix.

Calculating light can be done in either view space or world space.
If the light calculations are in view space, light positions and directions need to be transformed from world space to view space. This conversion is commonly done on the CPU before each frame, setting up light parameter uniforms with view space coordinates. Since the view position is (0, 0, 0) in view space, the view vector becomes the normalized and inverse vertex position (in view space).

https://i.sstatic.net/PcUn5.png


Performing light calculations in view space is feasible when the light direction and position are set in view space (refer to three.js - Light). First, the normal vector must be transformed to world space before converting it to view space. This process mirrors how vertex position transformations are handled. Start by adding the normal vector to the vertex position, then transform this combined position to world space. The normal vector in world space is derived from the difference between the calculated position and the vertex position in world space.

vec3 wNPosition = position + normal;
vec3 wNV        = cross(orientation.xyz, wNPosition);
wNPosition      = wNV * 2.0 * orientation.w + cross(orientation.xyz, wNV) * 2.0 + wNPosition;
vec3 wNormal    = normalize( wNPosition - vPosition );


Given these assumptions, your shader code may resemble the following:

vec3 wPosition       = position;
vec3 wV              = cross(orientation.xyz, wPosition);
wPosition            = offset + wV * 2.0 * orientation.w + cross(orientation.xyz, wV) * 2.0 + wPosition;
vec4 ecPosition      = modelViewMatrix * vec4(wPosition, 1.0);
vUv                  = uv;
gl_Position          = projectionMatrix * ecPosition;

// transform normal vector to world space
vec3 wNPosition      = position + normal;
vec3 wNV             = cross(orientation.xyz, wNPosition);
wNPosition           = offset + wNV * 2.0 * orientation.w + cross(orientation.xyz, wNV) * 2.0 + wNPosition;
vec3 ecNormal        = normalize(mat3(modelViewMatrix) * (wNPosition - wPosition));

// ambient light
vLightFactor         = ambientLightColor;

// diffuse light
vec3  ecToLight      = normalize(directionalLights[0].direction);
float NdotL          = max(0.0, dot(ecNormal, ecToLight));
vLightFactor        += NdotL * directionalLights[0].color; 


For the addition of specular light, the procedure is as follows:

// specular light
vec3  ecReflectLight = reflect( ecFromLight, ecNormal );
vec3  ecViewDir      = normalize(-ecPosition.xyz);
float VdotR          = max(0.0, dot(ecViewDir, ecReflectLight));
float kSpecular      = 4.0 * pow( VdotR, 0.3 * shininess ); // <--- set shininess parameter
vLightFactor        += kSpecular * directionalLights[0].color;


Expanding on the answer: Per fragment lighting

While Gouraud shading handles light calculations in the vertex shader, Phong shading manages light calculations in the fragment shader.
(see also GLSL fixed function fragment program replacement)

Vertex shader:

precision highp float;

uniform mat4 modelViewMatrix;
uniform mat4 projectionMatrix;
uniform mat3 normalMatrix;

attribute vec3 position;
attribute vec3 offset;
attribute vec3 normal;
attribute vec2 uv;
attribute vec4 orientation;

varying vec2 vUv;
varying vec3 ecPosition;
varying vec3 ecNormal;

void main()
{
    vec3 wPosition   = position;
    vec3 wV          = cross(orientation.xyz, wPosition);
    pos              = offset + wV * 2.0 * orientation.w + cross(orientation.xyz, wV) * 2.0 + wPosition;
    vec4 vPos        = modelViewMatrix * vec4(wPosition, 1.0);
    ecPosition       = vPos.xyz;
    vUv              = uv;
    gl_Position      = projectionMatrix * vPos;

    // transform normal vector to world space
    vec3 wNPosition  = position + normal;
    vec3 wNV         = cross(orientation.xyz, wNPosition);
    wNPosition       = offset + wNV * 2.0 * orientation.w + cross(orientation.xyz, wNV) * 2.0 + wNPosition;
    ecNormal         = normalize(mat3(modelViewMatrix) * (wNPosition - wPosition));
}

Fragment shader:

precision highp float;

varying vec2 vUv;
varying vec3 ecPosition;
varying vec3 ecNormal;

uniform sampler2D map;
uniform mat4 modelViewMatrix;

struct DirectionalLight {
    vec3 direction;
    vec3 color;
    int shadow;
    float shadowBias;
    float shadowRadius;
    vec2 shadowMapSize;
};
uniform DirectionalLight directionalLights[ NUM_DIR_LIGHTS ];
uniform vec3 ambientLightColor;

void main()
{
    // ambient light
    float lightFactor = ambientLightColor;

    // diffuse light
    vec3  ecToLight      = normalize(directionalLights[0].direction);
    float NdotL          = max(0.0, dot(ecNormal, ecToLight));
    lightFactor         += NdotL * directionalLights[0].color; 

    // specular light
    vec3  ecReflectLight = reflect( ecFromLight, ecNormal );
    vec3  ecViewDir      = normalize(-ecPosition.xyz);
    float VdotR          = max(0.0, dot(ecViewDir, ecReflectLight));
    float kSpecular      = 4.0 * pow( VdotR, 0.3 * shininess ); // <--- set up shininess parameter
    lightFactor         += kSpecular * directionalLights[0].color;

    gl_FragColor = texture2D(map, vUv) * vec4(vec3(lightFactor), 1.0);
}


Further references:

  • Transform the modelMatrix
  • How does this faking the light work on aerotwist?
  • GLSL fixed function fragment program replacement

Answer №2

My idea is for the fragment shader to be structured as follows:

precision highp float;

varying vec2 vUv;
varying vec3 ecPosition;
varying vec3 ecNormal;

uniform sampler2D map;
uniform mat4 modelViewMatrix;

struct DirectionalLight {
    vec3 direction;
    vec3 color;
    int shadow;
    float shadowBias;
    float shadowRadius;
    vec2 shadowMapSize;
};
uniform DirectionalLight directionalLights[ NUM_DIR_LIGHTS ];
uniform vec3 ambientLightColor;

void main()
{
    // ambient light
    vec3 lightFactor = ambientLightColor;

    // diffuse light
    vec3  ecFromLight    = normalize(directionalLights[0].direction);
    //vec3  ecToLight      = -ecFromLight;
    float NdotL          = max(0.0, dot(ecNormal, ecFromLight));
    lightFactor         += NdotL * directionalLights[0].color; 

    // specular light
    /*
    float shininess = 10.01;
    vec3  ecReflectLight = reflect( ecFromLight, ecNormal );
    vec3  ecViewDir      = normalize(-ecPosition.xyz);
    float VdotR          = max(0.0, dot(ecViewDir, ecReflectLight));
    float kSpecular      = 4.0 * pow( VdotR, 0.3 * shininess ); // <--- set up shininess parameter
    lightFactor         += kSpecular * directionalLights[0].color;
    */

    gl_FragColor = texture2D(map, vUv) * vec4(lightFactor, 1.0);
}

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Error in Material UI: Cannot redeclare identifier 'Switch' - parsing issue

I am trying to incorporate the Material UI Switch component alongside the react-router-dom Switch in a single component. This is how I have included them in my react component: import { BrowserRouter as Router, Switch, Route } from "react-router-dom ...

Having trouble accessing portlet resource URL from JavaScript in Liferay 6.2

I have been working with Liferay Portal 6.2 CE GA3 and I am facing an issue where I need to execute a custom portlet resource method from another portlet's JSP file. Below is the code snippet I am currently using. <a href ="#" onclick="myfunctio ...

Is it possible to fire a Socket.io emit with a scroll event?

Can you trigger an emit on a gesture or scroll event, similar to how it works on a click event? I'm trying to create something like this: https://www.youtube.com/watch?time_continue=38&v=tPxjxS198vE Instead of relying on a click, I would like to ...

Is it possible to link an HTML select element to a changing array in JavaScript?

Let's say I have an empty HTML select element. <select id="options"> </select> Can I link a JavaScript array to this select so that when the array is modified, the select options update accordingly? Alternatively, do I need to resort to ...

``There seems to be an issue with the Express app when trying to access it

I have set up an express app that I want other devices on the same WIFI network to access without requiring internet connectivity. The main computer hosting the app is assigned with a fixed IP address: 192.168.1.60 In my server.js file, I have included t ...

The DELETE function in express.js with MySQL integration is encountering a problem where it is unable to

As I work on setting up my website, the backend utilizes express.js to send queries to a MySQL Database. However, when attempting to delete rows, no action seems to take place. function establishConnection() { return mysql.createConnection({ multipl ...

Access JSON file without considering any custom comments

Is there a way to extract the JSON data from 'file.json' without including any comments? # Comment01 # Comment02 { "name": "MyName" } I have tried using this code snippet: var fs = require('fs'); var obj; fs.readFile('./file. ...

Implementing a dialog box pop-up from a separate React file

My journey with React is just beginning, so forgive me if this question seems basic. I have a delete icon in one of my files, and when it's clicked, I want to display a confirmation dialog box. I found an example on the official Material-UI website: h ...

Ways to run evaluations on 'private' functions within an angular service using Karma and Jasmine

In my Angular application, I have a BracketService that consists of various functions for comparing weights and creating brackets based on weight groups. The service includes functions like compareByWeight, filterWeightGroup, and createBracketsByWeightGrou ...

Enhance Canvas when React State Changes

I am currently working on integrating a canvas into my React project. The main goal is to overlay styled text (with custom font color, style, and size) on an image. I've set up a basic form to input the styling attributes and the desired text. Whenev ...

Sequelize is unable to retrieve a table from the database

I am configuring Sequelize in order to streamline the manipulation of an MSSQL database. My attempt to define a table called 'Stock' has resulted in unexpected behavior when trying to query it. Below is the code snippet I used for defining the t ...

Inserting an HTML element into Handlebars.js during a specific iteration of an each loop

I have a channel.json file containing 7 objects of data which are iterated in hb. I am looking for a way to insert a date between these iterations without modifying the .json file. How can I add an html tag to display after the 3rd iteration within the loo ...

Why is it that the condition of being undefined or not functioning properly in state?

I am currently facing an issue with a piece of code I wrote in React JS. The state variable is not functioning as expected and even after modifying it upon button click, nothing seems to be working. After checking the console, I noticed that the state rema ...

Searching in binary for the number closest to the target float value

Seeking assistance with a float array conundrum! I am trying to identify the closest float value to a target number within the array. My attempt involved utilizing a basic binary search algorithm in JavaScript, but I've hit a roadblock. function bina ...

Clicking on the parent div with a Jquery onclick event does not trigger when the child image is clicked

I'm running into an issue with a simple code containing an image within a div Oddly enough, clicking on the div itself (even with padding) triggers the function, but clicking directly on the image does not. $(".parent0").click(function() { conso ...

Vue.js component unable to validate HTML input patterns

Attempting to create HTML forms with the 'pattern' input attribute has been an interesting challenge for me. When implementing this through Vue.js components, I encountered some strange behavior. Check out this fiddle to see it in action. Vue.co ...

Explore the wonders of Jest and Playwright by heading over to youtube.com and discovering an exciting video!

As part of my job responsibilities, I have been tasked with automating tests for a web application that our team developed. Using Playwright and Jest, I am required to write automation scripts from scratch. To practice utilizing Playwright, I decided to cr ...

What is the best way to include a title and a close button at the top of a menu, before the first item, when a button menu is clicked in material-ui?

One of the challenges I'm facing is adding a title to an icon button menu that contains a checkbox list of menu items. When the user clicks on the icon and opens the dropdown, I want the title to appear at the top of the dropdown along with a close bu ...

What is the best way to have a sound play when the page is loaded?

Is there a way to automatically play a sound when the page loads? <audio src="song.mp3"> Your browser does not support the audio element. </audio> I've attempted it with the following method: <script type="text/javasc ...

Send a PHP object to JavaScript using AJAX

I have a PHP script that successfully uploads a video to the Microsoft Azure service using an API. The API returns a StdObject file, which I then want to send back to JavaScript via AJAX. However, when I try to access the "asset" object in JavaScript, it a ...