I am attempting to generate a displacement map effect to smoothly transition between three images using the ThreeJs library, but I have hit a roadblock

My journey with ThreeJS started a few months back, leading me into the fascinating world of webgl. I delved into numerous courses on YouTube, but most of them were centered around 3D compositions.

While I have developed a basic understanding of how things work, there are still areas where I struggle to grasp concepts fully.

Here's where I need help:

I'm currently trying to figure out how to extract data, specifically pixel positions and values, from my displacement texture. My goal is to be able to swap values between my images for creating transitions.

I attempted to follow an example provided here, but I am using webpack and Three JS. My intention is not to simply copy code but to truly understand the process, which seems to lack information or comprehension on my part.

Researching online didn't yield much information, but as far as I could gather, my images are converted to textures in my code. This means I should have access to pixel values, convert them to UV coordinates, and then manipulate them to achieve the desired effect.

If you're interested in diving deeper, check out my repository.

Let me now show you what I've managed to accomplish so far with loading the images:

import * as THREE from 'three';
import { imgTexture1, imgTexture2, imgTexture3 } from './imgLoaderManager.js';

import vertexShader from './glsl/vertex.glsl';
import fragmentShader from './glsl/fragment.glsl';

//
const texturesArr = [imgTexture1, imgTexture2, imgTexture3];



const planeGeometry = new THREE.PlaneBufferGeometry(3, 3);

const planeMaterial = new THREE.ShaderMaterial({
  vertexShader: vertexShader,
  fragmentShader: fragmentShader,
  side: THREE.DoubleSide,
});
planeMaterial.uniforms.uTime = { value: 0 };
planeMaterial.uniforms.uTexture1 = { value: imgTexture1 };
planeMaterial.uniforms.uTexture2 = { value: imgTexture2 };
planeMaterial.uniforms.uTexture3 = { value: imgTexture3 };
planeMaterial.uniforms.uTexture = {
  value: new THREE.TextureLoader().load(texturesArr),
};

console.log(planeMaterial);
console.log(planeGeometry.attributes);
console.log(imgTexture1);
console.log(imgTexture2);
console.log(imgTexture3);

export const plane = new THREE.Mesh(planeGeometry, planeMaterial);

Take a look at the Fragment and Vertex Code below:

varying vec2 vUv;
varying vec3 vPosition;
uniform sampler2D uTexture1;
uniform sampler2D uTexture2;
uniform sampler2D uTexture3;

void main() {

    vec3 img1 = texture2D(uTexture1, vUv).xyz;
    vec3 img2 = texture2D(uTexture2, vUv).xyz;
    vec3 img3 = texture2D(uTexture3, vUv).xyz;

    gl_FragColor = vec4(img1 * img2 * img3, 1);
    //gl_FragColor = vec4(vPosition, 1);

}
//////////////////////////////////////////////

varying vec2 vUv;

varying vec3 vPosition;

void main() {
    vPosition = position;
    vUv = uv;

    gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);

}

Answer №1

I am having trouble grasping the desired effect/result you are aiming for... If it resembles the one in your provided example, you can achieve it by utilizing the mix function and incorporating value interpolation. Instead of focusing on colors, textures could be used as well.

In addition to this, there are coordinates and pixel values displayed.

<script type="importmap">
  {
    "imports": {
      "three": "https://unpkg.com/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="90e4f8e2f5f5d0a0bea1a5a9bea0">[email protected]</a>/build/three.module.js",
      "three/addons/": "https://unpkg.com/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="57233f25323217677966626e7967">[email protected]</a>/examples/jsm/"
    }
  }
</script>


<canvas class="webglHH"> </canvas>

<script type="module">
import * as THREE from "three";
import GUI from 'https://cdn.jsdelivr.net/npm/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="78141114551f0d1138485649415649">[email protected]</a>/+esm';
const scene = new THREE.Scene();

const sizes = {
  width: window.innerWidth,
  height: window.innerHeight,
};

const canvas = document.querySelector("canvas.webglHH");

const camera = new THREE.PerspectiveCamera(
  75,
  window.innerWidth / window.innerHeight,
  0.1,
  1000
);
camera.position.z = 3;

const renderer = new THREE.WebGLRenderer({
  canvas: canvas,
  antialias: true,
});
renderer.setSize(sizes.width, sizes.height);
renderer.render(scene, camera);

const vertexShader = `
varying vec2 vUv;
void main() {
    vUv = uv;
    gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`;

const fragmentShader = `
varying vec2 vUv;
uniform float iProgress;

void main() {
    vec3 color1 = vec3(0.556, 0.216, 0.757);
    vec3 color2 = vec3(1.0, 0.941, 0.133);

    // Interpolate between two colors based on iProgress / or if you want textures.
    vec3 mixedColor = mix(color1, color2, iProgress);
    gl_FragColor = vec4(mixedColor, 1);
}
`;

const material = new THREE.ShaderMaterial({
  uniforms: {
    iProgress: { value: 0.0 },
  },
  vertexShader,
  fragmentShader,
});

const geometry = new THREE.PlaneGeometry(3, 3);
const plane = new THREE.Mesh(geometry, material);
scene.add(plane);

const gui = new GUI()
gui.add(material.uniforms.iProgress, 'value', 0.0, 1.0).name('Progress');

function animate() {
  requestAnimationFrame(animate);
  renderer.render(scene, camera);
}
animate();
canvas.addEventListener('mousemove', (event) => {
  const mouseX = (event.clientX / sizes.width) * 2 - 1;
  const mouseY = -(event.clientY / sizes.height) * 2 + 1;
  const raycaster = new THREE.Raycaster();
  raycaster.setFromCamera({ x: mouseX, y: mouseY }, camera);
  const intersects = raycaster.intersectObject(plane);
  if (intersects.length > 0) {
    const pixelX = Math.floor(intersects[0].uv.x * sizes.width);
    const pixelY = Math.floor((1 - intersects[0].uv.y) * sizes.height);
    console.log(`Pixel Position: (${pixelX}, ${pixelY})`);
    console.log('Color Values:', intersects[0].object.material.uniforms.iProgress.value);
  }
});
function resize() {
  sizes.width = window.innerWidth;
  sizes.height = window.innerHeight;
  camera.aspect = sizes.width / sizes.height;
  camera.updateProjectionMatrix();
  renderer.setSize(sizes.width, sizes.height);
}
window.addEventListener('resize', resize);


</script>

Answer №2

I have discovered a solution for this particular problem. For those who are interested in understanding the mechanics behind it.

void main(){

///The displacement image is represented by vec4 dispImg
vec4 dispImg = texture2D(uTexture3, vUv.yx);

///Use this formula to retrieve values from the displacement image and distort the second image.
///iProgress is a uniform value ranging from 0 to 10 that needs to be created in your shader material.
vec2 displacedUv = vec2(vUv.x, vUv.y + iProgress * dispImg.r );

vec4 img1 = texture2D(uTexture1, vUv);

///Replace vUv with the newly created displacedUv
vec4 img2 = texture2D(uTexture2, displacedUv);


gl_FragColor = img2;
}

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Tips for troubleshooting JavaScript errors in Internet Explorer 7 & 6 using Dreamweaver CS3

Is there a way to track and debug JavaScript errors in Internet Explorer 7 & 6 on web pages using Dreamweaver CS3? I am experienced with debugging in Visual Studio, but unsure how to do it in Dreamweaver CS3. ...

Creating POST requests using the FormData API

I am trying to pass the username and form_data object to a PHP file using http.post. When I only pass form_data, my picture upload works fine. However, I also want to pass some other information like the username. Can someone please help me with how to p ...

Triggering a jQuery ajax request upon pressing a key on the keyboard

When a user inputs text into an <input> element, I'm using ajax to send a request to the server. Here's how it looks: $('#my-input').bind("input", function(event){ // ajax request code }); My concern is that too many requests ...

Utilize Object literal manipulation to include properties in a specific sequence

Working on a tool to generate Nassi-Shneiderman diagrams online, where each diagram is represented as an object literal with unlimited possible children. As I aim to add a sequence into the while loop following the first sequence, I encounter the challeng ...

AngularJS Table Checkbox Selection Helper Functions

I am feeling completely lost and could really use some assistance. I have been looking at different examples but nothing seems to be helping me out. I have created a dynamic table with checkboxes. Whenever a row is selected, its id gets added to an array ...

Tips for effectively utilizing JavaScript regex to precisely match a string before triggering the submit button

I have a form in Angular with a text input field that requires the entry of lowercase letters separated by commas, like this: c, d, e, g, a, f etc... Currently, the submit button activates as soon as any part of the input matches the required format, allo ...

Encountering a 404 Not Found error while attempting to reach a Node/Express endpoint from the client side

I've developed a Node.js/Express REST API to be utilized by a frontend React application for an inventory management system intended for a garage sale. When attempting to add a new product, I'm trying to access the POST route http://localhost:300 ...

Finding the Row Number of an HTML Table by Clicking on the Table - Utilizing JQuery

Seeking assistance to tackle this issue. I currently have an HTML table with a clickable event that triggers a dialog box. My goal is to also retrieve the index of the row that was clicked, but the current output is not what I anticipated. //script for the ...

Efficient Loading of Angular Material Grid Lists

I managed to design a grid page with the help of angular Material Grid list. Is there a method to combine the Material "Virtual Repeat" feature (Lazy Loading on scroll) with the grid list? This would allow for loading more grids as the user scrolls. Any a ...

Resetting the Angular provider configuration whenever the service is injected into a different location

Trying to wrap my head around a rather complex issue here. I have a service set up as a provider in order to configure it. Initially, this service has an empty array of APIs which can be dynamically added to by various configuration blocks. When adding API ...

Error: Node requires the use of 'import' to load ES modules

Here is the code snippet that I am currently working with: import { toString } from 'nlcst-to-string'; import { retext } from 'retext'; import retextPos from 'retext-pos'; import retextKeywords from 'retext-keywords' ...

Add and condense sub-row

I am struggling to make the child row collapse and append when clicking on the parent row. It seems like my code is not working properly. Can anyone offer some guidance or assistance? This is the code for the parent row: $j(document).ready(function(){ $ ...

What are the steps to produce a high-quality WebP image without losing any data?

I recently started experimenting with the WebP image format in Chrome by utilizing the canvas element. After researching on MDN, I discovered that the toDataURL function has a second parameter that determines the quality of the resulting image. My goal is ...

Embrace positive practices when working with nodejs

What is the best way to enhance the readability of my code by replacing the extensive use of if and else if statements when detecting different file extensions? Currently, I have over 30 else ifs in my actual code which makes it look cluttered. // Retrie ...

How can I dynamically apply an active class when clicking on a group of buttons?

Iterating through the buttons array, I retrieve three buttons. My goal is to determine which button has been clicked and apply an active class to it. import React from "react"; import { useState } from "react"; import style from "./years.module.scss"; co ...

Adding data from PHP into a designated div element

update: After some trial and error, I managed to find a solution using jQuery and append() methods. From my PHP code, I created an array containing all the necessary information that I wanted to insert into the tab divs. I then encoded this array into JSON ...

How can I adjust the border width of outlined buttons in material-ui?

Currently, I am in the process of modifying a user interface that utilizes Material-UI components. The task at hand involves increasing the thickness of the borders on certain outlined buttons. Is there a method through component props, themes, or styles t ...

What is the best way to structure files within the css and js folders after running the build command in Vue-cli?

Vue-cli typically generates files in the following structure: - dist -- demo.html -- style.css -- file.commom.js -- file.commom.js.map -- file.umd.js -- file.umd.js.map -- file.umd.min.js -- file.umd.min.js.map However, I prefer to organize them this way: ...

How can I set a header to be automatically included in every "render" response in Node.js/Express?

One of the challenges I face involves implementing "controllers" in my code: app.get('/',function(req,res){ var stuff = { 'title': 'blah' }; res.render('mytemplate',stuff); }); In particular, I'm worki ...

Guide on how to store HTML content into a variable within next.config.js

I kickstart my project using a standard nextron setup (Next.js + electron.js) from https://github.com/saltyshiomix/nextron In a specific scenario, I am required to fetch the contents of a file named template.html and store it in a variable (String). This ...