Converting a Three.js Vector3 to 2D screen coordinates while considering a rotated scene

I need help positioning a div so that it is always at the highest point of an object, slightly offset to the left and up from the vertex. You can check out the example here. If the jsfiddle link is not working, refer to the working with no scene rotation snippet below. It works remarkably well.

However, for a work project, I have to rotate the scene itself. This seems to affect the conversion to 2D screen coordinates. You can see the issue here. Also, take a look at the not working with scene rotation snippet below. As you can observe, the label doesn't update correctly in the y-direction when rotating the scene but does so horizontally. To achieve the desired effect, I need to rotate the entire scene.

If someone could provide guidance or create an example on any of the html/js/css snippet sites (like jsfiddle), they would be a lifesaver!

  • side-note: I attempted, albeit unsuccessfully, to adjust the x- and y-coordinates of each vertex by multiplying them with sin(sceneRotation) and cos(sceneRotation), but it only made matters worse.

  • side-note 2: I also spent an hour rotating each object in the scene, but since the objects are stored in THREE.Groups, it had the same undesirable effect.

working with no scene rotation

click run snippet below

// JS code for working snippet...
// CSS code for working snippet...
// HTML content for working snippet...

not working with scene rotation

click run snippet below

// JS code for non-working snippet...
// CSS code for non-working snippet...
// HTML content for non-working snippet...

Answer №1

When rendering an image, each mesh within the scene undergoes a series of transformations involving the model matrix, view matrix, and projection matrix.

  • Model matrix:
    The model matrix specifies the position, orientation, and size of a mesh in the scene. It transforms the vertex positions of the mesh from object space to world space.

  • View matrix:
    The view matrix determines the viewing direction and position of the scene. It transforms from world space to view (eye) space. In the viewport coordinate system, the X-axis points left, the Y-axis points up, and the Z-axis extends out of the view (In a right-hand system, the Z-Axis is the cross product of the X-Axis and the Y-Axis).

  • Projection matrix:
    The projection matrix maps 3D points in the scene to 2D points on the viewport. It transforms from view space to clip space, where the coordinates are then transformed into normalized device coordinates (NDC) in the range (-1, -1, -1) to (1, 1, 1) by dividing by the w component of the clip coordinates.

To determine the position of a point in the geometry on the viewport, all these transformations must be carried out along with conversion from normalized device coordinates (NDC) to window coordinates (pixels).

The transformation using the view matrix and projection matrix can be achieved through the project function:

vector.project(camera);

To convert from normalized device coordinates (NDC) to window coordinates (pixels), you can use the following method:

vector.x = Math.round( (   vector.x + 1 ) * w / 2 );
vector.y = Math.round( ( - vector.y + 1 ) * h / 2 );

Don't forget about the transformation with the model matrix, which can be executed as follows:

var modelMat = cube.matrixWorld;
vector.applyMatrix4(modelMat);

Modify the `getScreenPosition` function in a similar manner: function getScreenPosition(position) { var vector = new THREE.Vector3( position.x, position.y, position.z ); // model to world var modelMat = cube.matrixWorld; vector.applyMatrix4(modelMat);
    // world to view and view to NDC
    vector.project(camera);

    // NDC to pixel
    vector.x = Math.round( (   vector.x + 1 ) * w / 2 );
    vector.y = Math.round( ( - vector.y + 1 ) * h / 2 );
   
    return vector;
}

Explore the code snippet below:

/* cannot be seen here; thanks jsfiddle */
    
    /* Camera Variables */
    
    /* Three.js Elements */
    

    render();

    function render() {
       
    }

    function getScreenPosition(position, object) {
       

        
        return vector;
    }

    function updateLabel() {
        
        
    }

    function handleMouseDown(ev) {
       
    }

    function handleTouchStart(ev) {
        

    }

    function mouseOrTouchDown(downX, downY, touch) {


    }

    function handleMouseMove(ev) {
       

    }

    function handleTouchMove(ev) {

    }

    function mouseOrTouchMove(x, y) {
        
    }

    function updateCamera() {
        
    }
body {
   
}

#label {
   
}
<div id="label">label</div>
<script src="https://cdn.jsdelivr.net/npm/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="b2c6dac0d7d7f2829c838384">[email protected]</a>/build/three.js"></script>
<script src="https://cdn.jsdelivr.net/npm/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="493d213b2c2c09796778787f">[email protected]</a>/examples/js/controls/OrbitControls.js"></script>

Answer №2

If anyone is still curious about this issue, I have a different approach to the solution. The current solution may be a bit excessive as it essentially recreates basic orbit control functionalities from scratch. While this may be beneficial for learning purposes, if you require additional features of the orbit controls, things can get complicated.

The problem isn't necessarily that the orbit controls "mess up the conversion to 2D screen coordinates". It's more so that they don't automatically update the camera matrices. These will only update if set to do so on render, but this can lead to inconsistencies in what is displayed.

My suggestion is to manually update the matrix yourself at the appropriate times when changes occur. Use the "change" event listener provided by orbitControls to trigger this update and then perform any necessary calculations. This way, you can ensure everything stays synchronized, including updates to labels or other elements.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Tips on adding custom fonts for text geometry in three.js

Currently, I am delving into the realm of three.js to create text geometry, although my expertise in JavaScript is fairly limited. Initially, I attempted to utilize my custom JSON font, which resulted in the same error encountered when using fonts from th ...

What is the solution for resolving the Next.js 14 next/link routing 404 error page problem?

After performing a fresh installation of Next.js on Windows using an elevated shell with the command npx create-next-app@latest, I encountered an issue. In my /src/app/components/Header.tsx file, the code looks like this: import React from 'react&apos ...

How to invoke a function from a different ng-app in AngularJS

I have 2 ng-app block on the same page. One is for listing items and the other one is for inserting them. I am trying to call the listing function after I finish inserting, but so far I haven't been successful in doing so. I have researched how to cal ...

Is there a way to iterate through this?

I have data that I need to loop over in JavaScript. Since it is not an array, the map function is not working. Can someone help me loop over it or convert it into an array so that I can iterate through it? { "7/15/2021": { "date": ...

Iterating through JSON array using Jquery and making POST/GET requests

I can't seem to get my code to loop over the entire JSON array, despite trying multiple tricks and making various changes. Using AJAX to Retrieve Data function FetchData() { $.ajax({ type: "POST", url: "htt ...

Where is the location to access the real-time camera position coordinates in Potree/Three.js?

Currently working on an app using React that incorporates the Potree viewer to showcase large points cloud files. My goal is to provide users with real-time feedback on camera coordinates (X, Y, Z) within the viewer and utilize this information for optimiz ...

Exploring MongoDB's Aggregation Framework: Finding the Mean

Is there a way to use the Aggregation Framework in MongoDB to calculate the average price for a specific Model within a given date range? Model var PriceSchema = new Schema({ price: { type: Number, required: true }, date: { ...

The username index is not defined in the file path C:xampphtdocsAppX1signin.php on line 6

Experiencing some challenges with a php script I recently created. As a beginner in php, I understand that my code may not be optimal. These error messages are displayed when the form is submitted: Notice: Undefined index: username in C:\xampp&bsol ...

Scrolling seamlessly within a container that is fixed in position

For hours, I've been attempting to incorporate smooth scrolling into my project with no success. The issue lies in the container where the anchor tags are located. If it's set to fixed position or absolute, none of my attempts seem to work. In ...

Guide on triggering an open modal when clicking a link using jQuery

Creating a modal popup has always been my goal. After designing the CSS to style it, I found myself stuck as I lack knowledge in JQuery or JavaScript to code the functionality that animates and opens the modal upon clicking a button or link. Despite search ...

Save an automatically generated number into a variable and use it to reference an image file for display. This process can be accomplished using JavaScript

I'm having trouble getting my images to display randomly on a page. The images are named 0 - 9.png and I am using a pre-made function for random number generation. However, when I try to call on this function later down the page, nothing appears. It ...

Adding variables and appending using jQuery

When using the append method to add HTML content dynamically to a div, I encountered an issue while trying to insert an iframe. Specifically, I needed to pass a variable's value to this frame for it to function properly. Here is the code snippet: va ...

The AJAX response consistently returns a 405 status code

I am experiencing an issue with an AJAX post request. $.ajax({ type: "POST", contentType: "application/json", url: "/rating/save", data: JSON.stringify(rating), dataType: "json", mimeType: "application/json" ...

Tips for passing a value or argument to the map function in JavaScript

Can someone guide me on how to pass a variable in the map function using JavaScript? const obj = ["Singapore", "Malaysia"] const pr = ["SG", "MY"] // need to pass value to map function render(){ obj.map((val, index) => { return html` <p ...

When performing the operation number.tofixed in Typescript, it will always return a string value instead of a double datatype as expected from parseFloat

let value = 100 value.toFixed(2) -> "100.00" parseFloat(value.toFixed(2)) -> 100 I am encountering an unexpected result with the double type when input is 100.36, but not with 100.00. Framework: loopback4 ...

When a button is clicked, a Chrome Extension will display a new window

I have developed a Chrome extension that includes a button. When the user clicks on this button, it launches a pre-created HTML page. Here is the code for the button: var btn = document.createElement("BUTTON"); var t = document.createTextNode("Clic ...

Custom Shader in Three.js for Post Processing with a Clear Background

While working with this shader, I encountered an issue where it renders a black background instead of a transparent one. I understand that this is controlled within the fragmentShader, but I am struggling to modify it and unsure if it's even feasible. ...

No action detected following the import into Redux form

This is my very first project in React. After completing a training course on Udemy, I have reached the following code but am stuck trying to integrate the action into the container (Is Container correct?). Here is the code I have put together from variou ...

Safari causing issues with textures in Three.js CanvasRenderer

I am currently working on a Three.js based sample project that will serve as the foundation for a larger project. The project involves a plane with a texture which orbits a center point. The plane is attached to the center point and then the center point i ...

Update the state when a button is clicked and send a request using Axios

Currently in my front end (using react): import '../styles/TourPage.css'; import React, { Component } from 'react'; import axios from 'axios' class TourPage extends Component { constructor(props) { super(p ...