Creating a three-dimensional representation by projecting onto the surface of a sphere in ThreeJS

Question

Creating a three-dimensional representation by projecting onto the surface of a sphere in ThreeJS

3D animation is a realm filled with numerous terms and concepts that are unfamiliar to me. I am particularly confused about the significance of "UV" in 3D rendering and the tools used for mapping pixels onto a mesh.

I have an image captured by a 360-degree camera, which I intend to use as the texture for a sphere. However, I want the center of this image to represent the top of the sphere, with the circular radius translating into an arc along the surface from top to bottom.

Starting with code snippets borrowed directly from Three.JS documentation:

var video = document.getElementById( "texture-video" );

var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.1, 1000 );

var renderer = new THREE.WebGLRenderer();
renderer.setSize( window.innerWidth, window.innerHeight );
document.body.appendChild( renderer.domElement );

var texture = new THREE.VideoTexture( video );
texture.minFilter = THREE.LinearFilter;
texture.magFilter = THREE.LinearFilter;
texture.format = THREE.RGBFormat;

var material = new THREE.MeshBasicMaterial( { map: texture } );

var geometry = new THREE.SphereGeometry(0.5, 100, 100);
var mesh = new THREE.Mesh( geometry, material );

scene.add( mesh );

camera.position.z = 1

function animate()
{
    mesh.rotation.y += 0.01;
    requestAnimationFrame( animate );
    renderer.render( scene, camera );
}

animate();

The output result exhibits some issues:

~~The texture orientation needs correction (rotated 90 degrees)~~
~~The ground seems distorted, potentially linked to the rotation issue?~~
Update: Further examination reveals that the image aligns properly with the sphere when considering its top and bottom poles; however, the edges create a distorted effect resembling sideways ground textures
The current setup places the image on the outside of the sphere, but my goal is to project it onto the inside while positioning the camera within the sphere

Placing the camera inside results in a solid black display. As per Three.JS guidelines, lighting should not be required with a MeshBasicMaterial. The likely problem could stem from outward-facing normals on the sphere's faces, suggesting a need for reversal akin to skybox implementations that I'm unsure how to execute.

Research indicates that adjusting the "UV"s may solve this issue, although the method and implication remain unclear...

javascript 3d three.js textures mesh

Answer 1

Answer №1

Example in Action

I took inspiration from @manthrax's CodeSandbox.io project and made some modifications of my own:

https://codesandbox.io/s/4w1njkrv9

The Breakthrough

After dedicating a full day to unraveling the concept of UV mapping for better comprehension, I delved into applying trigonometry to map points from a sphere onto my stereographic image. The process boiled down to the following steps:

Using arccosine of the Y coordinate to ascertain the magnitude of a polar coordinate on the stereographic image
Employing the arctangent of the X and Z coordinates to determine the angle of the polar coordinate on the stereographic image
Utilizing x = Rcos(theta), y = Rsin(theta) to calculate the rectangular coordinates on the stereographic image

If time allows, I might create a visual aid using Illustrator or similar software to illustrate the mathematical principles, though it essentially boils down to standard trigonometry.

Further exploration led me to address an issue stemming from the 240-degree vertical viewing angle of my camera, causing distortions in the image especially near ground level. By subtracting the vertical viewing angle from 360 and dividing by two, a specific angle relative to the vertical axis emerged, marking a threshold where no mapping should transpire. Aligning the sphere along the Y-axis facilitated correlating this angle with a particular Y-coordinate - delineating areas with data above and below accordingly.

Determine the "minimum Y value"
For all sphere points:
- If a point surpasses the minimum Y value, linearly scale it so that the initial occurrence is considered as "0," while the top still signifies "1" for mapping purposes
- If a point falls beneath the minimum Y value, exclude it from the mapping

Unusual Observations

Curiously, my code implementation resulted in an upside-down image rendering. The root cause could be attributed to inaccuracies in my trigonometric calculations or potential misconceptions regarding UV maps. Regardless, rectifying this anomaly was straightforward - simply rotating the sphere by 180 degrees post-mapping.

In addressing uncertainty around how to "exclude" values in the UV map, I opted to assign all points below the minimum Y value to the image corner (displayed as black).

With a 240-degree viewing angle, the vacant space at the bottom of the sphere devoid of image data appeared notably prominent on my screen. Displeased with this visual artifact, I substituted 270 for the vertical FOV, albeit resulting in minor distortions near ground level compared to when employing 360.

The Implementation

Presented here is the code segment responsible for updating the UV mappings:

// Insert the vertical FOV for the camera here
var vFov = 270; // = 240;

var material = new THREE.MeshBasicMaterial( { map: texture, side: THREE.BackSide } );
var geometry = new THREE.SphereGeometry(0.5, 200, 200);

function updateUVs()
{
    var maxY = Math.cos(Math.PI * (360 - vFov) / 180 / 2);
    var faceVertexUvs = geometry.faceVertexUvs[0];
    // The sphere consists of many FACES
    for ( var i = 0; i < faceVertexUvs.length; i++ )
    {
        // For each face...
        var uvs = faceVertexUvs[i];
        var face = geometry.faces[i];
        // A face is a triangle (three vertices)
        for ( var j = 0; j < 3; j ++ )
        {
            // For each vertex...
            // x, y, and z refer to the point on the sphere in 3d space where this vertex resides
            var x = face.vertexNormals[j].x;
            var y = face.vertexNormals[j].y;
            var z = face.vertexNormals[j].z;

            // Because our stereograph goes from 0 to 1 but our vertical field of view cuts off our Y early
            var scaledY = (((y + 1) / (maxY + 1)) * 2) - 1;

            // uvs[j].x, uvs[j].y refer to a point on the 2d texture
            if (y < maxY)
            {
                var radius = Math.acos(1 - ((scaledY / 2) + 0.5)) / Math.PI;
                var angle = Math.atan2(x, z);

                uvs[j].x = (radius * Math.cos(angle)) + 0.5;
                uvs[j].y = (radius * Math.sin(angle)) + 0.5;
            } else {
                uvs[j].x = 0;
                uvs[j].y = 0;
            }
        }
    }
    // Correcting the inverted UV mapping by adjusting the math and rotating the sphere by 180 degrees
    geometry.rotateZ(Math.PI);
    geometry.uvsNeedUpdate = true;
}
updateUVs();

var mesh = new THREE.Mesh( geometry, material );

The Outcome

Integrating this mesh into a scene yields flawless results:

https://i.sstatic.net/MKwlN.png https://i.sstatic.net/UqwGj.png https://i.sstatic.net/JnnYz.png https://i.sstatic.net/aLNj1.png

An Enigma Yet Unsolved

Notably, around the "hole" at the sphere's base, an intriguing multi-colored ring emerges resembling a mirrored reflection of the sky. The presence of this phenomenon remains puzzling to me, raising questions about its origin and significance. Any insights or clarifications shared in the comments would be greatly appreciated!

Answer 2

Example in Action

I took inspiration from @manthrax's CodeSandbox.io project and made some modifications of my own:

https://codesandbox.io/s/4w1njkrv9

The Breakthrough

After dedicating a full day to unraveling the concept of UV mapping for better comprehension, I delved into applying trigonometry to map points from a sphere onto my stereographic image. The process boiled down to the following steps:

Using arccosine of the Y coordinate to ascertain the magnitude of a polar coordinate on the stereographic image
Employing the arctangent of the X and Z coordinates to determine the angle of the polar coordinate on the stereographic image
Utilizing x = Rcos(theta), y = Rsin(theta) to calculate the rectangular coordinates on the stereographic image

If time allows, I might create a visual aid using Illustrator or similar software to illustrate the mathematical principles, though it essentially boils down to standard trigonometry.

Further exploration led me to address an issue stemming from the 240-degree vertical viewing angle of my camera, causing distortions in the image especially near ground level. By subtracting the vertical viewing angle from 360 and dividing by two, a specific angle relative to the vertical axis emerged, marking a threshold where no mapping should transpire. Aligning the sphere along the Y-axis facilitated correlating this angle with a particular Y-coordinate - delineating areas with data above and below accordingly.

Determine the "minimum Y value"
For all sphere points:
- If a point surpasses the minimum Y value, linearly scale it so that the initial occurrence is considered as "0," while the top still signifies "1" for mapping purposes
- If a point falls beneath the minimum Y value, exclude it from the mapping

Unusual Observations

Curiously, my code implementation resulted in an upside-down image rendering. The root cause could be attributed to inaccuracies in my trigonometric calculations or potential misconceptions regarding UV maps. Regardless, rectifying this anomaly was straightforward - simply rotating the sphere by 180 degrees post-mapping.

In addressing uncertainty around how to "exclude" values in the UV map, I opted to assign all points below the minimum Y value to the image corner (displayed as black).

With a 240-degree viewing angle, the vacant space at the bottom of the sphere devoid of image data appeared notably prominent on my screen. Displeased with this visual artifact, I substituted 270 for the vertical FOV, albeit resulting in minor distortions near ground level compared to when employing 360.

The Implementation

Presented here is the code segment responsible for updating the UV mappings:

// Insert the vertical FOV for the camera here
var vFov = 270; // = 240;

var material = new THREE.MeshBasicMaterial( { map: texture, side: THREE.BackSide } );
var geometry = new THREE.SphereGeometry(0.5, 200, 200);

function updateUVs()
{
    var maxY = Math.cos(Math.PI * (360 - vFov) / 180 / 2);
    var faceVertexUvs = geometry.faceVertexUvs[0];
    // The sphere consists of many FACES
    for ( var i = 0; i < faceVertexUvs.length; i++ )
    {
        // For each face...
        var uvs = faceVertexUvs[i];
        var face = geometry.faces[i];
        // A face is a triangle (three vertices)
        for ( var j = 0; j < 3; j ++ )
        {
            // For each vertex...
            // x, y, and z refer to the point on the sphere in 3d space where this vertex resides
            var x = face.vertexNormals[j].x;
            var y = face.vertexNormals[j].y;
            var z = face.vertexNormals[j].z;

            // Because our stereograph goes from 0 to 1 but our vertical field of view cuts off our Y early
            var scaledY = (((y + 1) / (maxY + 1)) * 2) - 1;

            // uvs[j].x, uvs[j].y refer to a point on the 2d texture
            if (y < maxY)
            {
                var radius = Math.acos(1 - ((scaledY / 2) + 0.5)) / Math.PI;
                var angle = Math.atan2(x, z);

                uvs[j].x = (radius * Math.cos(angle)) + 0.5;
                uvs[j].y = (radius * Math.sin(angle)) + 0.5;
            } else {
                uvs[j].x = 0;
                uvs[j].y = 0;
            }
        }
    }
    // Correcting the inverted UV mapping by adjusting the math and rotating the sphere by 180 degrees
    geometry.rotateZ(Math.PI);
    geometry.uvsNeedUpdate = true;
}
updateUVs();

var mesh = new THREE.Mesh( geometry, material );

The Outcome

Integrating this mesh into a scene yields flawless results:

https://i.sstatic.net/MKwlN.png https://i.sstatic.net/UqwGj.png https://i.sstatic.net/JnnYz.png https://i.sstatic.net/aLNj1.png

An Enigma Yet Unsolved

Notably, around the "hole" at the sphere's base, an intriguing multi-colored ring emerges resembling a mirrored reflection of the sky. The presence of this phenomenon remains puzzling to me, raising questions about its origin and significance. Any insights or clarifications shared in the comments would be greatly appreciated!

Answer 3

Answer №2

After spending about 10 minutes experimenting with a polar unwrapping of the uv's, here is the closest result I achieved.

If you want to improve the mapping, feel free to tweak the polarUnwrap function...

Check out the code here

https://i.sstatic.net/2QJqr.jpg

You can swap out TextureLoader().loadTexture() with

//assuming you have created a HTML video element with id="video"
var video = document.getElementById( 'video' );

var texture = new THREE.VideoTexture( video );
texture.minFilter = THREE.LinearFilter;
texture.magFilter = THREE.LinearFilter;
texture.format = THREE.RGBFormat;

to incorporate your video into the design...

For more details, visit:

Three.js VideoTexture documentation

You might also find this resource helpful:

Guide on displaying dual fisheye videos with Three.js

Answer 4

After spending about 10 minutes experimenting with a polar unwrapping of the uv's, here is the closest result I achieved.

If you want to improve the mapping, feel free to tweak the polarUnwrap function...

Check out the code here

https://i.sstatic.net/2QJqr.jpg

You can swap out TextureLoader().loadTexture() with

//assuming you have created a HTML video element with id="video"
var video = document.getElementById( 'video' );

var texture = new THREE.VideoTexture( video );
texture.minFilter = THREE.LinearFilter;
texture.magFilter = THREE.LinearFilter;
texture.format = THREE.RGBFormat;

to incorporate your video into the design...

For more details, visit:

Three.js VideoTexture documentation

You might also find this resource helpful:

Guide on displaying dual fisheye videos with Three.js

Answer 5

Answer №3

Modifying the UVs to make the stereographic projected image fit can be quite challenging. The UVs of a sphere are specifically designed to align with textures using equirectangular projection.

If you need to convert the image from stereographic to equirectangular, tools like PTGui or Hugin could be helpful. Alternatively, Photoshop can be used by applying Filter > Distort > Polar Coordinates > polar to rectangular.

https://i.sstatic.net/aJqct.jpg An example of equirectangular projection created with Photoshop, resized to a 2:1 aspect ratio (not required for texture mapping)

To ensure the texture is inside the sphere (or normals flipped), set the material to THREE.BackSide.

var material = new THREE.MeshBasicMaterial( { map: texture, side: THREE.BackSide } );

You may also need to horizontally flip the texture, as discussed in How to flip a Three.js texture horizontally.

Answer 6

Modifying the UVs to make the stereographic projected image fit can be quite challenging. The UVs of a sphere are specifically designed to align with textures using equirectangular projection.

If you need to convert the image from stereographic to equirectangular, tools like PTGui or Hugin could be helpful. Alternatively, Photoshop can be used by applying Filter > Distort > Polar Coordinates > polar to rectangular.

https://i.sstatic.net/aJqct.jpg An example of equirectangular projection created with Photoshop, resized to a 2:1 aspect ratio (not required for texture mapping)

To ensure the texture is inside the sphere (or normals flipped), set the material to THREE.BackSide.

var material = new THREE.MeshBasicMaterial( { map: texture, side: THREE.BackSide } );

You may also need to horizontally flip the texture, as discussed in How to flip a Three.js texture horizontally.

Creating a three-dimensional representation by projecting onto the surface of a sphere in ThreeJS

Answer №1

Example in Action

The Breakthrough

Unusual Observations

The Implementation

The Outcome

An Enigma Yet Unsolved

Answer №2

Answer №3

Similar questions

Selenium Tips: Ensuring RemoteDriver Remains Connected to the Active Browser Tab

Ionic: setInterval/setTimer not functioning after 5 minutes in the background

Ensure the md-sidenav remains open by default

Automate populating input fields with data

Utilizing HTML documents instead of images in Owl Carousel 2

What is the best way to create a self-referencing <div> in HTML?

Tips for organizing and sorting date data in a JSON datatable

AWS Lambda applies quotes to create the event input

JavaScript Regular Expression Assistance Domain Snatcher

Incorporating a new data series into your candlestick chart with Highstock

Error with Laravel and Vue template integration

Is it possible to use the .focus() event on an entire form in JQuery? If so, how

JSON function invocation is failing to load

An error is triggered by serializing a TinyBox POST form into an Array

"Successfully uploading files with Ajax on Postman, however encountering issues when attempting to do so in

Address Book on Rails

Working with npm objects across multiple files

Ways to showcase angular scope data within a placeholder while avoiding the use of angular expressions

Unable to access res.name due to subscription in Angular

Sending forms through the .NET platform