Transforming an equirectangular panorama with a 2:1 aspect ratio into a cube map

Currently, I am in the process of developing a basic 3D panorama viewer for a website. To ensure optimal mobile performance, I have decided to utilize the Three.js CSS 3 renderer, which necessitates a cube map composed of six individual images.

To capture the images on my iPhone, I am using Google Photo Sphere or similar applications that generate 2:1 equirectangular panoramas. Subsequently, I resize and convert these images into a cubemap using this site: (Flash).

Ideally, I would like to handle the conversion process myself, either dynamically within Three.js, if feasible, or through Photoshop. While Andrew Hazelden's Photoshop actions appear promising, they do not offer a direct conversion method. Is there a mathematical formula or script available for this task? I aim to avoid utilizing a 3D software such as Blender, if possible.

This may be wishful thinking, but I thought it was worth inquiring about. Although I possess a moderate understanding of JavaScript, I am relatively new to Three.js. Additionally, I am cautious about relying on the WebGL capabilities due to concerns regarding speed and compatibility issues on mobile devices. Furthermore, support for WebGL still appears inconsistent.

Answer №1

For those interested in handling this server-side, there are numerous options available. Utilizing ImageMagick, which offers a variety of command-line tools, could effectively slice images into sections. By incorporating the necessary command into a script, you can conveniently execute it each time a new image is processed.

Determining the specific algorithm employed by the program may be challenging, but one approach involves reverse engineering the process. Creating a grid from Wikimedia Commons and feeding it into the program can provide insight:

https://i.sstatic.net/53O8zRAH.png

This visual representation sheds light on how the box is structured.

Imagine envisioning a sphere with longitude and latitude lines alongside a surrounding cube. Projecting from the sphere's center point results in a distorted grid on the cube.

In terms of mathematics, consider polar coordinates r, θ, ø for the sphere where r=1, 0 < θ < π, and -π/4 < ø < 7π/4:

  • x= r sin θ cos ø
  • y= r sin θ sin ø
  • z= r cos θ

Centrally projecting these onto the cube divides it into four regions based on latitude (-π/4 < ø < π/4, π/4 < ø < 3π/4, 3π/4 < ø < 5π/4, 5π/4 < ø < 7π/4), which then project to either the sides or the top/bottom accordingly.

A Python implementation integrating these concepts:

import sys
from PIL import Image
# Additional code snippets

The function projection maps theta and phi values to cube coordinates, while cubeToImg translates these to output image coordinates.

Testing this algorithm with an image of Buckingham Palace indicates accurate geometric rendering, especially evident in replicating the paving lines.

However, some image artifacts persist due to a lack of one-to-one pixel mapping. To address this, an inverse transformation method is suggested to enhance accuracy.

import sys
from PIL import Image
# Additional code snippets

By applying this inverse transformation technique, more refined outputs can be achieved without compromising pixel correspondence during conversion.

Interested individuals seeking to explore the reverse aspect can refer to this JS Fiddle page.

Answer №2

After reviewing the insightful accepted solution, I felt compelled to share my own C++ implementation, utilizing OpenCV as a base.

For those unfamiliar with OpenCV, think of Mat as representing an image. The process involves constructing two maps that facilitate remapping from an equirectangular image to the corresponding cubemap face, followed by intricate remapping using OpenCV.

If readability is not a concern, the code can be optimized for compactness.

// Declaration of cube faces
// 0 - 3: Side faces in clockwise order
// 4 and 5: Top and bottom faces respectively
float faceTransform[6][2] =
{
    {0, 0},
    {M_PI / 2, 0},
    {M_PI, 0},
    {-M_PI / 2, 0},
    {0, -M_PI / 2},
    {0, M_PI / 2}
};

// Mapping a section of the equirectangular panorama (in) to a cube face
// (face) identified by faceId. Width and height parameters determine the size.
inline void createCubeMapFace(const Mat &in, Mat &face,
        int faceId = 0, const int width = -1,
        const int height = -1) {

    float inWidth = in.cols;
    float inHeight = in.rows;

    // Map allocation
    Mat mapx(height, width, CV_32F);
    Mat mapy(height, width, CV_32F);

    // Triangle properties for sphere center to cube face mapping
    const float an = sin(M_PI / 4);
    const float ak = cos(M_PI / 4);

    const float ftu = faceTransform[faceId][0];
    const float ftv = faceTransform[faceId][1];

    // Processing source coordinates for each target image point
    for(int y = 0; y < height; y++) {
        for(int x = 0; x < width; x++) {

            // Mapping face pixel coords to [-1, 1] on plane
            float nx = (float)y / (float)height - 0.5f;
            float ny = (float)x / (float)width - 0.5f;

            nx *= 2;
            ny *= 2;

            // Scaling [-1, 1] plane coords to [-an, an]
            nx *= an;
            ny *= an;

            float u, v;

            // Plane to sphere surface projection
            if(ftv == 0) {
                // Center faces
                u = atan2(nx, ak);
                v = atan2(ny * cos(u), ak);
                u += ftu;
            } else if(ftv > 0) {
                // Bottom face
                float d = sqrt(nx * nx + ny * ny);
                v = M_PI / 2 - atan2(d, ak);
                u = atan2(ny, nx);
            } else {
                // Top face
                float d = sqrt(nx * nx + ny * ny);
                v = -M_PI / 2 + atan2(d, ak);
                u = atan2(-ny, nx);
            }

            // Angular coord scaling to [-1, 1]
            u = u / (M_PI);
            v = v / (M_PI / 2);

            // Handling out of bounds coords
            while (v < -1) {
                v += 2;
                u += 1;
            }
            while (v > 1) {
                v -= 2;
                u += 1;
            }

            while(u < -1) {
                u += 2;
            }
            while(u > 1) {
                u -= 2;
            }

            // Mapping to in texture space [-1, 1]
            u = u / 2.0f + 0.5f;
            v = v / 2.0f + 0.5f;

            u = u * (inWidth - 1);
            v = v * (inHeight - 1);

            // Saving result for this pixel in map
            mapx.at<float>(x, y) = u;
            mapy.at<float>(x, y) = v;
        }
    }

    // Regenerating output image if required
    if(face.cols != width || face.rows != height ||
        face.type() != in.type()) {
        face = Mat(width, height, in.type());
    }

    // Resampling using OpenCV's remap function
    remap(in, face, mapx, mapy,
         CV_INTER_LINEAR, BORDER_CONSTANT, Scalar(0, 0, 0));
}

Using the input image:

https://i.sstatic.net/KIdfB.jpg

The resulting faces are:

https://i.sstatic.net/KAFjI.jpg

Image credit to Optonaut.

Answer №3

UPDATE 2: It appears that another individual has already created a more advanced web application than my own. Their conversion process is done client-side, eliminating any concerns about uploads and downloads.

If you have a dislike for JavaScript or need to use this on your mobile device, then the web application I've provided below might be suitable.

UPDATE: I've introduced a basic web application where you can submit a panorama and receive back the six skybox images in a ZIP file. You can access it here.

This source code is a refined version of what's included here, and it's open-source on GitHub which you can find here.

The current operation of the application is reliant on a single free-tier Heroku dyno. However, please refrain from utilizing it as an API. If automation is necessary, deploy your own version; a one-click "Deploy to Heroku" option is available.

Original: This is a modified take on Salix Alba's fantastic answer which converts each face individually, producing six distinct images while preserving the original image format.

Aside from most scenarios expecting multiple images, processing one face at a time significantly reduces memory usage when working with large images.

#!/usr/bin/env python
import sys
from PIL import Image
from math import pi, sin, cos, tan, atan2, hypot, floor
from numpy import clip

# Function to transform output image pixel coordinates to x,y,z coords
# (i,j) are pixel coords
# faceIdx represents face number
# faceSize is edge length
def outImgToXYZ(i, j, faceIdx, faceSize):
    a = 2.0 * float(i) / faceSize
    b = 2.0 * float(j) / faceSize

    if faceIdx == 0: 
        (x,y,z) = (-1.0, 1.0 - a, 1.0 - b)
    elif faceIdx == 1: 
        (x,y,z) = (a - 1.0, -1.0, 1.0 - b)
    elif faceIdx == 2: 
        (x,y,z) = (1.0, a - 1.0, 1.0 - b)
    elif faceIdx == 3: 
        (x,y,z) = (1.0 - a, 1.0, 1.0 - b)
    elif faceIdx == 4: 
        (x,y,z) = (b - 1.0, a - 1.0, 1.0)
    elif faceIdx == 5: 
        (x,y,z) = (1.0 - b, a - 1.0, -1.0)

    return (x, y, z)

# Convert using inverse transformation technique
def convertFace(imgIn, imgOut, faceIdx):
    inSize = imgIn.size
    outSize = imgOut.size
    inPix = imgIn.load()
    outPix = imgOut.load()
    faceSize = outSize[0]

    for xOut in xrange(faceSize):
        for yOut in xrange(faceSize):
            (x,y,z) = outImgToXYZ(xOut, yOut, faceIdx, faceSize)
            theta = atan2(y,x) 
            r = hypot(x,y)
            phi = atan2(z,r) 

            uf = 0.5 * inSize[0] * (theta + pi) / pi
            vf = 0.5 * inSize[0] * (pi/2 - phi) / pi

            ui = floor(uf)  
            vi = floor(vf)
            u2 = ui+1       
            v2 = vi+1
            mu = uf-ui      
            nu = vf-vi

            A = inPix[ui % inSize[0], clip(vi, 0, inSize[1]-1)]
            B = inPix[u2 % inSize[0], clip(vi, 0, inSize[1]-1)]
            C = inPix[ui % inSize[0], clip(v2, 0, inSize[1]-1)]
            D = inPix[u2 % inSize[0], clip(v2, 0, inSize[1]-1)]

            (r,g,b) = (
              A[0]*(1-mu)*(1-nu) + B[0]*(mu)*(1-nu) + C[0]*(1-mu)*nu+D[0]*mu*nu,
              A[1]*(1-mu)*(1-nu) + B[1]*(mu)*(1-nu) + C[1]*(1-mu)*nu+D[1]*mu*nu,
              A[2]*(1-mu)*(1-nu) + B[2]*(mu)*(1-nu) + C[2]*(1-mu)*nu+D[2]*mu*nu )

            outPix[xOut, yOut] = (int(round(r)), int(round(g)), int(round(b)))

imgIn = Image.open(sys.argv[1])
inSize = imgIn.size
faceSize = inSize[0] / 4
components = sys.argv[1].rsplit('.', 2)

FACE_NAMES = {
  0: 'back',
  1: 'left',
  2: 'front',
  3: 'right',
  4: 'top',
  5: 'bottom'
}

for face in xrange(6):
  imgOut = Image.new("RGB", (faceSize, faceSize), "black")
  convertFace(imgIn, imgOut, face)
  imgOut.save(components[0] + "_" + FACE_NAMES[face] + "." + components[1])

Answer №4

I have developed a custom script to slice the created cubemap into separate files such as posx.png, negx.png, posy.png, negy.png, posz.png and negz.png. Furthermore, this script is capable of packaging these individual files into a compressed .zip folder.

You can access the source code for this script here: https://github.com/dankex/compv/blob/master/3d-graphics/skybox/cubemap-cut.py

If necessary, you can modify the array in order to specify the image files:

name_map = [ \
 ["", "", "posy", ""],
 ["negz", "negx", "posz", "posx"],
 ["", "", "negy", ""]]

The resulting converted files are available at the following links:

https://i.sstatic.net/FD0Di.png https://i.sstatic.net/YwIpE.png https://i.sstatic.net/I3qMC.png https://i.sstatic.net/4ekPy.png https://i.sstatic.net/REnrs.png https://i.sstatic.net/xXQBK.png

Answer №5

Before jumping into converting images, consider this: unless absolutely necessary due to specific software requirements, it might be best not to do so.

The issue lies in the complexity of mapping between equirectangular and cubic projections. While there is a straightforward connection between the two, translating individual points can result in a rough approximation when considering pixel size, ultimately lowering image quality.

If conversion is unavoidable at runtime, question whether it's truly essential. Opting for a skybox approach by placing an equirectangular texture on a large sphere may suffice, especially since Three.js offers the necessary tools.

For those who require precise projection conversions, NASA offers a comprehensive tool called G.Projector — Global Map Projector that simplifies the process significantly.

An alternative solution worth exploring is 'erect2cubic', a utility that streamlines equirectangular to cubemap conversion through a script designed for hugin integration.

While 'erect2cubit' may not come pre-installed in standard distributions like Ubuntu, following online resources for installation guidance proves worthwhile for achieving optimal results.

Answer №6

Using the power of OpenGL, I devised a solution to tackle this issue and developed a versatile command-line tool around it. This tool excels in processing both images and videos, making it one of the fastest options available.

If you're interested, check out the Convert360 project on GitHub.

For those curious about the technical details, here's the OpenGL Shader - the fragment shader responsible for re-projection.

The usage of this tool is straightforward:

pip install convert360
convert360 -i ~/Pictures/Barcelona/sagrada-familia.jpg -o example.png -s 300 300

With this simple setup, you can achieve results like the following:

https://i.sstatic.net/M9ujV.jpg

Answer №8

Kubi Converter offers the functionality to transform equirectangular images into cube faces efficiently. Designed with speed and versatility in mind, it allows users to select the desired output layout (with six separate images as the default) and customize the resampling method.

Answer №9

Below is a JavaScript implementation based on Benjamin Dobell's original code. The function convertFace requires two ìmageData objects and a face ID (0-6) as parameters.

This code snippet can be executed within a web worker without needing any external dependencies.

// function to convert an image using an inverse transformation
function convertFace(imgIn, imgOut, faceIdx) {
    var inPix = shimImgData(imgIn),
                outPix = shimImgData(imgOut),
                faceSize = imgOut.width,
                pi = Math.PI,
                pi_2 = pi/2;

    for(var xOut=0; xOut<faceSize; xOut++) {
            for(var yOut=0; yOut<faceSize; yOut++) {

            var xyz = outImgToXYZ(xOut, yOut, faceIdx, faceSize);
            var theta = Math.atan2(xyz.y, xyz.x); // range -pi to pi
            var r = Math.hypot(xyz.x, xyz.y);
            var phi = Math.atan2(xyz.z, r); // range -pi/2 to pi/2

            // source image coordinates
            var uf = 0.5 * imgIn.width * (theta + pi) / pi;
            var vf = 0.5 * imgIn.width * (pi_2 - phi) / pi;

            // Use bilinear interpolation between the four surrounding pixels
            var ui = Math.floor(uf);  // coordinate of pixel to bottom left
            var vi = Math.floor(vf);
            var u2 = ui + 1;       // coordinates of pixel to top right
            var v2 = vi + 1;
            var mu = uf - ui;      // fraction of way across pixel
            var nu = vf - vi;

            // Pixel values of four corners
            var A = inPix.getPx(ui % imgIn.width, clip(vi, 0, imgIn.height-1));
            var B = inPix.getPx(u2 % imgIn.width, clip(vi, 0, imgIn.height-1));
            var C = inPix.getPx(ui % imgIn.width, clip(v2, 0, imgIn.height-1));
            var D = inPix.getPx(u2 % imgIn.width, clip(v2, 0, imgIn.height-1));

            // interpolate
            var rgb = {
              r:A[0]*(1-mu)*(1-nu) + B[0]*(mu)*(1-nu) + C[0]*(1-mu)*nu + D[0]*mu*nu,
              g:A[1]*(1-mu)*(1-nu) + B[1]*(mu)*(1-nu) + C[1]*(1-mu)*nu + D[1]*mu*nu,
              b:A[2]*(1-mu)*(1-nu) + B[2]*(mu)*(1-nu) + C[2]*(1-mu)*nu + D[2]*mu*nu
            };

            rgb.r = Math.round(rgb.r);
            rgb.g = Math.round(rgb.g);
            rgb.b = Math.round(rgb.b);

            outPix.setPx(xOut, yOut, rgb);

        } // for(var yOut=0; yOut<faceSize; yOut++) {...}

     } // for(var xOut=0;xOut<faceSize;xOut++) {...}

} // function convertFace(imgIn, imgOut, faceIdx) {...}

// get x, y, z coordinates from out image pixels coordinates
// i,j are pixel coordinates
// faceIdx is face number
// faceSize is edge length
function outImgToXYZ(i, j, faceIdx, faceSize) {
    var a = 2 * i / faceSize,
            b = 2 * j / faceSize;

    switch(faceIdx) {
        case 0: // back
        return({x:-1, y:1-a, z:1-b});
    case 1: // left
        return({x:a-1, y:-1, z:1-b});
    case 2: // front
        return({x: 1, y:a-1, z:1-b});
    case 3: // right
        return({x:1-a, y:1, z:1-b});
    case 4: // top
        return({x:b-1, y:a-1, z:1});
    case 5: // bottom
        return({x:1-b, y:a-1, z:-1});
    }
} // function outImgToXYZ(i, j, faceIdx, faceSize) {...}

// function to ensure value stays within specified range
function clip(val, min, max) {
    return(val<min ? min : (val>max ? max : val));
}

// helper function to manage image data
function shimImgData(imgData) {
    var w = imgData.width*4,
            d = imgData.data;

    return({
        getPx:function(x, y) {
            x = x*4 + y*w;
            return([d[x], d[x+1], d[x+2]]);
        },
        setPx:function(x, y, rgb) {
            x = x*4 + y*w;
            d[x] = rgb.r;
            d[x+1] = rgb.g;
            d[x+2] = rgb.b;
            d[x+3] = 255; // alpha
        }
    });
} // function shimImgData(imgData) {...}

Answer №10

It appears that there may be an error in the transformation code provided for converting a spherical panorama to cube faces. The code seems to use a Cartesian <-> cylindrical transformation instead of a Cartesian <-> spherical transformation.

You can learn more about Spherical Coordinates.

While the current method may work if the calculations are reversed when going from cube faces back to the panorama, using a spherical transformation could yield slightly different results compared to using a cylindrical transformation.

Starting with this equirectangular (spherical panorama):

https://i.sstatic.net/7Lo6s.jpg

If a cylindrical transformation is applied (although its correctness is uncertain), you would get this result:

https://i.sstatic.net/63PbS.jpg

Alternatively, using a spherical transformation would produce this result:

https://i.sstatic.net/1OOoi.jpg

The two results are not identical. However, it seems that my spherical transformation result matches Danke Xie's outcome, although his transformation method is unclear based on the provided link.

This raises questions about whether many contributors in this discussion are utilizing the correct code for the transformation process.

Answer №11

There are many ways to represent environment maps, each offering a unique perspective. Check out this comprehensive overview.

Panoramic Images - An Overview

If you've ever used Photosphere or a similar panorama app, you're probably familiar with the horizontal latitude/longitude representation. You can easily create a textured three.js SphereGeometry using this data. For a step-by-step guide on rendering the earth, take a look at this tutorial.

How to Make the Earth in WebGL? - A Tutorial

Good luck on your projects :).

Answer №12

Inspired by a technique shared by Salix Alba, a straightforward C++ program was created to transform an equirectangular panorama into a cube map:

Photo Panorama Converter

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Incorporate data into the div element

I've been attempting to populate a div after selecting 3 different options from select boxes, but nothing seems to be happening. I have confirmed that the PHP function works as expected when tested manually, but I'm unable to get it to populate a ...

"Implement a feature to recognize and handle various file extensions within an npm

I need help with my npm script where I am trying to include both ts and tsx file extensions. My current code snippet is as follows: "test": "mocha ..... app/test/**/*.spec.{ts,tsx}" Unfortunately, the above syntax is not working correctly. Can someone pl ...

The call stack limit has been exceeded due to the combination of Node, Express, Angular, and Angular-route

Embarking on a new SPA journey, my tech stack includes: Back-end: NodeJS + Express Front-end: Angular + Angular-route. Twitter Bootstrap Underscore Having followed many tutorials with similar stacks, my project files are structured as follows: pac ...

Tips for operating an Express application with React in production mode

My web application has a specific file structure as shown in the image below. https://i.stack.imgur.com/Etzdb.png I'm facing difficulties running my web app with the React Production Build. The following code snippet in my index.js doesn't seem ...

How can I ensure that the images span the entire width of the page in ResponsiveSlides?

I am trying to make my images occupy the entire width of the page just like in this example: However, I have not been able to find a property for this. I'm new to using Jquery but I have a good understanding of Javascript Can someone please help me ...

"Utilize jQuery to target textboxes that are positioned below others, rather than adjacent to them

My goal is to focus on textboxes that are arranged vertically, one below the other. However, when I write a query for pressing "Enter" to act as a tab key, the focus does not shift to the next adjacent textbox. Instead, I want the focus to move to the text ...

Issues with datatables jquery click event failing to work after pagination functionality is used

I have implemented a solution using http://datatables.net/ <button class='btn btn-success activeAccount'>Activate Account</button> When the button is clicked, an ajax call is triggered. Here's the code for the ajax call: $(".a ...

A critical bug has been discovered in Safari and Chrome where HTML5 canvas fails to work in a specific scenario, while Firefox performs perfectly

I am currently using html5 canvas to develop a function-graph software. There seems to be a critical bug that I cannot figure out. For example, when I use the following code: yOFxGraph(plotFUNCinputYofX,-4,5,-4,4,"rgb(66,44,255)",1); everything operates ...

"Exploring the New Features of Bootstrap 5: Button Collapse and

I am struggling to get two buttons working with Bootstrap 5. The first button is supposed to trigger two JavaScript functions to open a bootstrap carousel. The second button is a simple collapse button designed to reveal a long text string. Unfortunately, ...

Utilizing util.format to enclose every string within an array with double quotation marks

Currently .. var utility = require("utility"); var carsInput = 'toyota,chevrolet'; var cars = carsInput.split(','); var queryString = utility.format('Cars: [%s]', cars); console.log(queryString); // Cars: [toyota,chevrolet] ...

When attempting to use dynamic imports with `react-icons`, NextJS will import all necessary components and dependencies

My current task involves incorporating an Icon from the react-icons package into my project. However, when I attempt to do so using an import statement, the resulting bundle size looks like this: Route (pages) Size First Lo ...

Struggling with a minor glitch in a straightforward coin toss program written in JavaScript

I'm a newcomer to JavaScript programming and I am struggling with understanding how the execution flow is managed. I attempted to integrate an animation from "Animation.css" into a coin toss program, but encountered an issue. When trying to use JavaSc ...

What is the best way to retrieve information from the YouTube API v3 for multiple YouTube channels using just one request?

I am currently developing an endpoint to retrieve data for videos from two specific YouTube channels. I am using the fetch() method along with URL requests and would like to continue using this approach. How can I construct a single URL that will return da ...

Managing the uploading of files when a new property is included directly from the JavaScript File Object

When processing files using the POST method, I am able to access the default information such as name, type, size, tmp_name, and error through PHP. However, the file being sent contains additional data that I need PHP to extract. https://i.sstatic.net/bGj ...

"Adding a class with jQuery on a selected tab and removing it when another tab is clicked

I have 3 different tabs named Monthly, Bi-monthly, and Weekly. The user can set one of these as their default payroll period, causing that tab to become active. I was able to achieve this functionality by utilizing the following code within the <script& ...

Div with Sticky Header and Scrolling Ability

I am working on a project that involves creating a scrollable div with "title" elements inside. I want the title element of each section to stick to the top of the div as I scroll, similar to how a menu sticks to the top of a webpage. An example of this ca ...

Stop automatic image sliding by hovering in jQuery

How can I make the image slider pause on hover? Here is the code that I have been using: $(".col-md-8").hover(function() { clearTimeout(timer); }); It seems like this should work, but for some reason it's not. Can anyone ...

Download attachments using Django from a web browser

Trying to send an image as an attachment. This is the code I am using: resp = FileResponse(open(fullImgPath, "rb")) resp['Content-Disposition'] = 'attachment; filename="{}"'.format(os.path. basename(fullImgPath)) resp["Content-Type"]= ...

Disappearing text editor content in React Js

I have created a basic accordion feature with a text editor inside each accordion. Accordion.js <div className="wrapper"> {accordionData.map((item, index) => ( <Accordion> <Heading> <div styl ...

What is the best way to display a Facebook-like pop-up on a website

I am looking to implement a Facebook like box pop-up on my website for visitors. While I know how to show the pop-up, I am trying to determine how I can detect if someone has already liked my page in order to prevent the Facebook pop-up from appearing agai ...