Dynamically manipulate the perspective camera by moving and rotating it using a 4x4 matrix

Greetings, esteemed community members,

For the past few days, I have been struggling with an issue related to updating the View Matrix (4x4) of my camera. This update is crucial for positioning objects within an AR-Scene created using three.js.

The custom matrix contains data from Direct Sparse Odometry (DSO), which is retrieved as a string via the WebView's JavaScript interface. This string provides information about my current pose and needs to be converted into a 4x4 matrix. However, when attempting to set:

  1. the matrixWorldInverse or ViewMatrix, no updates occur even after using the common update functions
  2. the projectionMatrix causes the scene to zoom in and out without any actual movement or rotation. It seems to only affect certain characteristics?

Is there another method to adjust the camera's position and orientation within the scene?

Despite trying various update functions provided by three.js like Matrix.set() or fromArray(), there has been no change in the outcome.

Thank you in advance!

   var renderer, scene, camera, box, geometry, material, poseMatrix;

   init();
   animate();
   render();

   function init() {
        // Setting up the renderer
       renderer = new THREE.WebGLRenderer({
           canvas: document.getElementById("mCanvas"),
           alpha: true
       });
       renderer.setClearColor(0xffffff, 0);
       renderer.setSize(window.innerWidth, window.innerHeight);
       document.body.appendChild(renderer.domElement);
       // Initializing the scene
       scene = new THREE.Scene();

       // Creating the camera
       camera = new THREE.PerspectiveCamera(
           75,
           window.innerWidth / window.innerHeight,
           1,
           1000
       );

       scene.add(camera);

       scene.add(new THREE.AxesHelper(100));

       placeBox();
       initPose();

   }

   function render() {
       renderer.render(scene, camera);
   }

   function placeBox()
   {
   // Using a test object which needs to have an absolute  position in world coordinates
       geometry = new THREE.BoxBufferGeometry(3, 5, 3 );
       material = new THREE.MeshLambertMaterial({color: 0xfece46, wireframe: true, wireframeLinewidth: 3.1});
       box = new THREE.Mesh(geometry, material);

       // Positioning the box in global coordinates
       box.position.set(0, 0, -10);
       scene.add(box);
       }

   function animate() {
       setCurrentPose();
       renderer.render(scene, camera);
   }

   function initPose(){
       // Default value provided by DSO
       var mTmpPose = "1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1"; //initial point of DSO
       var mDefaultPose = mTmpPose.split(',');

       // Applying DSO Standard Pose 4x4 Matrix
      camera.matrixAutoUpdate = false;
      camera.matrixWorldInverse.fromArray(mDefaultPose);

       camera.updateWorldMatrix();

     //camera.updateProjectionMatrix(); <- does not work
    //camera.updateMatrix(); <- same issue

   }
   function setCurrentPose(){

   //Acquiring a string with current pose values
       var tmp = Android.getCurrentPose(); // example: 0.8953773, 0.39648485, 0.20272951, 0.0,
                                           //         -0.3792577, 0.44037524, 0.81377715, 0.0,
                                           //           0.23337325,-0.8055243, 0.5446719, 0.0,
                                           // .          -2.5238492, 13.470952, 0.715593, 1.0
       var pose = tmp[0].split(',');

       camera.matrixWorldInverse.fromArray(pose);
   // camera.projectionMatrix.fromArray(pose);

       camera.updateWorldMatrix();
    //   camera.updateProjectionMatrix(); <- does not work
   // camera.updateMatrix(); <- same issue
   }



Answer №1

Augmented Reality lacks the ability to manually set the camera matrix, unlike in Virtual Reality where it is possible. In AR, the user has control over the camera position, and this characteristic is consistent across all major AR frameworks, not just limited to Aframe/Three.js.

Typically, the scene's origin aligns with the initial position of the camera, which simplifies placing content relative to the origin (the camera's starting point).

Once the scene's origin is defined, the camera can move freely through the scene at the user's discretion, preventing any forced direction for the camera to look because of limitations within the rendering engine.

[update]

If there is a need to guide the user's focus towards specific content within the scene, adjusting the placement of the content so that it directly aligns with the user's view can be a workaround.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Toggle between the Angular test/development module and the production module

Currently, I am working on implementing an Angular app. During my e2e tests, I encountered a need to mock some server requests while allowing others to pass through using e2e httpBackend. Vijitta has provided an example of how to utilize the HttpBackend: ...

Fetching data using JSONRequest sample code

I am new to web development and this is my first attempt at using JSON. On the JSON website (http://www.json.org/JSONRequest.html), they recommend using JSONRequest.get for certain tasks. However, I keep running into an error stating "JSONRequest is not ...

React TSX file not recognizing JSON data stored in an HTML data attribute

I am having some trouble with implementing the password toggle component from the Preline UI. Here is how the component looks: "use client" import React, { ChangeEvent, MouseEventHandler, useEffect } from "react"; export default functi ...

The issue of the callback not being triggered by the sinon.js fakeServer when making a $.ajax call was encountered

I am currently working on a jasmine specification that involves testing a jQuery plugin I created: describe "plugins", -> beforeEach -> @server = sinon.fakeServer.create() afterEach -> @server.restore() describe "reviewStatu ...

Monitoring multiple items in OPC UA with Node.js

Utilizing the node-opcua module, my goal is to efficiently monitor multiple opc ua nodes through subscriptions. The desired workflow involves a user selecting nodes in an HTML UI, clicking a Monitor button which then sends the corresponding nodeIds as para ...

In the world of coding, the trio of javascript, $.ajax,

I need help with iterating over an array and assigning a variable using a for loop. Here is the scenario: function Person(name, status){ this.name = name; this.status = status; } var status = []; var array = ["bill","bob","carl","ton"]; function exAj ...

Problem with Marionette AppRouter compatibility when used with Browserify

app.js module.exports = function () { if (!window.__medicineManager) { var Backbone = require('backbone'); var Marionette = require('backbone.marionette'); var MedicineManager = new Marionette.Application(); Medicin ...

What is the correct method for service injection in Angular 8?

I have encountered an issue while trying to inject a service into my main "App" component. The error message is shown in the screenshot below: constructor(private newsApi: NewsApiService) {} Upon importing the service using the code above, I received the ...

Receiving the most recent data in a protractor examination as a text string

My goal is to fetch an input value for a specific operation in protractor. I am attempting to make an ajax request using protractor and need to assign a unique value (referred to as groupCode) to a JSON object that will be sent to the server. Initially, I ...

Django issue: A Tuple or struct_time argument is necessary

Currently, I am developing a code that deals with 2 variables - showdate and viewtype. Both of these variables are transferred via JavaScript using the POST method. viewtype = send an srt showdate = send a date from javascript Within this code snippet, ...

Using FlatShading alongside a texture in three.js to achieve a minimalist flat color effect

Currently, I am working on creating a terrain using three.js and texturing it with a grass texture. I have applied FlatShading to give it a low poly look, but the texture is still visible on each face. I want to achieve a flat color on each face instead of ...

Converting JavaScript QML objects to C++ QT components

I have a QML JavaScript function that generates and returns a component called Item: function createComponent() { var component = Qt.createComponent('MyComponent.qml'); var obj = component.createObject(container, {'x': 0, &apos ...

Utilizing JavaScript to pass parameters to an IFRAME within Dynamics CRM

While trying to access http://localhost:80/testsite in an IFRAME, everything seems to work perfectly. However, once I attempt to pass field values as parameters, nothing happens. Oddly enough, accessing the page directly from a browser with parameters work ...

Java has trouble decoding non-ASCII characters sent through Ajax

When I send an AJAX request using jQuery post() and serialize, the encoding used is UTF-8. For instance, if 'ś' is input as a name value, JavaScript displays name=%C5%9B. Despite my attempts to set form encoding, it has not been successful. < ...

Is it more beneficial to keep a function inside or outside another function if it is only being used within it?

Within the topic at hand, I have a function structured as follows. There are numerous auxiliary functions declared within this function (twice the amount shown in the example) because they are solely utilized by this function. My query is: should I extrac ...

Ways to efficiently solve an underdetermined system using least squares method?

I am currently working on a program in R that involves calculating a large number of least squares solutions (over 10,000, often exceeding 100,000+). Upon profiling the code, I have identified certain bottlenecks in the program. The setup consists of a mat ...

npm fails during the build process due to webpack issues

I find myself lost in trying to pinpoint the right question to ask, but I am encountering a failure while running npm run build. > <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="395a5655564b14564b5e585750435c4b790817091709" ...

"Encountering a hiccup with the Firebase service worker in Messaging and Firebase

I am interested in developing a small web application to explore the capabilities of Firebase Cloud Messaging for web apps. My intention is to utilize Firebase Hosting as the hosting platform for my app. ISSUE: Upon allowing the notification pop-up on my ...

How to unload and remove an object imported with OBJLoader in Three.js

I'm facing a small issue that I can't seem to figure out. I've successfully added an object to my scene using OBJLoader, but now I need to remove it. I've tried clearing the scene.children with code, but it doesn't delete the "flow ...

The following middleware is not functioning properly on a local SSL server

When I run my Nextjs app without SSL using "next dev", the middleware functions as expected without any errors. However, if I attempt to run the app with SSL enabled, an empty middleware function triggers an error. The code for the middleware function (l ...