Utilizing WebXR controllers for interactive button actions in three.js

Exploring the realm of mapping out controls for the Oculus Quest and other devices using three.js and webXR is an intriguing challenge I am currently facing. The current code I have developed successfully allows me to manipulate the controller, attach cylinders to each control, and alter the color of the cylinders by using the trigger controls. However, I am stuck when it comes to implementing axis controls for the joystick, grip, and other buttons as there is a lack of documentation on this specific topic. It seems like the solution could be as simple as knowing which event to call, but without knowledge of all available events, progress is hindered.

For reference, I followed a tutorial which served as the foundation for my current work. You can find the tutorial at this link: https://github.com/as-ideas/webvr-with-threejs

It's important to note that the existing code is functional and meets expectations, but I am eager to enhance it further and explore additional possibilities.

function createController(controllerID, videoinput) { 
//RENDER CONTROLLER AS YELLOW TUBE
        const controller = renderer.vr.getController(controllerID);
        const cylinderGeometry = new CylinderGeometry(0.025, 0.025, 1, 32);
        const cylinderMaterial = new MeshPhongMaterial({ color: 0xffff00 });
        const cylinder = new Mesh(cylinderGeometry, cylinderMaterial);
        cylinder.geometry.translate(0, 0.5, 0);
        cylinder.rotateX(-0.25 * Math.PI);
        controller.add(cylinder);
        cameraFixture.add(controller);
        //TRIGGER
        controller.addEventListener('selectstart', () => {
            if (controllerID === 0) {
                cylinderMaterial.color.set('pink')
            } else {
                cylinderMaterial.color.set('orange');
                videoinput.play()
            }
        });
        controller.addEventListener('selectend', () => {
            cylinderMaterial.color.set(0xffff00);
            videoinput.pause();
            console.log('I pressed play');
        });
    }

Answer №1

Starting from version 0.119 of three.js, the integrated 'events' like buttons, trackpads, haptics, and thumbsticks of a touch controller do not have full support. Only select and squeeze events are currently available. three.js functions on a 'just working' approach regardless of the input device, focusing on managing events common to all input devices, such as 'select'.

Despite the limitations in three.js, there is the option to directly poll controller data, allowing for more direct access.


Touch controllers operate similarly to 'gamepad' controls, providing immediate feedback on their state. By polling the gamepad, we can capture current button values, keep track of their status, and generate events for button presses, trackpad movements, and thumbstick changes.

While in a webXR session, access to instant controller data is crucial.

const session = renderer.xr.getSession();
let handedness = "unknown";

if (session) {
        for (const source of session.inputSources) {
            if (source && source.handedness) {
                handedness = source.handedness; //determine left or right controllers
            }
            if (!source.gamepad) continue;
            const controller = renderer.xr.getController(i++);
            const old = prevGamePads.get(source);
            const data = {
                handedness: handedness,
                buttons: source.gamepad.buttons.map((b) => b.value),
                axes: source.gamepad.axes.slice(0)
            };
            //process data to create 'events'

Haptic feedback is executed through a promise (note that not all browsers currently support webXR haptic feedback, but some like Oculus Browser and Firefox Reality on Quest do). When available, the haptic feedback is triggered via:

var didPulse = sourceXR.gamepad.hapticActuators[0].pulse(0.8, 100);
//80% intensity for 100ms
//subsequent promises cancel any previous promise still underway

To demonstrate this functionality, I enhanced the threejs.org/examples/#webXR_vr_dragging example by incorporating a camera to a 'dolly' for movement through touch controller thumbsticks within a webXR session. Various haptic feedback is triggered for events like raycasting onto an object or thumbstick axis alterations.

With each frame, controller data is polled and processed accordingly. It's necessary to store data between frames to detect changes, generate events, and filter out data anomalies (e.g., false 0's and up to 20% random drift from 0 in thumbstick axis values on certain controllers). Additionally, the current heading and orientation of the webXR camera are accessed each frame through:

    let xrCamera = renderer.xr.getCamera(camera);
    xrCamera.getWorldDirection(cameraVector);
    //heading vector for webXR camera stored in cameraVector

Explore the example on CodePen here: codepen.io/jason-buchheim/pen/zYqYGXM

With the 'ENTER VR' button for a debug view exposed (debug view) here: cdpn.io/jason-buchheim/debug/zYqYGXM


The complete code with modifications from the original threejs example is highlighted within comment blocks.

//// Modifications from the webxr_vr_dragging example https://threejs.org/examples/#webxr_vr_dragging
import * as THREE from "https://cdn.jsdelivr.net/npm/three@0.116.1/build/three.module.min.js";
import { OrbitControls } from "https://cdn.jsdelivr.net/npm/three@0.116.1/examples/jsm/controls/OrbitControls.min.js";
import { VRButton } from "https://cdn.jsdelivr.net/npm/three@0.116.1/examples/jsm/webxr/VRButton.min.js";
import { XRControllerModelFactory } from "https://cdn.jsdelivr.net/npm/three@0.116.1/examples/jsm/webxr/XRControllerModelFactory.min.js";

[...]

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Harvesting the local image file

Currently, I have implemented a form that allows users to upload images and crop them. The process flow has been established as follows: 1. User uploads the image 2. Server processes the image and sends it back to the browser 3. User crops the image a ...

Employing a for loop to verify the existence of a JSON key

Currently, I am attempting to loop through an EJS JSON object and verify the existence of values. If a value does exist, I want to add the corresponding URL to an array further down. Upon loading the page, I am encountering an issue where I am informed th ...

`Creating a functional list.js filter``

I'm having trouble making the List.js filter function work properly. The documentation for List.js is not very helpful for someone new to development. Below is the code I am using: HTML: <div class="col-sm-12" id="lessons"> <input class= ...

How can I utilize ng-repeat in AngularJS to iterate through an array of objects containing nested arrays within a field?

Here is the structure of an array I am working with: 0: {ID: null, name: "test", city: "Austin", UserColors: [{color: "blue"},{hobby:"beach"} ... ]} }... I am currently attempting to ng-repeat over this initial array in my code. However, when I tr ...

What is the process for including echo HTML field in the WooCommerce order view?

I have successfully added HTML input fields within functions.php. However, these echoed fields are not displaying in WooCommerce orders after placing an order. I am looking for a way to include the values and labels of these fields in the orders view wit ...

Shuffle letters in a word when hovered over, then unscramble them back to their original order

I have noticed a fascinating effect on several websites. To give you an idea, here are two websites that showcase this effect: Locomotive and TUX. Locomotive is particularly interesting to look at as the desired effect can be seen immediately when hovering ...

An observer is handed to me when I receive an array as a parameter

How can I use an array as a parameter instead of just receiving an observer? When trying to utilize the array, all I get is an observer. The data appears correctly when using console.log in the function that fetches information from the DB. Despite attem ...

Modify the color scheme of an HTML webpage

Looking to enhance my HTML page with a new feature. The page is responsive and currently using Bootstrap CSS and JS. My goal is to allow users to change the color of the page dynamically by selecting a background or text color. Check out this example link ...

Tips on incorporating a changing variable into a JavaScript Shader

I have successfully created a primitive sphere using THREE.SphereGeometry and applied a displacement shader to give it a bumpy effect. My goal now is to animate the scale of the bumps based on input volume from the microphone. Despite my efforts, I am stru ...

Executing function after completion of ajax call

I have 2 buttons and 3 links that trigger an ajax request. I want to display an alert only when the request initiated by one of the buttons is completed, but not when a link is clicked. Below is my code: HTML: <button id="ajax1">Ajax 1</button&g ...

Can AJAX be considered a backend tool for retrieving data from servers?

I'm curious to find out if ajax is primarily used as a backend technology for retrieving data or if it's mainly considered a frontend tool. My attempts to research this on Google have not yielded a definitive answer. ...

Ways to retrieve information from a different website's URL?

Having a bit of an issue here. I'm currently browsing through some reports on webpage #1 () and I have a specific requirement to extract the object named "data" from webpage #2 (). However, the code I've used seems to fetch the entire webpage ins ...

Tips for retrieving a value returned by a Google Maps Geocoder

geocoder.geocode( { 'address': full_address}, function(results, status) { lat = results[0].geometry.location.lat(); lng = results[0].geometry.location.lng(); alert(lat); // displays the latitude value correctly }); alert(lat); // does ...

Express Node fails to launch

I recently decided to try my hand at Express and learn more about it. However, I encountered an issue while trying to start a server. Initially, I attempted to access 127.0.0.1:8080 through Express but nothing seemed to be happening. Although I copied most ...

Resolving issues with jQuery input values

I have been working on a form that allows users to choose an amount or specify another amount. When the user clicks on "other amount," it changes the displayed amount to $5. My goal is to make it so that when the user specifies an amount other than 5, it u ...

Using Jquery to set values for css properties

I've recently started using jquery and I have a task related to a drop-down menu that I'm struggling with: if (scrnwidth <= 761) { if (display was block) { //Defaultly testi has display:none property. testi = m ...

Can a single button click be shared across multiple forms?

The main concept involves a grid where when a user double-clicks on a row, a modal window (Bootstrap panel) opens with a panel-body section for editing the data and a panel-footer containing a btn-group for actions like "Save", "Cancel", or "Close". This s ...

What is the best way to determine the dimensions of a KonvaJs Stage in order to correctly pass them as the height/width parameters for the toImage function

Currently, I am using KonvaJs version 3.2.4 to work with the toImage function of the Stage Class. It seems that by default, toImage() only captures an image of the visible stage area. This is why there is a need to provide starting coordinates, height, and ...

Tips for incorporating a User ID object into an Ajax request

Currently, I am delving into the world of web development and tackling a client-server project that involves allowing users to authenticate themselves and store their blood pressure readings. While the authentication part is functioning properly, I am enco ...

Accessing state in Vuex modules is crucial for managing and manipulating data effectively

Feeling a bit lost here... I'm utilizing Vuex in my project and have a module called common stored in the file common.js: const initState = { fruits: [] } export default { state: initState, mutations: { SET_FRUITS(state, fruits) { cons ...