Accessing the current position of skeleton bones in Three.js while an animation is playing

Question

Accessing the current position of skeleton bones in Three.js while an animation is playing

I currently have a Three.js project with an animated scene and I am trying to obtain the positions of skeleton bones at various points in time.

If I visit, for instance, and inspect the Three.js scene:

  const modelViewer = document.querySelector("model-viewer");
  const scene = modelViewer[Object.getOwnPropertySymbols(modelViewer)[14]];

I can then retrieve the position of the LowerArmR bone which is being animated in this example:

scene.children[0].children[0].children[0].children[0].children[1].children[0].skeleton.bones.find(b => b.name == "LowerArmR").position

However, regardless of the animation progress, the retrieved position remains constant:

{x: 1.86264503820865e-10, y: 0.00575238140299916, z: -1.02445485428149e-9}

Is there a way to access real-time skeletal positions during animation?

Upon applying this method to a humanoid avatar undergoing animation, I only see the default T-pose rather than the actual bone positions as they change over time: https://i.sstatic.net/MILfq.png

According to this source:

Skeletal Animations are managed on the GPU, making it challenging to access this data directly through JavaScript code in real-time.

Although potential workarounds may exist, none are widely known or established.

Therefore, I am seeking information on any workaround or standard approach to accomplishing this task.

javascript three.js model-viewer

Answer 1

Answer №1

When it comes to skeletal animation (also known as "skinning"), the individual mesh vertices undergo transformations on the GPU¹. This approach is taken because there are typically more vertices than bones, making it more efficient to update them on the GPU rather than the CPU.

In three.js, however, the bone transformations themselves are calculated on the CPU. It's important to understand some key distinctions here:

The .position attribute of a bone, which is inherited from THREE.Object3D's .position attribute, represents a local position relative to its parent and ancestors.
Skeletal animation primarily involves rotating bones rather than translating them. For instance, rotating the shoulder bone will both translate and rotate its descendants, such as the arm.

To determine the world position rather than the local position of a specific bone, you can use the object.getWorldPosition method provided by the Object3D parent class:

const position = new THREE.Vector3();

bone.getWorldPosition( position );

It's worth noting that this method incurs a slight performance overhead as it updates the bone's world matrix by traversing its ancestors before computing the current world position. If you need the positions of multiple bones or other properties like world rotation and scale, it's recommended to ensure all global transforms are up to date once using scene.updateMatrixWorld() if necessary, and then decompose the object's .matrixWorld:

const position = new THREE.Vector3();

position.setFromMatrixPosition( bone.matrixWorld );

¹ In case you do require computing the position of a specific vertex on the CPU, the SkinnedMesh class in three.js offers a helpful method for this: .boneTransform().

Answer 2

When it comes to skeletal animation (also known as "skinning"), the individual mesh vertices undergo transformations on the GPU¹. This approach is taken because there are typically more vertices than bones, making it more efficient to update them on the GPU rather than the CPU.

In three.js, however, the bone transformations themselves are calculated on the CPU. It's important to understand some key distinctions here:

The .position attribute of a bone, which is inherited from THREE.Object3D's .position attribute, represents a local position relative to its parent and ancestors.
Skeletal animation primarily involves rotating bones rather than translating them. For instance, rotating the shoulder bone will both translate and rotate its descendants, such as the arm.

To determine the world position rather than the local position of a specific bone, you can use the object.getWorldPosition method provided by the Object3D parent class:

const position = new THREE.Vector3();

bone.getWorldPosition( position );

It's worth noting that this method incurs a slight performance overhead as it updates the bone's world matrix by traversing its ancestors before computing the current world position. If you need the positions of multiple bones or other properties like world rotation and scale, it's recommended to ensure all global transforms are up to date once using scene.updateMatrixWorld() if necessary, and then decompose the object's .matrixWorld:

const position = new THREE.Vector3();

position.setFromMatrixPosition( bone.matrixWorld );

¹ In case you do require computing the position of a specific vertex on the CPU, the SkinnedMesh class in three.js offers a helpful method for this: .boneTransform().

Accessing the current position of skeleton bones in Three.js while an animation is playing

Is there a way to access real-time skeletal positions during animation?

Answer №1

Similar questions

Is React-Apollo encountering a 404 network error?

How do I change the 'CSS Theme' of my website?

Shadows in Three.js are malfunctioning

I keep encountering errors with TypeGuard

Issues with the functionality of the shopping cart are causing a disruption

Suspend features for moving between pages in history

I am encountering issues with getting jquery.multiple.select.js to properly validate and send the checked boxes to PHP

Updating objects in Angular 8 while excluding the current index: a guide

The 'id' key is causing issues in React when used for mapping

Issue encountered with the carousel plugin while transitioning to a different page (utilizing Durandal)

Is there a way to store my collection data in a variable and access it externally from the file?

Attempting to convert the code from react documentation into class components within a react-create-app, but encountering unexpected behavior

Use Javascript to display or conceal events on a fullcalendar interface

What is the purpose of adding "/dist" to the library import statement?

Transform JSON data into a JavaScript object

Assist with JavaScript Programming

Aggregate values through an onclick event using a get request and iterate through each using

Employing the Confirm() function within the onbeforeunload() event

Executing RegisterStartupScript repeatedly in C#

Site performance stalls when a trailing slash is present in the URL while using Express routing