Using the same bone structure to share skeletons among SkinnedMeshes in Three.JS

Question

Using the same bone structure to share skeletons among SkinnedMeshes in Three.JS

I've been grappling with the challenge of incorporating and removing clothing pieces to an existing skeleton (Clothing meshes and body meshes should share skeleton) but I always seem to encounter strange results.

All the clothing items that I intend to connect to the shared skeleton have been exported from Blender with the same skeleton, they all possess identical bone names and structure, which led me to believe it would be a simple process.

I managed to achieve this effortlessly in Native iOS SceneKit by simply performing something like

clothing.skeleton = body.skeleton

However, when attempting the same in Three.JS, it didn't prove as straightforward.

I experimented with approaches such as

clothingMesh.bind(bodySkeleton,clothingMesh.worldMatrix);

And also tried:

clothingMesh.skeleton = bodySkeleton;
bodySkeleton.update();

Yet, the results were consistently odd. While exploring solutions, I came across some code utilizing "retargeting" functions, though these are typically used when bone names of skeletons do not align, which isn't my case here.

JSFiddle playground link:

I've been tinkering with this on JSFiddle through this link, where you can find the complete code detailing my attempts.

In addition to resolving the binding issues with skeletons, my aim is to replicate Skeleton animations from different sources so they can be downloaded on-the-fly and applied dynamically to various characters. Unfortunately, I'm encountering bizarre outcomes even when the animated skeleton shares identical bone names and structure with the target skeleton for animation.

Here are some online resources and examples related to this topic:

Example 1

Example 2

Example 3

Example 4

https://i.sstatic.net/gPMHf.jpg

The ultimate goal is to seamlessly integrate and remove clothing pieces from a character's body, ensuring all meshes utilize a single shared skeleton (for optimal performance), and simultaneously facilitate the addition and removal of animations from external sources.

javascript three.js 3d skeletal-mesh

Answer 1

Answer №1

Hello, I am encountering the same issue and attempted to address it using the function provided below, but unfortunately, I was unsuccessful in resolving the problem. Have you found a solution yet?

My approach involved utilizing bakeSkeleton(MyCharacterName) after modifying the geometry skinnedMesh of my character. Although the function did not generate any console errors, the character appeared deformed as if exploded.

    function bakeSkeleton ( target ) {
      var v1 = new THREE.Vector3();

      target.traverse( function ( object ) {
            if ( !object.isSkinnedMesh ) return;
            if ( object.geometry.isBufferGeometry !== true ) throw new Error( 'Only BufferGeometry supported.' );

            var positionAttribute = object.geometry.getAttribute( 'position' );
            var normalAttribute = object.geometry.getAttribute( 'normal' );

            for ( var j = 0; j < positionAttribute.count; j ++ ) {
                  object.boneTransform( j, v1 );
                  positionAttribute.setXYZ( j, v1.x, v1.y, v1.z);

                  getBoneNormalTransform.call( object, j, v1 );
                  normalAttribute.setXYZ( j, v1.x, v1.y, v1.z );
            }

            positionAttribute.needsUpdate = true;
            normalAttribute.needsUpdate = true;

            object.skeleton.bones.forEach(bone => bone.rotation.set(0,0,0));
      } );
}

const getBoneNormalTransform = (function () {

      var baseNormal = new THREE.Vector3();

      var skinIndex = new THREE.Vector4();
      var skinWeight = new THREE.Vector4();

      var vector = new THREE.Vector3();
      var matrix = new THREE.Matrix4();
      var matrix3 = new THREE.Matrix3();

      return function ( index, target ) {

            var skeleton = this.skeleton;
            var geometry = this.geometry;

            skinIndex.fromBufferAttribute( geometry.attributes.skinIndex, index );
            skinWeight.fromBufferAttribute( geometry.attributes.skinWeight, index );

            baseNormal.fromBufferAttribute( geometry.attributes.normal, index ).applyNormalMatrix( matrix3.getNormalMatrix(this.bindMatrix) );

            target.set( 0, 0, 0 );

            for ( var i = 0; i < 4; i ++ ) {

                  var weight = skinWeight.getComponent( i );

                  if ( weight !== 0 ) {

                        var boneIndex = skinIndex.getComponent( i );

                        matrix.multiplyMatrices( skeleton.bones[ boneIndex ].matrixWorld, skeleton.boneInverses[ boneIndex ] );

                        target.addScaledVector( vector.copy( baseNormal ).applyNormalMatrix( matrix3.getNormalMatrix(matrix) ), weight );

                  }

            }
            
            matrix3.getNormalMatrix(this.bindMatrixInverse);
            
            return target.applyNormalMatrix( matrix3 );

      }; 
      
}());

Answer 2

Hello, I am encountering the same issue and attempted to address it using the function provided below, but unfortunately, I was unsuccessful in resolving the problem. Have you found a solution yet?

My approach involved utilizing bakeSkeleton(MyCharacterName) after modifying the geometry skinnedMesh of my character. Although the function did not generate any console errors, the character appeared deformed as if exploded.

    function bakeSkeleton ( target ) {
      var v1 = new THREE.Vector3();

      target.traverse( function ( object ) {
            if ( !object.isSkinnedMesh ) return;
            if ( object.geometry.isBufferGeometry !== true ) throw new Error( 'Only BufferGeometry supported.' );

            var positionAttribute = object.geometry.getAttribute( 'position' );
            var normalAttribute = object.geometry.getAttribute( 'normal' );

            for ( var j = 0; j < positionAttribute.count; j ++ ) {
                  object.boneTransform( j, v1 );
                  positionAttribute.setXYZ( j, v1.x, v1.y, v1.z);

                  getBoneNormalTransform.call( object, j, v1 );
                  normalAttribute.setXYZ( j, v1.x, v1.y, v1.z );
            }

            positionAttribute.needsUpdate = true;
            normalAttribute.needsUpdate = true;

            object.skeleton.bones.forEach(bone => bone.rotation.set(0,0,0));
      } );
}

const getBoneNormalTransform = (function () {

      var baseNormal = new THREE.Vector3();

      var skinIndex = new THREE.Vector4();
      var skinWeight = new THREE.Vector4();

      var vector = new THREE.Vector3();
      var matrix = new THREE.Matrix4();
      var matrix3 = new THREE.Matrix3();

      return function ( index, target ) {

            var skeleton = this.skeleton;
            var geometry = this.geometry;

            skinIndex.fromBufferAttribute( geometry.attributes.skinIndex, index );
            skinWeight.fromBufferAttribute( geometry.attributes.skinWeight, index );

            baseNormal.fromBufferAttribute( geometry.attributes.normal, index ).applyNormalMatrix( matrix3.getNormalMatrix(this.bindMatrix) );

            target.set( 0, 0, 0 );

            for ( var i = 0; i < 4; i ++ ) {

                  var weight = skinWeight.getComponent( i );

                  if ( weight !== 0 ) {

                        var boneIndex = skinIndex.getComponent( i );

                        matrix.multiplyMatrices( skeleton.bones[ boneIndex ].matrixWorld, skeleton.boneInverses[ boneIndex ] );

                        target.addScaledVector( vector.copy( baseNormal ).applyNormalMatrix( matrix3.getNormalMatrix(matrix) ), weight );

                  }

            }
            
            matrix3.getNormalMatrix(this.bindMatrixInverse);
            
            return target.applyNormalMatrix( matrix3 );

      }; 
      
}());

Using the same bone structure to share skeletons among SkinnedMeshes in Three.JS

Answer №1

Similar questions

Tips for achieving proper styling and formatting of elements in jQuery UI

Guide to dividing a URL in reactjs/nextjs

Trigger event when user ceases to click

Top recommendation for showcasing a numerical figure with precision to two decimal points

How can I extract the initial values from PHP after receiving the JSON + serialized data in the Ajax response following the first submission?

When developing a multi-page application using React, encountering the error "TypeError: Cannot read property 'pathname' of undefined" may indicate an issue

Submitting the form without utilizing Ajax, but instead sending data directly to a PHP script

Retrieving Blocked Images with Selenium: A Step-by-Step Guide

Leverage AJAX for real-time Django Model updates

What is the best way to effectively adjust the code structure in a Node.JS project?

Converting Typescript objects containing arrays into a single array

Issue with Angular Route Guard - Incorrect redirection to login page

Showcasing Portfolio Work in a User-Friendly Mobile Design

Error: ...[i] has not been defined

Issue with background image resizing when touched on mobile Chrome due to background-size property set to cover

Using functions as properties in React.js

Checking for correct HTML tags using JavaScript

Is Protractor compatible with Internet Explorer 9?

JavaScript seems to have difficulty correctly parsing objects from JSON

What are some strategies for improving search efficiency in arrays containing over 50,000 elements?