Utilize three.js to animate a virtual reality humanoid model mapped to skeleton coordinates

As someone who is new to three.js and animation, I find myself struggling with concepts like rotation angles, VRM interactions, and humanoid animations. Despite my confusion, I will do my best to articulate my question below.

I have a series of frames that contain coordinates (xyz) for human joints such as left foot, right foot, and left shoulder. My goal is to animate a 3D model based on these coordinates.

Based on what I've observed from others, it seems they modify the z-axis rotation of the human bone node to map human actions onto the animated model. This leads me to the following questions:

  1. In cases where I have 3D coordinates instead of a 2D video input, how do I calculate the rotation value? The documentation suggests Euler angles, but how does one derive Euler angles from something like the coordinate of the left shoulder?

    i. Which body parts should I calculate these angles for? Does it make sense to define rotation for specific body parts like the Left Shoulder or nose?

    ii. Is it feasible to directly set the xyz coordinate values as the position attribute of the humanoid bone node? For example:

    currentVrm.humanoid.getBoneNode(THREE.VRMSchema.HumanoidBoneName.Neck).position = (10, -2.5, 1)
    . Would this accurately represent the movement seen in the provided coordinates?

  2. How are humanoid bone nodes represented? The documentation mentions them as Object3D objects, but can they simply be vectors? Considering that vectors cannot rotate like cylinders, how should angle calculations be approached for each humanoid bone node, such as leftShoulder = (3, 11.2, -8.72)? Do I need to compute angles for each axis and apply them to the rotation attributes of the bone node?

Answer №1

While I may not be an expert on three.js, I do have some insights to share about VRM. The structure of bones in VRM follows a hierarchy pattern, starting from root-hips-spine-chest-neck, and so on. For instance, branching out from the chest, you'll find left/right shoulder - upper/lower arm - hand, while from the hips, there are legs and feet.

Each bone is characterized by 3 position coordinates (X,Y,Z) and a quaternion (X,Y,Z,W). This means that determining the position of a specific bone in the world coordinate system involves traversing through the entire hierarchy (beginning from the root), applying quaternions, and adding positions along the way.

Let's take the example of finding the position of the 'neck bone':

  1. Start with the 'root' coordinates and apply the 'root' quaternion
  2. Use the 'hips' position, apply the 'hips' quaternion, and add the resulting coordinates to the 'root' coordinates
  3. Moving up, use the 'spine' coordinates, apply the 'spine' quaternion, and add the resulting coordinates to the 'hips' coordinates
  4. Continue this process for 'chest' and 'neck', applying their respective quaternions and combining positions

Furthermore, when "applying quaternion," it's essential to consider the multiplication of previous quaternions; thus, the resulting quaternion for the 'neck' would involve a combination of all preceding quaternions: qneck_res = qneck * qchest * qspine * qhips * qroot

If necessary, there are methods available for converting between Euler angles and quaternions as well.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the method for including an echo inside a URL call?

I am attempting to achieve the following: <?php include_once ("includes/'<?php echo $mymodule1; ?>'.php"); ?> Previously, I tried using: <?php include_once ("includes/'.echo $mymodule1.'.php"); ?> Unfortunately, it ...

Error: The function 'stepUp' was invoked on an object lacking the HTMLInputElement interface during an AJAX request

It's always frustrating to have to ask a question that has already been asked, but I'm having trouble finding a solution that works for me. My issue involves retrieving the value of an input and sending it via AJAX. $("#cell_number").on("change" ...

Retrieving a portion of a file using Jquery ajax

I am struggling with extracting specific content from loaded data using the following simple sample code: $.ajax({ url: 'demo2.htm', success: function(loadeddata){ $("#loaded_data").after(loadeddata); alert('success'); }, ...

Verify whether the element within an iFrame contains any content

After conducting extensive research, I have been unable to find a satisfactory answer to my question. Therefore, I am reaching out to see if anyone has the knowledge I seek. The Goal: I aim to check the contents within an iFrame and determine whether it ...

Is it possible to use the AngularJS function with the <div> HTML element?

I am facing an issue with displaying a modal when clicking on a <div> element. The function assigned to show the modal by changing the CSS property 'display' from 'none' to 'block' works fine when attached to a button, b ...

Leveraging multi-processing with NextJS and NodeJS

My tech stack includes NextJS for frontend development and NodeJS for backend development. I'm looking to implement child processes in my project. Could someone guide me on setting up NodeJS for this purpose? Additionally, I'm interested in utili ...

What is the reason behind receiving the error message "`Foo` only represents a type, but is being treated as a value here" when using `instanceof` in TypeScript?

As the creator of this code interface Foo { abcdef: number; } let x: Foo | string; if (x instanceof Foo) { // ... } Upon running this code, I encountered an error in TypeScript: 'Foo' is recognized only as a type and cannot be used a ...

What is the method to create a resizable table column border rather than resizing the bottom corner border?

At the moment, we are able to resize the table border when we drag the corner. However, my goal is to enable resizing on the right side of the full border. https://i.sstatic.net/yFI24.png Below is the CSS code for resizing: th{ resize: horizontal; ove ...

Traverse through the JSON data until a certain condition is satisfied, then proceed to tally the number

Looking to parse a json file containing user data and points. The goal is to loop through the data until the _id matches the userId, then determine the position of that user in the list based on the number of objects. The json file provided below already ...

Looking to dynamically track an event in Firestore?

Is it possible to implement a method in Node.js that allows me to listen for changes on a node, similar to the following path? organizations/{org_slug}/projects/{pro_slug}/calculations/{calc_slug} I have successfully done this in a Firebase Cloud Functio ...

I'm having trouble figuring out why this React Router setup is not functioning properly. Can anyone provide any insights

As I delve into react routing practice, I've put together a geography-based web app. Starting off, I configured the router paths: import { StrictMode } from "react"; import { createRoot } from "react-dom/client"; import { BrowserRo ...

Do not include the "dist" folder when exporting sub-modules in TypeScript

I've developed a basic module called modA: modA/ - package.json - dist/ - index.js - db.js - stuff.js My goal is to use the submodules "db" and "stuff" like this: import * as db from modA/db -- how can I achieve this? My package.json has main: ...

Attempting to incorporate a factory within a restricted function

Just starting out with AngularJS and I have a question. Can I use a factory code (logger) inside a private function like the example below? I'm still working on understanding angular concepts. Thanks for your help: (function () { 'use strict ...

Can you please provide me with the correct JQuery syntax to verify if an element already exists within an array? Below is an uncomplicated example to illustrate

I decided to create a JS Fiddle to experiment with various techniques. The main query presented is: "What is the correct method for displaying only 4 unique items in the dropdown list, while avoiding duplicates of the same item?" My if statement implemen ...

Adjust background color on click using JavaScript

Could someone provide me with the JavaScript code to change the background color of a page from blue to green when a button is clicked? I have seen this feature on many websites but haven't been able to write the code myself. I am specifically lo ...

What is the best way to fetch values from individual buttons using PHP?

<form action="posts-ayarlar.php" method="POST" id="demo-form2" data-parsley-validate class="form-horizontal form-label-left"> <table class="table table-striped table-bordered" ...

Tips for adjusting the button color in Material UI by utilizing the ":active" pseudo-class

In the project I am currently working on, I am incorporating Material UI. One of the tasks I need to complete is changing the active state of a button. I have implemented both hover and active states from Material UI in my code: const useStyles = makeStyle ...

jQuery not functioning properly when attempting to add values from two input boxes within multiple input fields

I am facing an issue with a form on my website that contains input fields as shown below. I am trying to add two input boxes to calculate the values of the third input box, but it is only working correctly for the first set and not for the rest. How can ...

What is preventing me from retrieving these JSON values?

One of the functionalities on my website involves making an AJAX call to retrieve a member's profile information for editing purposes. The code snippet responsible for this operation is shown below: function loadProfileData() { var ...

Node.js command prompt claims that 'react is non-existent'

Hello everyone! I'm excited to ask my first question on this site. I am relatively new to coding and currently learning React. Yesterday, I started working on my first project. Today, when I tried to initialize live-server and babel compiler for my J ...