Drive skeleton model with data provided frame by frame in real-time

Messages
15th May 2017 - 11:57:53 AM
I try to get data from a Kinect and then drive a human avatar created with Sea3D. However, I would like to drive the skeleton when a frame of data is captured from Kinect. I downloaded a human avatar from https://github.com/lo-th/Avatar.lab.

Some code I'm currently using is as follow



Thank you.
15th May 2017 - 04:15:41 PM
Hi is more complex but is possible

but the problem's is
kinect do world rotation for each bones
and need local rotation for model bones

so you need recreate with matrix quaternion in local
15th May 2017 - 10:31:09 PM
Hi lth, could you please kindly give some sample code to implement the function assuming I have converted the world rotation to local rotation? Thanks a lot.
16th May 2017 - 01:15:19 PM
code is complex i have to update that when i have time
also kinect is not precise enough and need some smooth algorithm
17th May 2017 - 10:22:12 PM
Thank you, lth. Looking forward to seeing your code. Yes, the precision from Kinect is a problem. I will do some research to improve it and smooth it.
18th May 2017 - 04:44:10 PM
some software have good algorithm but is not free
but main idea is from data make curve without noise
19th May 2017 - 06:02:15 AM
Thank you for the suggestions. As far as I know, I can use joint quaternion to drive the model. I will try to find a way to smooth quaternion. Alternatively, maybe I can smooth joints' positions first and then calculate quaternion after that. If you get chance, please give me some clue about driving the model with Kinect output. Thanks.
19th May 2017 - 06:35:46 PM
i have the process, i make this code last year
for the work i made, we finaly adopte a position detection instead of real motion capture
some part like foot head have realy noisy output from node ( you can see that on demo from kinect )
the best option is to combine data with kinematics and noise reducer but is udge work

i make many version of that work code so have to make research
but anyway, is a option i want add to my avatar 2.0 demo

i try to send something soon
21st May 2017 - 10:51:32 PM
Hi lth, it will be great to combine the data with some kinematic constraints, such as the length of bones and angle between bones. I read some research articles using this way to optimise the joint positions. Currently, I am able to use Kalman Filter to eliminate some noise based on joint positions.

Thank you for sharing your ideas.
23rd May 2017 - 04:59:11 PM
here my app preview
preview2
24th May 2017 - 10:39:43 PM
Hi lth, it looks great. Do you drive it in real-time? I am currently only able to draw the skeleton man. Still have no clue how to drive a 3D avatar in real-time.
25th May 2017 - 05:00:46 PM
yes real time and saved point frame ( to test )
26th May 2017 - 05:39:03 AM
It is great to know that you can drive the avatar in real-time. Would you like to share some pieces of code that how you drive the avatar with data collected in real-time. I have tried a few weeks already but still could not get it done. Thanks
8th June 2017 - 08:52:42 PM
sorry for long time but here the main file i use call K2.js
in these you have function call frame is here you send your bodyFrame from kinect with socket.io
i'm on implementation with avatar 2.0 so soon full code is aviable on github
K2.js - 13.62 kB

Help