Start work on full body inverse kinematic animation system

A place to discuss everything related to Newton Dynamics.

Moderators: Sascha Willems, walaber

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Mon Jan 07, 2019 7:17 pm

as far as I know the FBX file format for not has the notion of quaternion.

FBX is actually a original file format, it was the internal format of an animation editing tool called Film Box how was bought by Autodesk and renamed to Motion Builder.

Their specialty is quick editing of high density motion capture key framed sequences. this is why so extensive.
anyway, I just need to figure out how to extract the information I need regardless of what options where exported.

Anyway I made more progress, I am now doing two passes, one to get the complete animation period and the on a second pass the animation is resampled as a fix rate, I am using 1/60 of a second.
then there is an optimization face that remove all of the key frame that can be interpolated from the two neighbors with is some tolerance.
The model looks much better now, but there are still other bugs I need to resolve.

if you run the demo you will see that the left leg is animated wrong, I think this is actually in my code. because before when the character was facing the camera, the same bug was of the right left, but after I applied a 180 rotation to the engine animation, now the character faces away from the camera but the the bug switched yo the left leg. somewhere there is a sign bug or maybe where the animated the model the left leg has a marrow animation, which is also a feature of animation systems when you can garb keyframes and paste of another chain of bones, but because the bones where original created by mirroring the skeleton, them the animation is play as a mirror of the flipped axis.
These are the kind of bug that you can go over the entire code and not find a single bug yet is still wrong.

anyway it is much better now but more to come.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby Dave Gravel » Mon Jan 07, 2019 8:28 pm

Do you use the animation walking_inPlace ?
This is what I get here with walking_inPlace :
https://www.youtube.com/watch?v=2n8dySgu7EE
Here the same anim from all view directions :
https://www.youtube.com/watch?v=QwmD-HHQZ_U

Something seen wrong with some bone angles.
You search a nice physics solution, if you can read this message you're at the good place :wink:
OrionX3D Projects & Demos:
https://orionx3d.sytes.net
https://www.facebook.com/dave.gravel1
https://www.youtube.com/user/EvadLevarg/videos
User avatar
Dave Gravel
 
Posts: 800
Joined: Sat Apr 01, 2006 9:31 pm
Location: Quebec in Canada.

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Mon Jan 07, 2019 9:20 pm

There are still bugs.
One of the point of the full ik is to extract motion from. The animation. The in place animation maybe be more dedible.

When all the bug are fixed the simplest blonde tree will be a file and a walk in place.
And from there it should make a reasonably player that should walk without slipping over uneven terrain and even take small steps.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Tue Jan 08, 2019 4:35 pm

Oh I think I discovered what the problem that I am having,
basically it seem that in fbx animation aren't absolute they are transform matrices relative to the base post. this is why when I load the base post the animation looks one way and when I merge the body an the animation is looks different.
basically the stand alone model have a base posit, by when I merge first frame in the animation now all the key frame has a different reference, and if I apply it to the base pose the look the way the do.

so there has to be a function in the FBX api that set the skeleton to the base post before exporting the animation. so that no matter how an animation is exported the same reference even if is in a different file.

I try to fix that tonight.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Wed Jan 09, 2019 4:05 pm

ok I think I finally figure out what is going on with the animation in FBX.
basically it seems that trusting the curve stuff or the local space node function do really work or I do no know what to do the make work.

My guess is the a local function simple read a local for a nod disregarding any global setting but that's jut me speculation.
In me desperation I said well le me read the global transform and multiply by the parent inverse.
an guess what that works.

I now finally have positions and rotation bee applied correctly by the but since I ignored the scale in the animation the model look stretched like the plastic man in the fantastic 4
is not a big deal, is eassy to fix.

now the imported has to do more work because sine I am no reading the key frames, instead I am play9on the animation and getting a full transform I have to store all three values: scale position and rotations, them apply an optimization pass that compressed and remove the track no needed.

for example most bone will only have rotations, but not scale and position but this no apparent is the data is a full matrix.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Wed Jan 09, 2019 4:37 pm

actually it is still broken, I do not really understand the fbx file format, if this keeps giving me such a hard time I will probably try some other format.

This seem far more complex that it nee to be. I am following the intrusion in the tutorial by is doe no seem you work at all.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Thu Jan 10, 2019 1:43 am

Ok guys I finally figured out the problem, I also bet that the first way of exporting the animation was correct. basically there where two huge problems both with the model.

the first one is the the mesh has a scale that was apply to the solid mesh, and whe that operation is apply in max you lose the ability to freeze the scale. The only way I know to freeze the scale is to collapse the mesh and reskin it again and that a no no for me.
It is not a big problem, since the ngd format does supports that functionality, the only problem was that I have not implemented for animations. after I did that now the model is scale correctly.

The secund problem and that was a realy big one and quite hard to figure it out, after I exhausted of my debugging tricks I started to suspect the bug may be some where else. and in fact is was.
the reason the Mesh model and the walk model are so different in because in FBX there is not way to get the binding pose so when I load the mesh the mesh was on the davinci pose while the skeleton was on the T pose so the are arm were push inside the body whenne appling the walk animation.

but whe using the walk animation to load the mesh the are where on the davinci post and the left leg was at an angle.

I bet there must be a way to read the binding pose from a FBX file but for the light of me I can't find how, so I went to Max and I aligned the skelton to the mesh model and reset the binding pose in Max. (the fact there is a command to reset the binding pose tells me that the most exist in FBX)
Anyway now the model load more or lest right.

I will do some clean up of the FBX imported before start working of the Inverse Kinematic
If any body follwing this sync, that person should see an walking mand in place, and that all.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby JoeJ » Fri Jan 11, 2019 9:11 am

I see the man standing still and breathing, looks nice.
Seems quite a lot of people have similar problems with rest / bind pose, and even FBX can not end this never ending source of frustration, haha :roll:
User avatar
JoeJ
 
Posts: 1453
Joined: Tue Dec 21, 2010 6:18 pm

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Fri Jan 11, 2019 11:14 am

Untitled.png
Untitled.png (75.72 KiB) Viewed 10395 times
here is a more clean explanation of the problem, the image above.
the were picture is how the solid mesh is created and how the skeleton t pose is.

the is what yiou get you you read the mesh clusters, to decide which vertices and weight are asigne to each bones, as you can see thet is a huge gap beween teh has bones and the hand mesh.

the top one is how the mesh lok when in max you select the skin, so some how mat can determine how the move the skeleton to align with the mesh, but I do not knwo how to read that information because when I call select evaluation and call evaluate, I get a T pose, whi is incorrect for build the sking mesh.

therefore ther have to be a way to read the bind pose (teh one the ligh the skeleton to teh solid mesh) but that funtionality is not clear in FBX sdk, but is is clear that is must contatined so where because you can load any mesh in max and the get the correct binding.

as you can see build the sking doing by selection the animation evalutor will result on a diffrent skin mesh for every diffrent animation.

I looking of the forums over autodesk and you are right ther are ton of people wit the same problem working for Maya and 3dsmax what I did not find was a clear answer from autodesk.

This is a rare case, in any case because normally an animation will make a one frame animation with the bone aligned to the mesh, so that the is not difference between the binding pose and the animated frame.
My guess is that this model are mesh is and old style system with T pose that is not use by animation anymore.
But still the fact that Max can figure out how to buid teh sking from any animation tell me that indeed the binding pose is stres with eacn fbx model.

nwo that I can thin of maybe I do no find it because I was serachin for it on the herschy, but maybe this information is stored wit the skin mesh, I bet that is the sking there most be a piece of information that is an array of transforms that is used for calculating the weights instead of getting fomr the skeleton
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Fri Jan 11, 2019 11:40 am

in any case, thats not some I am going to spend time now, any animation can eassy fix that the same way I did, by fiscally aligning the T pose to teh solid mesh.

I am now start to work on the IK solver, basically this goal of this is to make a solver the
will have the hip as parent, the and effector at each heel, and another at each toe.

the toe effects use the ray or convex cast to find the ground contact and from there I will apply some rules to determine how the heels and hip you be aligned. the process is as follows:

-get the pose from the animation tree, whi is controlled by the game logic.
-inject the pose to the IK solver
-for the poses that are manipulated by the solver, the solve will run the effectors to modified those poses are constrained by the environment integrations. crate a modified target pose.
-run the solver to calculate the effect of the local changes on the neighbor poses.
-apply the final pose to the mesh.

as you can see is not really that complex and we already have each one of those face implemented separated, all I nee to do is put the together

the typical example with demo that is commited now, is that you see the man standing still breading with his feet planted on teh ground.
the us say that I take the pivot an lower it by 0.2 meters. (I commited that now)

now the feets will be buried on the ground, the ray case will detect that change and displace the feet.

the solver will propagate this change to the rest of teh body altering the pose and should result of the player crouching.

now imagine this effect be generate by modification of teh terrains instead for me just lowering down the hip, the result should be similar.

and if I get carry away myself, the final polished version should no care if an animation has the translation of not, It sould be capable to extracted for the integration with the evirimnet, but I still expect some unforeseen issues.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby Dave Gravel » Fri Jan 11, 2019 2:25 pm

I think the T pose is wrong because when I have make the rig with mixamo the original mesh don't have the normal T pose.
The original model is like /\ pose without bones.
The default pose with bones for this model is saved like this /\ not T
In the zip you can find the original mesh in obj format without nodes and bones.

I have try to fit the mesh and bones this way but it is surely not the best fbx model.
In my importer I have seen too, When I scale all matrix very low the mesh seen to fit better with the bones.
I'm not sure why some nodes is saved with the fbx Rotation and PreRotation name tag, Maybe this name only come from a bad export save i'm not sure.

I need to test more model types, Currently I can animate this model and simple md2 models.
I have try more models but all use similar bones setup, I need to find a model create from a other tool and not from mixamo about the bones.

I don't have test model from exemple: makehuman app but I think similar model use different rootnode parent, child method at the creation.
Currently I can load and animate model when I use assimp tools, but when I save the data in a personal format I get a lot more problems in the animations parts.
I don't not have get time to test more, It is very snowing here this week I have need to remove all this snow plus the other normal life things.

I try to work more on this tonight and this week-end, It's long to debug and many things can become wrong or cause problems :(
It's why here I try to have a little bones editor and use always the most possible the same bones setup with all models.

I can see and test more the newton sdk demo tonight.
Edited: I'm not sure if it is the same with your importer code, With assimp the matrix is transposed.
Last edited by Dave Gravel on Fri Jan 11, 2019 3:14 pm, edited 1 time in total.
You search a nice physics solution, if you can read this message you're at the good place :wink:
OrionX3D Projects & Demos:
https://orionx3d.sytes.net
https://www.facebook.com/dave.gravel1
https://www.youtube.com/user/EvadLevarg/videos
User avatar
Dave Gravel
 
Posts: 800
Joined: Sat Apr 01, 2006 9:31 pm
Location: Quebec in Canada.

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Fri Jan 11, 2019 3:08 pm

I think I found why I do no get it right.
I look at the class FBXSDK_DLL FbxCluster : public FbxSubDeformer
in the fbx sdk and guess what there are functions to get the bone transform matrix.

Code: Select all
    /** Get matrix associated with the node containing the link.
      * \param pMatrix     Transformation matrix to be filled with appropriate data.
      * \return            Input parameter matrix filled with appropriate data.
      */
    FbxAMatrix& GetTransformMatrix(FbxAMatrix& pMatrix) const;
    /** Get matrix associated with the parent node.
      * \param pMatrix     Transformation matrix to be filled with appropriate data..
      * \return            Input parameter matrix filled with appropriate data.
      */
    FbxAMatrix& GetTransformParentMatrix(FbxAMatrix& pMatrix) const;


that I did no use, instead I get the bone matrix to the the inde and assume the the traform on tha bone will be the one use for the skking operation.

I will probably have to make a node that store that information, so that when I build the skin I set the skelelon to those matrices instead of to the frame zero of the animation.

I have no tested yet but I think that's why 3dsmax get right form any model and I can onel get it afte I aligned the bone to the solid mesh in the whiteman model.

I am not doing that now, bu later when I go to import oeth model, I will try it out and see if it work.
the one thing that I wnat is to be reliable, whit the need to modify the model like I did.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby JoshKlint » Thu Jan 17, 2019 4:49 pm

Julio, you can use my GLTF loading code if you keep that component closed-source. I don't want that code to be given away to the public.
JoshKlint
 
Posts: 163
Joined: Sun Dec 10, 2017 8:03 pm

Re: Start work on full body inverse kinematic animation syst

Postby Julio Jerez » Thu Jan 17, 2019 5:53 pm

Thanks for the offer, but the idea of using a popular file format is that people can see the asset instead of boxes and collision shape.
At this point FBX is one of the most popular format out there, with plugin for most 3d modeling packages, and offer the funtionlitity that is needed.
Plus I am not going back to close source code under any circumstances.

I look at the GLTF format and quickly saw that is quite a mediocre brains child of some self appointed expert from the chronous group trying to make yet another collada fiasco.
The file is not even an specification, is just a bunch of incomplete rules some of which contradict each other, not GLFT is not some I use under any circumstances.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Start work on full body inverse kinematic animation syst

Postby JernejL » Mon Jan 21, 2019 3:42 am

I won't try to convince you, as i agree that GLTF is an abomination, but it has positioned itself as a really excellent intermediate format when converting file formats. It's supported in every major 3d modelling package with much more interoperability than fbx, and it works rather well in the end - at least complexity of various ways to get a model made is much simplified in the end (fbx is kinda terrible to work with, a lot of things have 3-4 versions of approaches you need to support, it's messy).

I'd recommend this project to convert FBX files to gltf: https://github.com/facebookincubator/FBX2glTF

but do find a decent loader (format is json, so that should be easy to parse)
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

PreviousNext

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 15 guests