Start work on full body inverse kinematic animation system

[Julio Jerez]

as far as I know the FBX file format for not has the notion of quaternion.

FBX is actually a original file format, it was the internal format of an animation editing tool called Film Box how was bought by Autodesk and renamed to Motion Builder.

Their specialty is quick editing of high density motion capture key framed sequences. this is why so extensive.
anyway, I just need to figure out how to extract the information I need regardless of what options where exported.

Anyway I made more progress, I am now doing two passes, one to get the complete animation period and the on a second pass the animation is resampled as a fix rate, I am using 1/60 of a second.
then there is an optimization face that remove all of the key frame that can be interpolated from the two neighbors with is some tolerance.
The model looks much better now, but there are still other bugs I need to resolve.

if you run the demo you will see that the left leg is animated wrong, I think this is actually in my code. because before when the character was facing the camera, the same bug was of the right left, but after I applied a 180 rotation to the engine animation, now the character faces away from the camera but the the bug switched yo the left leg. somewhere there is a sign bug or maybe where the animated the model the left leg has a marrow animation, which is also a feature of animation systems when you can garb keyframes and paste of another chain of bones, but because the bones where original created by mirroring the skeleton, them the animation is play as a mirror of the flipped axis.
These are the kind of bug that you can go over the entire code and not find a single bug yet is still wrong.

anyway it is much better now but more to come.

[Dave Gravel]

Do you use the animation walking_inPlace ?
This is what I get here with walking_inPlace :
https://www.youtube.com/watch?v=2n8dySgu7EE
Here the same anim from all view directions :
https://www.youtube.com/watch?v=QwmD-HHQZ_U

Something seen wrong with some bone angles.

[Julio Jerez]

There are still bugs.
One of the point of the full ik is to extract motion from. The animation. The in place animation maybe be more dedible.

When all the bug are fixed the simplest blonde tree will be a file and a walk in place.
And from there it should make a reasonably player that should walk without slipping over uneven terrain and even take small steps.

[Julio Jerez]

Oh I think I discovered what the problem that I am having,
basically it seem that in fbx animation aren't absolute they are transform matrices relative to the base post. this is why when I load the base post the animation looks one way and when I merge the body an the animation is looks different.
basically the stand alone model have a base posit, by when I merge first frame in the animation now all the key frame has a different reference, and if I apply it to the base pose the look the way the do.

so there has to be a function in the FBX api that set the skeleton to the base post before exporting the animation. so that no matter how an animation is exported the same reference even if is in a different file.

I try to fix that tonight.

[Julio Jerez]

ok I think I finally figure out what is going on with the animation in FBX.
basically it seems that trusting the curve stuff or the local space node function do really work or I do no know what to do the make work.

My guess is the a local function simple read a local for a nod disregarding any global setting but that's jut me speculation.
In me desperation I said well le me read the global transform and multiply by the parent inverse.
an guess what that works.

I now finally have positions and rotation bee applied correctly by the but since I ignored the scale in the animation the model look stretched like the plastic man in the fantastic 4
is not a big deal, is eassy to fix.

now the imported has to do more work because sine I am no reading the key frames, instead I am play9on the animation and getting a full transform I have to store all three values: scale position and rotations, them apply an optimization pass that compressed and remove the track no needed.

for example most bone will only have rotations, but not scale and position but this no apparent is the data is a full matrix.

[Julio Jerez]

actually it is still broken, I do not really understand the fbx file format, if this keeps giving me such a hard time I will probably try some other format.

This seem far more complex that it nee to be. I am following the intrusion in the tutorial by is doe no seem you work at all.

[Julio Jerez]

Ok guys I finally figured out the problem, I also bet that the first way of exporting the animation was correct. basically there where two huge problems both with the model.

the first one is the the mesh has a scale that was apply to the solid mesh, and whe that operation is apply in max you lose the ability to freeze the scale. The only way I know to freeze the scale is to collapse the mesh and reskin it again and that a no no for me.
It is not a big problem, since the ngd format does supports that functionality, the only problem was that I have not implemented for animations. after I did that now the model is scale correctly.

The secund problem and that was a realy big one and quite hard to figure it out, after I exhausted of my debugging tricks I started to suspect the bug may be some where else. and in fact is was.
the reason the Mesh model and the walk model are so different in because in FBX there is not way to get the binding pose so when I load the mesh the mesh was on the davinci pose while the skeleton was on the T pose so the are arm were push inside the body whenne appling the walk animation.

but whe using the walk animation to load the mesh the are where on the davinci post and the left leg was at an angle.

I bet there must be a way to read the binding pose from a FBX file but for the light of me I can't find how, so I went to Max and I aligned the skelton to the mesh model and reset the binding pose in Max. (the fact there is a command to reset the binding pose tells me that the most exist in FBX)
Anyway now the model load more or lest right.

I will do some clean up of the FBX imported before start working of the Inverse Kinematic
If any body follwing this sync, that person should see an walking mand in place, and that all.

[JoeJ]

I see the man standing still and breathing, looks nice.
Seems quite a lot of people have similar problems with rest / bind pose, and even FBX can not end this never ending source of frustration, haha

[Julio Jerez]

Untitled.png (75.72 KiB) Viewed 3468 times

here is a more clean explanation of the problem, the image above.
the were picture is how the solid mesh is created and how the skeleton t pose is.

the is what yiou get you you read the mesh clusters, to decide which vertices and weight are asigne to each bones, as you can see thet is a huge gap beween teh has bones and the hand mesh.

the top one is how the mesh lok when in max you select the skin, so some how mat can determine how the move the skeleton to align with the mesh, but I do not knwo how to read that information because when I call select evaluation and call evaluate, I get a T pose, whi is incorrect for build the sking mesh.

therefore ther have to be a way to read the bind pose (teh one the ligh the skeleton to teh solid mesh) but that funtionality is not clear in FBX sdk, but is is clear that is must contatined so where because you can load any mesh in max and the get the correct binding.

as you can see build the sking doing by selection the animation evalutor will result on a diffrent skin mesh for every diffrent animation.

I looking of the forums over autodesk and you are right ther are ton of people wit the same problem working for Maya and 3dsmax what I did not find was a clear answer from autodesk.

This is a rare case, in any case because normally an animation will make a one frame animation with the bone aligned to the mesh, so that the is not difference between the binding pose and the animated frame.
My guess is that this model are mesh is and old style system with T pose that is not use by animation anymore.
But still the fact that Max can figure out how to buid teh sking from any animation tell me that indeed the binding pose is stres with eacn fbx model.

nwo that I can thin of maybe I do no find it because I was serachin for it on the herschy, but maybe this information is stored wit the skin mesh, I bet that is the sking there most be a piece of information that is an array of transforms that is used for calculating the weights instead of getting fomr the skeleton

[Julio Jerez]

in any case, thats not some I am going to spend time now, any animation can eassy fix that the same way I did, by fiscally aligning the T pose to teh solid mesh.

I am now start to work on the IK solver, basically this goal of this is to make a solver the
will have the hip as parent, the and effector at each heel, and another at each toe.

the toe effects use the ray or convex cast to find the ground contact and from there I will apply some rules to determine how the heels and hip you be aligned. the process is as follows:

-get the pose from the animation tree, whi is controlled by the game logic.
-inject the pose to the IK solver
-for the poses that are manipulated by the solver, the solve will run the effectors to modified those poses are constrained by the environment integrations. crate a modified target pose.
-run the solver to calculate the effect of the local changes on the neighbor poses.
-apply the final pose to the mesh.

as you can see is not really that complex and we already have each one of those face implemented separated, all I nee to do is put the together

the typical example with demo that is commited now, is that you see the man standing still breading with his feet planted on teh ground.
the us say that I take the pivot an lower it by 0.2 meters. (I commited that now)

now the feets will be buried on the ground, the ray case will detect that change and displace the feet.

the solver will propagate this change to the rest of teh body altering the pose and should result of the player crouching.

now imagine this effect be generate by modification of teh terrains instead for me just lowering down the hip, the result should be similar.

and if I get carry away myself, the final polished version should no care if an animation has the translation of not, It sould be capable to extracted for the integration with the evirimnet, but I still expect some unforeseen issues.

[Dave Gravel]

I think the T pose is wrong because when I have make the rig with mixamo the original mesh don't have the normal T pose.
The original model is like /\ pose without bones.
The default pose with bones for this model is saved like this /\ not T
In the zip you can find the original mesh in obj format without nodes and bones.

I have try to fit the mesh and bones this way but it is surely not the best fbx model.
In my importer I have seen too, When I scale all matrix very low the mesh seen to fit better with the bones.
I'm not sure why some nodes is saved with the fbx Rotation and PreRotation name tag, Maybe this name only come from a bad export save i'm not sure.

I need to test more model types, Currently I can animate this model and simple md2 models.
I have try more models but all use similar bones setup, I need to find a model create from a other tool and not from mixamo about the bones.

I don't have test model from exemple: makehuman app but I think similar model use different rootnode parent, child method at the creation.
Currently I can load and animate model when I use assimp tools, but when I save the data in a personal format I get a lot more problems in the animations parts.
I don't not have get time to test more, It is very snowing here this week I have need to remove all this snow plus the other normal life things.

I try to work more on this tonight and this week-end, It's long to debug and many things can become wrong or cause problems
It's why here I try to have a little bones editor and use always the most possible the same bones setup with all models.

I can see and test more the newton sdk demo tonight.
Edited: I'm not sure if it is the same with your importer code, With assimp the matrix is transposed.

[Julio Jerez]

I think I found why I do no get it right.
I look at the class FBXSDK_DLL FbxCluster : public FbxSubDeformer
in the fbx sdk and guess what there are functions to get the bone transform matrix.

Code: Select all

    /** Get matrix associated with the node containing the link.
      * \param pMatrix     Transformation matrix to be filled with appropriate data.
      * \return            Input parameter matrix filled with appropriate data.
      */
    FbxAMatrix& GetTransformMatrix(FbxAMatrix& pMatrix) const;
    /** Get matrix associated with the parent node.
      * \param pMatrix     Transformation matrix to be filled with appropriate data..
      * \return            Input parameter matrix filled with appropriate data.
      */
    FbxAMatrix& GetTransformParentMatrix(FbxAMatrix& pMatrix) const;


that I did no use, instead I get the bone matrix to the the inde and assume the the traform on tha bone will be the one use for the skking operation.

I will probably have to make a node that store that information, so that when I build the skin I set the skelelon to those matrices instead of to the frame zero of the animation.

I have no tested yet but I think that's why 3dsmax get right form any model and I can onel get it afte I aligned the bone to the solid mesh in the whiteman model.

I am not doing that now, bu later when I go to import oeth model, I will try it out and see if it work.
the one thing that I wnat is to be reliable, whit the need to modify the model like I did.

[JoshKlint]

Julio, you can use my GLTF loading code if you keep that component closed-source. I don't want that code to be given away to the public.

[Julio Jerez]

Thanks for the offer, but the idea of using a popular file format is that people can see the asset instead of boxes and collision shape.
At this point FBX is one of the most popular format out there, with plugin for most 3d modeling packages, and offer the funtionlitity that is needed.
Plus I am not going back to close source code under any circumstances.

I look at the GLTF format and quickly saw that is quite a mediocre brains child of some self appointed expert from the chronous group trying to make yet another collada fiasco.
The file is not even an specification, is just a bunch of incomplete rules some of which contradict each other, not GLFT is not some I use under any circumstances.

[JernejL]

I won't try to convince you, as i agree that GLTF is an abomination, but it has positioned itself as a really excellent intermediate format when converting file formats. It's supported in every major 3d modelling package with much more interoperability than fbx, and it works rather well in the end - at least complexity of various ways to get a model made is much simplified in the end (fbx is kinda terrible to work with, a lot of things have 3-4 versions of approaches you need to support, it's messy).

I'd recommend this project to convert FBX files to gltf: https://github.com/facebookincubator/FBX2glTF

but do find a decent loader (format is json, so that should be easy to parse)