Refactoring joints

[Julio Jerez]

The ball and socket is not fuuly finished yet, only the twist.
But you can see how many options are possible by moving some of the calculation to the joint callback, rather than trying to be smart and second guess what the joint wants to do.

With the ball socket, I believe we now have all the basic joints. And I will move to the specialised one like the servos.
I will also add you test as and indepndend demo to try different ways.
I. Hope this weekend we are ready to get back to the self balancing faith again.

[JoeJ]

Where is the code where you do new calculations within callback?

Oh, i see now, i was one version back.

So it's only NewtonUserJointCalculateRowZeroAccelaration that is new.

I see also NewtonUserJointGetRowAcceleration in Newton.h
Guess that's also a accurate value at the current 'substep'?

[Julio Jerez]

Where is the code where you do new calculations within callback?...
So it's only NewtonUserJointCalculateRowZeroAccelaration that is new. ...

Oh but is more than just that.
Before when a row violated a limit, the callback issued a friction limit and either used the joint derivative or a user calculated acceleration to stop the movement and to recover from the error.
This is a big mistake for two reasons.
1 the joint derivative does not include position, so this has to be added an adhock code by converting the violation to a velocity and then to an acceleration.
2 the joint derivatives assume the friction force is infinite, therefore they could zero any velocity, but this only works when the velocity was positive, negative velocities cause the joint to bounce back with unpredictable velocity because the friction is no infinite in that direction.

Now, the joint calculates the stop velocity for each row, but does not consider penetration error.
Then in the joint code, do some kind of one dimension prediction to see if with the current joint velocity the joint will reach the limit. look at the slider as an example.

When the prediction say the joint will reach or cross the limit,
The code calculates what speed the joint need to have to reach the limit exactly starting from zero speed.
We now can do that because the joint provides the zero speed acceleration, so all we need to do is calculated the new acceleration and add that to the zero speed acceleration.

The joint will move closer to the limit, if it does not cross the limit, the process repeat, but the joint ideally never change direction.

If because numerical errors and non linearyties the joint crosses the limit, the joint will have zero velocity and the correction is a very small velocity because the violation is not longer a constraint violation is a numerical error violation which should be an infinitesimal value. at least that's the conjecture I made, and it seem to be right in more cases. Very big difference.

The reason this is now in the joint, is because different joints measure errors in different ways,
Yes it makes the implementation more involved but, to me the results are worth the effort.

[JoeJ]

Ah yes, almost get it. Very cool. I already used that for my picking joint so it allows better motors as well

[Julio Jerez]

ok now I believe I have the standard joints completed.
There seem robust enough that we can expect prediction form their behavior to model stuff using then as the base of the design.

This was an aspect that was bothering me for a long time. teh joint were for teh most part fine, until hey hit a limit and then the behavior was unpredictable. now this is more predictable.

any way next I will re enable all the standard joints demos one by one in case I missed something,
and next start re enabling the high level demos starting with the basic ragdoll which now will be not a plain ragdoll, it will be like a department store mannequin, this because all the joint will have a sticky dry friction that will allow to make any pose out them.

I will make one fox to the fix joint to indicate that the user can pick limbs and orient then to make poses.

[JoeJ]

this because all the joint will have a sticky dry friction that will allow to make any pose out them.

That would be a nice feature to distinguish your engine from others at first glance.
Maybe set in on for any joint by default with a low friction value.

[Julio Jerez]

That would be a nice feature to distinguish your engine from others at first glance.

they do now and it does no have to be a default, is and option that apply to all joints.

I am now going down the list demos. and I notice that the Restitution demo was malfunctioning and the was because the floor normal was wrong by 1.0e-5 on one corner, is unebelibebale that was enough to tile the normals until the eventually tumble.

after I fixed that we, now I see another small problem.
as an optimization I am singling out one contact island and do doing the RK4 instead calculation the forces reaction forces and calling the free body solver.

These integration we know is better that the general multiple joint solver. because it implicit. whoever it now show one other problem.
The implicit integration apply only to the rotation, the translation is still simple Euler.

The result is that the demo of the right now slowly loses momentum, but at the collision by doing integration.
I should probably do the same for the translations.
another solution is to just sue the general solver.

can you sync an run the demo. and tell me if this is worth the effect.

edit:
actually I saw is was losing energy, but I take that back, I looked for a while and saw the top box on the right no reaching the max high, by I di no notice the lower ball moving faster, then after a while the all band together to kick the box high and the all lose the momentum, is really remarkable how cool it is.
It does loses some energy over time but that's normal for an euler integration.

a reason I do not want to change that is that the loop will required another pass, or anoteh variable to integrate velocity and position independelly, just for single bodies. that no worth the effort

[Julio Jerez]

Hey Joe you have to check this out.

I added the rag doll as I explained above, and it is like you can play with the limbs at will.

I am really bad at manipulating things on the screen, but I an sure that with the right tool this could be quite cool. It is almost like you want to have a virtual reality control to interact with a real virtual world.

This actually show that it is quite possible to manipulate those limbs with the inverse dynamics.
in reality that is what we are doing in this demo, and this can be a teaching tool to learn the how to know tricks on how to create poses.

I am now going more dolls to test performance, tand play one fixed to the world with a fix fix joint instead of making the pelvis infinite mass.

[Julio Jerez]

one really easy feature that occur to me for improving the editing functionality is to make it that
when you pick a joint to put in position, a routine walk the hierarchy saving all the friction values and changing then with a very high friction, except the parent joint picked by the that the mouse.

this will allow local limbs manipulation.

I think the could be a really cool and interesting editing functionality.
is not physics related but shows how goog physics can facilitate non physics things.

I will probably try it later

There is also another thing I noticed, almost all poses are cool poses, even when move one move the parent limbs.
I am not sure but I think we are getting something here.
I am having lot of fun playing around but there is so much tuff still to complete.

[Julio Jerez]

I now added a set of rag dolls and one is foxed to the world with a fix joint.
this style is far more expensive to simulate, so I added 10 instead of teh 25 that I have before.

I am no going to worry too much about that, teh simulation is manageable and we have multicores system now, any platform now has a t least three cores, so was wee nee to focus is in make the multicore solve more efficient whi will be teh focus of newton 3.15

also we can make the rag doll 4 time faster is we use viscous friction instead of dry friction,
in fact I think I made the friction too strong, and teh courses act as if they were solid bodies, which is cool bu need tuning.

anyway now will add teh collision mask because ether do not collide with each other and I will try to re enable the articulated traded tank, whi I have some idea for making it functional, and not just a for show demo.

Note: there is one thing you may notice
The head dismember, bu tha is because the neck is a very thong body with very small mass calculate by body density, tshi ass lot of high condition number the mass matrix, I will later single out the neck and make heavier.

also I think that skeleton has too many degree of freedom, the spine could have two segments instead of three. tha way the joint can have larger limits, but I do not have to start over with another model.

[JoeJ]

This looks all really good

also we can make the rag doll 4 time faster is we use viscous friction instead of dry friction,

Yes, for ragdolls we need only accurate motors. Limits do not need to be stiff, and in fact it feels unnatural. Even elastic limits would do.

also I think that skeleton has too many degree of freedom, the spine could have two segments instead of three. tha way the joint can have larger limits, but I do not have to start over with another model.

Oh no, that would be a big mistake: You need at least 3 bodies so torso and pelvis have independent control of orientation. 3 is minimum, but more would be a waste.

You also need shoulders (actually you link arms directly to torso), but you could add those details later.

For the feet i would stick with the single body for now, and later divide it into two bodies, so you can lift the heel while still stable support for the toes - imprtant for running. (I started with two bodies, but later reduced to one to keep things simpler. When walking would work, i would go back to two was my plan.)

I am having lot of fun playing around but there is so much tuff still to complete.

Do you know those high level IK animation rigs used for animation? You should look at this when you get a chance. The best i know comes with Autodesk XSI, there is a menu command that creates one for you with few clicks. This was my main inspiration for my IK solver. What your demo misses is mainly a gizmo to control orientation of effectors. But those rigs have some more controls that work in a very intuitive way and allow to animate very quickly.

[JoeJ]

I have an isea about the ragdoll joint, but no time to try out...

I think we should rotate the entire constraint space including the limits definition halfway between the parent and the child. This will eliminate all gimbal lock problems with the twist, and we need to set limits half to their previous value to get the same effect.

[Julio Jerez]

Ok joe you won when you warned me that doing teh Euler stull will never work.

I was trtu to re enable the thread vehicle, and I see thet the hinge and slider are too soft
so I was debugging thsi afternoon, and I race to this.
The old code was using this

Code: Select all

   dFloat sinAngle;
   dFloat cosAngle;

   // construct an orthogonal coordinate system with these two vectors
   dMatrix matrix1_1;
   matrix1_1.m_up = matrix1.m_up;
   matrix1_1.m_right = matrix0.m_front.CrossProduct(matrix1.m_up);
   matrix1_1.m_right = matrix1_1.m_right.Scale(1.0f / dSqrt(matrix1_1.m_right.DotProduct3(matrix1_1.m_right)));
   matrix1_1.m_front = matrix1_1.m_up.CrossProduct(matrix1_1.m_right);
   NewtonUserJointAddAngularRow(m_joint, CalculateAngle(matrix0.m_up, matrix1_1.m_up, matrix1_1.m_front), &matrix1_1.m_front[0]);
   NewtonUserJointAddAngularRow(m_joint, CalculateAngle(matrix0.m_up, matrix1_1.m_up, matrix1_1.m_right), &matrix1_1.m_right[0]);

   // the joint angle can be determined by getting the angle between any two non parallel vectors
   CalculateAngle(matrix1_1.m_front, matrix1.m_front, matrix1.m_up, sinAngle, cosAngle);
   m_curJointAngle.Update(cosAngle, sinAngle);


you can see that is project the axis to maximize the correction. and is very solid.
teh new code calculate the Euler angular and do the correction assuming the error is alwya small.
like this

Code: Select all

   dMatrix localMatrix(matrix0 * matrix1.Inverse());
   dVector euler0;
   dVector euler1;
   localMatrix.GetEulerAngles(euler0, euler1, m_pitchRollYaw);

   NewtonUserJointAddAngularRow(m_joint, -euler0[0], &matrix1[0][0]);
   NewtonUserJointSetRowStiffness(m_joint, m_stiffness);
   NewtonUserJointAddAngularRow(m_joint, -euler0[2], &matrix1[2][0]);
   NewtonUserJointSetRowStiffness(m_joint, m_stiffness);

the code is simpler by is only goo as long as the error is small, teh moment the correction is no enough the error accumulate and the joint bends.

hopefully this is no a big problem because is not hard to fix, but is soft enough that to me is not acceptable. I made a video show in teh tow method


so I think I will change the to that system as per need, hopefully now that joint are inherited this should not be a big problem

[JoeJ]

Whew - but that's not he kind of problem i expected coming from euler angle decomposition. Looks mor like you accidently rotate the constraint space where limits are defined.

Seeing from the code snippet you use 2 angular rows for this motorized hinge. Eventually using only one angular row for the twist and 2 linear rows to stabilize swing axis (as you usually do) would be better? (Years ago I've tried both approaches for my ragdoll hinges and there seemed no noticeabele difference.)

I think we should rotate the entire constraint space including the limits definition halfway between the parent and the child. This will eliminate all gimbal lock problems with the twist, and we need to set limits half to their previous value to get the same effect.

I tried this, not completely done yet, but very interesting.
Currently, when defining all limits relative to parent we might get into trouble if the bones bend towards 90 degrees, because twist axis rotation would only 'swing' the child bone, but not 'twist' it.
Gimbal lock issue. If the limits are defined in averaged orientation between them both, anything is mapped a lot better.
Another advantage might be that mathematically there is no more distinction between parent and child.

I've updated the gist: https://gist.github.com/JoeJGit/7e269bdee6994c34b54011918f69b8cd
There is a define on top AVERAGE_CONSTRAINT_SPACE you'd need to turn on and visualize joint limits to see how it works. (arc accidently rotates as well, i'll fix this sometime, but for simple cone limit it would do for now)

However it does not fix the motors going crazy with TEST_CASE 3, which is what i've initially hopped for.

The more i work on this, the more questions arise, even if just minor ...

I recommend you create an independent ragdoll joint, even independent of ball and socket. Many issues will show up only later when working with controlled ragdoll, tweaking, trying this trying that... You don't want to affect other joints while doing so

(Unfortunately my own ragdoll is still broken so i can not test joints with that, working a bit blindfolded when trying to improve joint without a really demanding usecase.)

[Julio Jerez]

those are good points, what I am learning is that there are multiple ways to make the joint and some seem better than others, but I do not think there will be a one method fix all solution.
what we need is the proper joint for the proper job.

al right, I now managed to put the treaded tank in he scene, and this time I did not even have to reject contacts like I was doing before.
also the threads are part of the vehicle, not need for fenders shape to keep the thread on the tracks. instead the links are constrained to the path with just one link.
I never though this would work, but so far seems stable enough.

The thread are too lose, i will try to tweak it by removing one link later. One of the problem is that with suspension is difficult to figure out the thread proper link count, by this is obviously too lose.

I will alter and add the engine and the boom latter, but for that I need to revisit the servo joints which were commend out for the changes of the low lever joints, and that will be all for this demo.

Please if you sync, try to check it out and tell me how is performance in your system.

Remember this is not intended for game production is just for show off and for people who do make non real time stuff like skechtup. but My goal is to make functional at interactive time.