Talk about inverse dynamics solver

[Julio Jerez]

This is tge sequence
On Star the com is offset. So there is a torque that will make fall.
The effector check the velocity of the com and calculate a displacement to move to the com.

For tge first few frame is does with great susscess.
But afte the com is balanced. The two bodies has momentum that is try to separate them appart..
So even whe the pivot is below the com. The relative velocity is zero. So the pivot is not corrected.

Since they are moving a a velocity.
So thevresult is that for a few seconds, the difference in velocity is generating the counter torque produced by gravity.
Only they reach a configuration that is not longer linear

[JoeJ]

I think what you try to do is similar to my proposal of the naive controller of nudging the COP, and you calculate that nudging with a PID. That's clear.

But maybe the bug is related to the bad final state i have described? This final state can't be explained with any torque or momentum. It would be only legal if the ball would be fixed to the floor with a fixed joint.

The effector check the velocity of the com and calculate a displacement to move to the com.

How do you move the com? Using external force? This would explain what i see. The external force keeps the box in air although there is no stable support to the ground.
I assume you move the com only indirectly, with some distance joint between box and ball, to form an IP. But if this would work correctly, they final state could not be that illegal.
So maybe you just watch the first seconds and did not notice. Just let it drive to the right, adjust camera so you can see when it stops moving. Not sure if it still behaves this way after the changes you did during the day ofc.

[Julio Jerez]

How do you move the com? Using external force? This would explain what i see. The external force keeps the box in air although there is no stable support to the ground.
I assume you move the com only indirectly, with some distance joint between box and ball, to form an IP. But if this would work correctly, they final state could not be that illegal..

No, no external forces. All internal forces.
If you have two bodies conneter by a joint.
If the joint has an acceleration, ( a motor) it will apply some momentum to one body and the exact amount but opposit to the other. This is a fundametal principle of nature.
The engine obey with in numerical tolerances.

Imagine a person swing a leg. If the leg goes forward the body will go back, but the body mass in higher so the velocity will be small.
However say the leg is pressed against a wall or say is over a floor contraied by the ground friction.
Now all the momentum make the body goes back.
We are seeing that, but is out of controll.

Nature has mastered that by evolution.
We are trying to emulated by used the effector as an mathematical artifact to calculate the motor acceleration.
Again this part is working.
What is missing is that after it get the acceration to give the mumetum,, the acceration is now zero.
The two body get thier change of momentum and they start move at different ocuty because they ha different masses, so they get different velocities.
So after thier are in motion, the equilibrium is reach fir a while by the fact that they are moving apart. So the effect keep generation the balance torqu.
Is as if the system get to an indifrent equilibrium where no matter where is at, it is at equilibrium.

But that only happen for a few sencund.

[JoeJ]

No, no external forces. All internal forces.

Ok, as assumed.

This is a fundametal principle of nature.

Yes, but i clearly saw all laws of nature being violated by the simulation.

Again this part is working.

Maybe - due to to the above - it is not working. Or something else, eventually causing you confusion and looking at the wrong things.

I know Newton usually behaves correctly. But not always. I often saw it breaking the conversation of momentum but adding energy when working on powered joint ideas.
One example was the hopper joint which is both ball socket and slider, which worked only pinned to COMs of bodies.
Another was a joint which can both rotate and position a child body at will, which did not work at all.
The issues i got looked similar to what i see on your current model.

To test this, when i worked on my own physics engine, i always did this:
Disable gravity and make the motors move. If the averaged com of the whole model stays in place, i'm fine. If the model starts moving into a certain direction, something is wrong.
Same for angular motion: If it starts to spin and keeps spinning once the motors are off, erroneous external force must have caused the net spin? (i'm not 100% sure about this angular claim to be right)

I keep pushing on the topic because you seemingly assume i talk about issues emerging from the control method, which i misunderstand. But no - the issues i saw are real and they certainly violated laws of physics way beyond any acceptable tolerance of error. It was so obvious i did not even mention them initially, assuming you see them too, are aware and work on it. But from your responses i conclude you're not aware.

Oh - sorry! I just run the demo again, and this time it ends at a valid state!
It takes much longer till it stops than i remembered (about 30 seconds), but all looks fine.
So now i see why our discussion diverges.
But the issues were there - i assure you. Will try some stuff as i remember having done, maybe i can reproduce it...

[JoeJ]

No. I all i did was playing with numbers for 'applifyEffect', but doing this again today, no issues show up.
Maybe i saw white mice.
Maybe, because i just overwrite files after an update but do not rebuild the project, parts of an older build were left and caused issues.
Maybe there is indeed a bug and i missed the opportunity to provide a reproducible case.

However - just ignore my complains. Sorry for the noise.

[JoeJ]

Side question: Does your math lib also have simd optimized version for doubles?
Last time i looked for it i could not find that, but i'm in a need for it.
I could write a nice templated math lib, but rememberig my bad experience with GLM, which is templated but 10 times slower than the old SIMD lib i'm still using, i'm worried it would be too slow.

[Julio Jerez]

The mass library use sse2 for double yes.

It uses avx2 only for the solver when using that slover.
That's why the solve copy and paste the functions needed to glue front the solver internal to the object.

But yes all math in newton is simd, wether it was 32 or 64 bit floats.

I too had the same experience when using templates to make the vector library.
I believe is is a visual studio thing. It is a black art to get it to generate really optimized code in simd.
It need all the special constructors and asigments so that all operation between statement are inlined in optimal way, if not it resolve to funtion calls.

If you look at the vector class. I had add special macros the generate the destructor property or else using the allocator will automatically make functions calls.

The math library in newton is by far the best I ever seem at generating binary code, it beats the Microsoft one and even beat me. Also seem vs 2015, the code generator for vs has gotten better at register assigmnet, produces much better binary and has gotten even better sence them so it is not as bad as it used to.

[JoeJ]

Ha cool, then i can just use Newton, thanks.

When i was puzzled by bad GLM perf., i also came to the conclusion it's related to constructors, because profiler showed so much time spent there. Would be interesting to see how it does using Clang, but currently Clang does not work anymore for me. Just a single error message which tells me nothing. I hope VS22 is out soon to fix this. Iirc, Clang gave me speedup of 30% for fluid sim, but not so much on other things.

I'm still very confused by this, and became generally worried about using C++11 features. I don't use templates so much, but lambdas a whole lot. Never did any performance comparisons for either of these.

[Julio Jerez]

If you see math function in a profiler in a really build that' a sign that is not issuing inle code. So is spending the ti.e in calling convention.
For the most part not math funtion should show in a profiler,, unless they are explicitly made funtion call or the compiler made them function calls.
The condition as to why a compiler will make function call are the part that are default to maximize. But even doing your best, a compiler can make a math operation a function call.
For example a a single statement of a very long sequence of operators could resul in a code line so long that may trigger some of the compiler optimization clue ad decide that making the calls is better.
Compiler has gotten so good now that this is not a worry anymore.
In newton I made some math function explicity function calls, stuff like matrix multiply, matrix construction. Would result in huge binaries.
Imagine a line that multiply 4 matrices. Inclined would be like 30 of four hundred line of assembly, but the call is bout a dozen.

All in all, I am very satisfied with the math library in the engine, it does what it needs and nit more.

[Julio Jerez]

wow, joe.
after going step by step checking that all math ins all the system checked out as expected.
everything checked out right within the expected numerical errors.
That was scary because it would mean that my premises and intuitions were all wrong, and my algorithm was valid but not sound, basically the proverbial garbage in garbage out.
But as I was doing those checks, I did find that that indeed I was inputting bad data to the effector.
Once I correct that, the pogo on a stick now behave exactly as I predicted and far better than I anticipated.

if you sync and try it you will see the big bug resting of top of the small one,
them after a few frames, then big bug receives a nudge to the right.

if no action is taken by the effector, the ball should fail, you can test that by commenting out
line 287 file ...\applications\ndSandbox\demos\ndInvertedPendulum.cpp

but when the effect tries to move to prevent the box from falling, you will see that start displacing then the box keeps falling, then it keeps trying to catch up again until the PD reaches the distance were the spring and damper cancel each other.
the result is the same as the moon earth gravity system.

in this analogy the big box is the moon, that keeps falling to the ground, but the small ball pushes up by the same amount that is falling, and at the same time translate to the right.

in this analogy the difference between the two frames on the small ball, represents the tide.

the reason I say it works far beyond what I expected, is that is tolerates fairly large velocities, at first a velocity of 0.2 caused what you saw and was wrong everywhere, but after applying the proper control strategy, it can handle velocities of 1.0 m/s probably more.

now the next step is
-add the PID functionality so that the control applies cumulative correction and reduce the error.
this should reduce the gap, over time.
-add the rotation control, so that it knows how to stop,
remember momentum is conserved all the way. you see it moving because when the impulse is applying some momentum to the system, so it should maintain it perpetually, if the model stopped, that would be wrong.
what that means is that there should be some internal drag to consume that momentum, and that's the next challenge.

[JoeJ]

I'm baffled this works so well.
I can throw it up to air, and if it lands on the ball, it instantly becomes balanced.
Looks like as you did some cheating

All it does is rolling the ball towards the com projection?
From the code it looks you only set effector position, so does the rolling happen just due to friction?

I think you should add visualization of the effectors, so we can easily see what you're doing.

[Julio Jerez]

no cheats, my friend, not cheat.
almost all of my intuitions where correct, it took a while, but in the end I still believe that walking and keeping balance, is not a big deal. If vision has evolved in nature about 40 times, Flying about a dozen, walking has evolved several thousand times, Plant and amount the only creatures that has not evolved that trait, but even a virus can transport.
To me, this is an indication that walking is such extreme basic functionality that even a deep learning machine using really bad physics engine can find a way around to walk.

but any way, a great science fiction writer once said.

Profiles of the Future: An Inquiry into the Limits of the Possible”, science fiction writer Arthur C. Clarke formulated his famous Three Laws, of which the third law is the best-known and most widely cited: “Any sufficiently advanced technology is indistinguishable from magic

this is how it looks like that what such a little action, it works so well, in fact too well.

the way we can make fall, is doing this say nudge is so strong that, that the translation for the effector, is larger than the workspace where it can move, the controller will clip the position of the angle so the, and the body should just fail.

I did some more experiments, just for the how to know part, and is actually looking better,

Joe, make no mistake, it seems simple, but there is ton of work under that simple translation, I just forgot that the goal what to make that simple since it took so long to get to this point.

but anyway, we keep experimenting, I will try offsetting the pivot and the see what happens when is has actual limbs. and yes I will add more debug display so that shows what is happing.

[Julio Jerez]

All it does is rolling the ball towards the com projection?
From the code it looks you only set effector position, so does the rolling happen just due to friction?

yes, and yes, in fact it turned out friction is actually too strong but that's a good thing.
you can see how the ball get stuck, and when pushed by the joint (in this case that will be like a joint muscle) it just rolls.
in a more realistic setting, we want the friction to break so that the ball just slide, this will be a situation like sand, ice, the problem is that a DP is not applying a strong enough angular acceleration to make break the friction. but I expect this to improve when adding the PID functionality.

[JoeJ]

Interesting. I guess you're coming up with very a different method of balance than i did.
Maybe i need to start saving to pay beer...

It indeed looks easy, and actually too perfect. But if you make it doing some stuff like following a path or standing upright, it will start looking alive.
Similar to my hopper. Those toy models we do would make nice next gen pokemons

[Julio Jerez]

ok, Joe
on the I start to make the PID, is not done yet.

but in case you want to see the behavior, that we are looking for, if you sync, you will see the same demo with old method cleaned up. IT is amazing that in the end if this work, a controller does no really needs to know about com, of support polygon, no on that is really important, all it needs to know is if is has a contact and is can work on that.

but anyway, I am display with a yellow dot, where the controller wants to place to place the sphere,
by since the sphere is pressed to the ground, it has a very strong friction so the only way is by rolling without sleeping, so there is a phase distance between the position and the target, cause the bog box to keep falling, and recovering perpetually.
Ideally, we would one that the controller is strong enough that the phase will break friction and sleep.
when I added that code, it starts to tumble, this is actually expected, all it means is that the is a max limit of recovering beyond that the falling is larger that joint correction.

now if you want to see what the expcnet behavior is,
syunc and running it.

you will see the yellow ball, moving ahead and point to the ideal support point.

the comment out file. ../newton-4.00\sdk\dNewton\dIkSolver\ndIk6DofEffector.cpp
line 289 //SubmitAngularAxis(matrix0, matrix1, desc);

this will let the ball not to be constrained to math the parent rotation, so it is free to move without generation counter torque. so int simply can translate as fast as it can to be at the support point,
and the result is that once the ball reach that angular momentum, it keeping because is enough to keep rolling to keep up with the top box. the top box god the falling toque for just few frames.

so what we look for, with the PID, is that the side force is stringer that the friction, and the ball just sleep, to keep up with the ball, so we are expecting a behavior int between those two extremes,
the closer to the behavior when the angular constraints are disabled