Does Newton enable floating point exceptions?

[JoeJ]

that's the kind of problems that I claim the inverse dynamics solves, if you want to get an accurate measure of the acceleration the invDynamcic was designed for that.
It will gives accurate joint force and toque, and the net body accelerations.
try looking at the demos, you will see they are not that hard follow.
believe, over the year I have read some many papers that the large majority of what the do is to solver for that, and I am still yet has not found one that does it right. this is simply because the problem is a close system. we have it, you might a well using it.

Hmm... i might confuse inverse dynamics with IK, but here is why i'm not using it:

1. Currently, using just one joint and two bodies, it's overkill and there should be no need for it.

2. To use it for a ragdoll, i can do so only by using effectors at the ends of a chain. Thus i lack precise control of internal bodies, which means i can't control the COM of the entire ragdoll, so i could not use my balance controller at all.
Besides COM control i expect further limitations from the effector approach. E.g. lacking control over the spine, which is all internal.
My own IK solver gives me all those options, but making it react to dynamics is extra work, and wont be as good as yours.

I've said this before, but is there some kind of middle ground that i miss?
Til yet i always was happy with using joint motors.

[Julio Jerez]

on this

1. Currently, using just one joint and two bodies, it's overkill and there should be no need for it.

you are right, but if you use PD (yes proportional derivative joints) which in newton 4 is a feature of all common joints, for the case of one joint, forward dynamics and invert dynamics are the same thing.
at the end of the day a forward dynamic pass assume the external forces are input and calcutate the joint forces, from there it gets the body acceleration in a linear pass
an inverse dynamics pass, takes the same inputs and calculate the bodies accelerations, and from that, calculate the joint forces. that is why reading the acceleration form a body is not reliable.
but going an IK pass will predict the acceleration, them the forward dynamic pass will simply calculate the same acceleration.


on this

2. To use it for a ragdoll, i can do so only by using effectors at the ends of a chain. Thus i lack precise control of internal bodies, which means i can't control the COM of the entire ragdoll, so i could not use my balance controller at all.
Besides COM control i expect further limitations from the effector approach. E.g. lacking control over the spine, which is all internal.

you are partially correct. yes it is true that using IK, you reduced the work space of a joint chan.
for example decomposing a position into angles will run in gimbal lock, but that's something that animation deal all the time, and what the do is that the put helper nodes that let the move the reference frame as the animate a rig. but that limitation is a small price to pay give the rage of flexibility the get in return.

on eth rest, yo are not right, IK effector are no part of the RIG, they are added to the RIG as external joint to calculate the the bodies acceleration, them there accelerations, are used to set the desires joints local motors, for the forward dynamic solver.
if the effectors is not added the the chain just add as a regular array of connected bodies.

also of the thing of using PD, pd are a very good way to dissipate momentum formed a rig, if you look at the Ragdoll demo, you will see that they all add a PD, with zero spring and small damp. this make the rag doll act in such way that almost in every landing it get a nice looking human plausible style.

just check it out befire dismissing it.

but in any case, this weekend I will try to give a crack at the controller, and see what we get. Maybe that will be a more powerful incentive.

[JoeJ]

I think, at some point you will add the option of extra effectors to control internal bodies.
And because i have more than enough work on my plate, i can just wait for that
(I really should focus on finishing my work on gfx and geometry first. Currently i should not work on physics at all. I do so only due to some motivation problems and lack of self discipline. I need a break, which should be short.)
But i won't be ignorant in any case.

Such internal effectors could be like soft joints for example, with reduced guarantees on accuracy. So i think that should be possible.
Actually i don't see how else you could achieve things like making character standing up from lying in bed, sit down on a chair, climb over a cliff, stuff like that. Such things seem very difficult to achieve when having only partial, so indirect control over hands, feet and head.
It might work, but the result may look very different than from what art direction intends. Spine is important too in cases.

[Julio Jerez]

Against, and I know I am alone on this.

The body now only has two effetors, but the full body will be have only five.

Two for the ankle feet,
One for torso to control the spine,
Two for the forward wrist arm.

All other body parts, the feets, the hands, the kneck and tge head, will be single pd joints. That will operate procedural or from the local animation pose.

For now, those can be substitute by fix joints, until the balance is completed.

Again, my claim is that those 5 effect are sufficient, to completely control a full humanoid body.

In fact if you look at a humanoid rig made by motion capture, or by an artist, the Skelton is reduced to just those.
And tge do all this motions.

I now have sample of motion data, that will be sufficient to demonstrate that, if I can get the balance going.

But even if not, Appling the motion to tge ik rig should generate the pose.

Anyway, that's is jumping too far ahead.

At this time I am focus on setting the traing demo to have the biped test 2 train the Ann,
On tge princess, I have to complete some funtionality if the Ann.

In need the adaptive solver, that make tge traing rate adjustable,
At the moment it take very small learning rate, to it will be just to ow to run thousand of passes per time step
When we start using the more complex rig.

Anyway, i guess you are right,
I just need to keep going, and if I get some results, hopefully that will make more clear, for now it is all hopes and speculation.

[Julio Jerez]

If you sync and just run the test demo.

It is set to training mode. This show some interesting things.

The ragdoll is place in the initial pose, then is does nothing yet for 300 hundred steps.

If look at the bottom as it falls, there are two dots.
The black is the zero moment point, that is the point that if the projection of the vertical line with origin at the center of mass is very close to that point them the model is balanced.

You can see that the red dot, representing that projection first is close move start to move away, it is slow enought that, at least in my opinion the controller south be responsive enough to achieve that.

The loop repeact every 300 tick, and you can see that even when doing nothing at all tge start location is close enough that the model goes ti rest. Because the effector are not saved, given it some slight random initial start.

So tge gal of the control, will be to just move the com to align the red dot with the black dot, and just folled it if it can.

[JoeJ]

Two for the ankle feet,
One for torso to control the spine,
Two for the forward wrist arm.

All other body parts, the feets, the hands, the kneck and tge head, will be single pd joints. That will operate procedural or from the local animation pose.

That does not sound restrictive at all. I did not expect the torso control, and i guess PD joints have motors and can be posed.

Just tried the demo. I got div by 0 exceptions from drawing the two points.
After commenting that out, i got another on disabling some shader.
Neither makes any sense, but i see it's related to AMD gfx driver, so probably you can't reproduce.
Will look at than again later. For now i have disabled the exceptions.

Now i see wireframe ragdoll and the two dots. It falls on its nose, and then nothing happens.
There seems no machine learning going on. It's not reposed after 300 frames to stand and doing another try or something like that. Is there something i need to do?

[Julio Jerez]

Yes not machine learning yet, that's just setting the
Environment.

The ML will start this weekend.
The is still some more work to do before that ml start training.

[JoeJ]

Capture.PNG (84.04 KiB) Viewed 9220 times

Here the exception.
I do no longer get a second one if i comment out drawing the points.
Maybe you have an idea what could cause the div by 0.
All other rendering stuff works without issues.

[Julio Jerez]

Ah thank.

It is a divide by zero in a library we do not ha access to
But can be fix eassy if I move the handler to the physic update.

That's where it was fir a while, until I start working with Ann,

Ann are very finique, to tune, the suffer from lot of edge cases float operation, since they deal a lot with multiplying numbers that are lower than zero million and millions of time.
That how they prune branches of the net, but it just happen that on a x86 that operation result on a denormal value which make the cpu hundred of time slower.

For that there is the round to zero denormal. And is the reason I use the exception.

What I will do. Is that I will change the define to allow divide by zero, and that should solve it for everyone.

It still baffles me that big companies write code with divide by zero, and do not handle properly.
Yes I know, according the cpp standards is not illegal, but does not mean it should be used.

[Julio Jerez]

as a curiosity Joe, are you using Clang compiler?

here is an explanation of how this denormal stuff work, and set by different compiler
https://en.wikipedia.org/wiki/Subnormal_number

I will take a second look, I wrote that class many, many years ago, before the SSE, and the rules has changed.

al right, the class is now updated to use the SSE macros, instead of the constant.
it also removes divide by zero exception.

[JoeJ]

No, i was using defaults, so MSVC.
Though, usually i do the download and then overwrite the files, not rebuilding the project with CMake but keeping the old one. This could eventually cause some unexpected problems, but i do a clean before i rebuild the project.

Since some years, there seems a bug in VS. I do code changes, rebuild, but the exe does not contain my changes. So sometimes not everything is updated.
This happens very rarely, but if so, it can drive you nuts. I thought this is related to my really old project files, but now it happens to my new one too, so i guess everybody is affected. The effect is similar as with changing clock and date, so the system is confused which files are new or outdated. Doing a clean and rebuild resolves it.

Regarding denormals, i assume using the fp:fast setting in Floating Point Model also sets flush to zero for the whole project. But now i'm no longer sure. Need to check this. Fluid simulator has both huge and tiny numbers, so my performance could be affected.

[Julio Jerez]

For some reason, vs does not do it.
I am not sure why, but intel compiler set by default, so are all game console compiler. And almost any non x86 processor that I know, intel is the only cou I know does that, just because on the early day of x87 the registers where 80 bit, so the handling was a given.
But now that almost every cpu has registersvthat are the same size in memory, that a big issue. So must cpu just flush to zero. Arme cpu do not even has as an option.

Yes check your fluid, if it deals with very small numbers, you could get some boost.

For neural net that is a huge killer. That's one of the thing that separate tensor cores. The long matrix vector operation are internally extended precision, but the result are flushed. Not doing that way and using low bit float loses too much information.
Anyway, I think we hasvthat covered better now.

[Julio Jerez]

I now added the very first step of the dpn method.

This is, what the call the epsilon greedy exploration.
Basically since it know nothing about the environment, it simple takes a random action, that is roll a dice from one to tree. And if is 0, moves back, 1 do nothing, 2 more forward.

Of course from that it will only fall.

It is hard to believe that a controller can be derive from that.

But what happen next is that each time it takes a random action, it save it in what is called a replay buffer.
This buffer store, the state, the accion and the equation of the state, that figure is called the reward.

As it keeps doing more an more actions. The buffer get populated.
Here is why it works.

The buffer keep getting filled with random action, so some action has better score than others, since there are random.
The length of each run will be different, which means, the buffer will be fill with more entries with runs of good actions and fewer entries with short runs of bad actions.

So after buffer is filled, say the capacity is 30 or 40 thousand state/action pairs, action start to be over write older entries,

But since at the beginning the buffer on average has the short runs, then the bad action will be replaced with new action.

This alone has been proven to converge to a solution. If it has an I finite number if trials.

But what makes it converge faster, is the epsilon gready strategy, after the buffer is full, the training start using a neural net traiend with the entire buffet to predict a future actions, from the replay buffer,
Since the buffer has more good actions than bad action, this selection is better than just a ramdom choice.
So what it does is that, at random reduce the probably of selecting from the neural net or from the ramdome exploration.

It kepts doing that every time reducing the gready selector, until after many trial, the buffer is full with not just good actions, but action that even they seem bad, thery are good in the future, simply because the came from a long survival run.

I committed just the ramdon sector, and it is funny how he now fall back, fall fired, and each run last for different steps.

[JoeJ]

I can see it is learning from former mistakes...

Btw, you mentioned ZMP. But i can't find where you calculate this.
Where is it? I'd like to compare our math.

I'm currently back to the sole valley of uncertainty about physics math.
After seeking days for issues which did not exist, i finally found my controller only works if the ankle joint is at the same plane than the contacts. The farther above it gets, the worse my results. So something is wrong for sure, and i should fix it.
I seek for papers, can't implement their math notations, guessing, trial and error... it's frustrating ;D

Edit: Never mind - i've found it.

I see you just down project com. I dare say that's wrong (as soon as there is some movement).
This paper seems very useful: https://www.cs.cmu.edu/~cga/legs/sardain-bessonnet.pdf
Because it shows generalization to angled ground. Most formulas assume ground plane is normal to gravity.

Though that's not really my problem. I fail to calculate ankle joint acceleration to generate a desired torque on the foot body.

[Julio Jerez]

Over the years, I have read so many papers, about zero moment point robotic controllers,
What I found is that nany authors defined differently.
Some introduce the dynamic equilibrium and some don't.

I probably shouldn't criticize it, because my susscess has been very limited. But I found that at best, the best zmp controllers generate extremely 70' and 80' robotic looking gait, stuff like Asimo or Atlas.
Where you can see that the gait of the robot just moves the com to the point where the horizontal velocity is negligible.
To me that's the best definition if zmp.

Anyway, this strategy, do not lead to a natural looking gait, because a humanoid motion does not try to keep dynamic or static equilibrium, in fact the motion is a coordinated fall that predict a point in the future where it can stop the fall.
This is not possible using pure control theory, because it is based on the state and what actions will make the state stable.

Some papers extend the algorithms, but they end up on complex state machines systems, that are hard to maintain, and even if you get it, them, they do not scale up.

This is why I am turning to deep neural net to substitute the controller for a function that will simply generate an interpolated action from a large set of state and successful actions.

In this example that I am doing now, it will be a static zmp. This is the controller will move the feed to the location that will move the center of mass to the point where the horizontal speed is negligible.
One thing that a controller cannot do is stuff like say the center of mass is about to fall, the control will indicate that it either do nothing or move back. Without any extra logic, it will never decide that the best action is to just move forward and keep falling and at the same time start, the movement of another effector to predict the fall few steps in the future.
For that a very complex state machine has to be coordinated, and once that's done, it is almost impossible to maintain.
While a neural net approach will try all the conservative action to go back, but as it keeps trying, the reward of the risky action to go forward will grow simply because it survives longer than the conservative actions. So, after a while it takes the risky action to find that it will also fall. But now the same scenario repeats but on a different location of the search space.
again, after that conservative actions are taken, the risk action reward will be larger and will do it again, and so on.

This controller however will just try to keep the com in equilibrium.
You can consider this like the Apollo project, and this will be Apollo 2.

If successful, it will only handle very small perturbation of the center if mass, stuff like idle animations, moving the arms, torso or head as long as the com does not veer too much from the zmp, it should work.

What attract me about neural net, is that it can make arbitrary functions.
For example, a humanoid com, is not part of any body part, so getting the relation from the com to an effector, you have to estimate it by writing the expression of the com and calculate partial derivatives, from the solver equations, and that always difficult, and error prompt.

While the net, can just take few samples of valid input and outputs,
And train the weights, them it can generate interpolated output form that.

So even when not using deep q learning, the neural net is a great toll for making arbitrary complex functions.

Here is an example. Imagine a robot arm.
After all the work, you get and look nice, but the moment you start using it, you find that controlling is not so easy, It is hard to determine the workspace area.

But with a neural net, you can write a supervise system where you move the robot effect to few hundred allow locations, and you save them as the training set, them, you train an Ann, and you get a system where you simply feet the location of the tip, and the net will tell you if the location is legal.

Neural net can have a lot of application like that.