Continuous Collision crash after Newton 2.36 to 3.04 upgrade

[JoeJ]

Those are 3 cases of test scenes. You can enable the first one with #ifdef 1 and so forth

1.st scene is CCD assert, which happens immideatly.
(I guess the other cases would crash too, if you enable CCD there)

2nd and 3rd sections cover the penetration bug, like described in 'collision to be hard' thread.
Both show the box starting jittering and sinking a little into the floor.
Looks not like a big problem, but for powered stuff it is.
It would be great if that can be improved.

[Julio Jerez]

Oh I see what the problem is, Newton 300, is similar to newton 1.xx where the solve handle impulse and force joint derivative differently.
in 2.00 I cut out the impulsive solver for performance reasons because this was done by having two solvers and it was too complicated, The problems is that by calculation average impulse the
physics behavior is too unpredictable when time step are too small, it is not really physically accurate to solve impulses contacts as if they were resting contact acting over a non zero timestep.
In core 300, the same solve can handle impulse and resting contacts, bur the callback have to calculate the derivative correctly.
The problem is that I forget to fix the joints so that the deal with the impulsive contact correctly. I had done it for contact joint so fact.

I will fix them all first, that will take care of the crash. Impulsive contacts are call with timestep set to zero, the code crashes because the joint try to calculate the reciprocal of zero.
for each joint and row most the values are the same, but some of the acceleration vanish. I need to add an engine function to calculate impulse at joint contact.

Then we see the other two case in the demo.

[JoeJ]

That explantation gives sense for all 3 cases to noobish me.
I don't remember a penetration problem while using 2.x.
Maybe all problems have similar reason

[Julio Jerez]

Ok Joe, I fixed the call back (only kinematic joint the rest will follow after we test this first)
Please sync to SVN and try again. see if this is what you expect.

[Julio Jerez]

Ok Joe I now tried the second and third test. I they show jitter.
This is in fact the correct behavior. I explain
you have and articulation with tow joints.
one is the contact joint wit eth ground, the second is a bilateral joint which a goal to move the box to a specific location.
The problem is that the location is not physically legal, since part of the Box penetrate the ground

this kind of systems yield ill condition constraint system matrices.
The interpretation of this is that in one step the contact joint try to eliminate the penetration by calculation the force to resolve it. The next sub step the try to undo what the contact joint did but calculation an even higher force to negate the effect of the joint. The joint sees his contribution reduce therefore it increases the force even more and so on.

This process continues until one of the two things happens.
1-the number of iterations reaches the maximum allow by the solver, and the system terminate with a set of unbalanced force the translate to the jitter that you see. I suspect is what going on.
2-the force contribution calculate by one of the joint reaches the maximum allow by the row limit and that row is remove from the system matrix. Is this happening, it is happening when the number of iteration is relatively high, therefore it does no leave enough iteration left for the contact joint to calculate an accurate reaction force.

Basically the system is spending all the time reducing the system of constraint from and ill conditioned to a well behaved system. In general this system would lead to an explosion.
In real life a system like that will simple fracture, in a mathematical simulation the values grows until the numerical representation can no represent the actually (a numerical explosion)
In a real time simulation there is no explosion because the solver run for a finite number of iteration, termination with a incorrect solution.


Please test this, and let me know.

[Julio Jerez]

The solution to this problem, is to have a high lever system that can more accurate predict the position of a constrained body.

for example, say you are controlling a foot by a contraction of a bunch of powered joints.
you cannot simple get the target position of the foot and set the contractions to calculate the position.

Part of the AI system determining the foot position, is responsible for calculating legal foot positions.
It can do that but using ray cast, or convex cast. there are a set of function for that in the engine.

[JoeJ]

Part of the AI system determining the foot position, is responsible for calculating legal foot positions.
It can do that but using ray cast, or convex cast. there are a set of function for that in the engine.

That needs further explantation to me.
What i've planned myself to do in case there is not much you can do about it is:
Use a extruded proxy-body ('kinematic' ?) to get contact points for feet.
Use that information to create 3 linear joints - one to fight penetration, two tangential to emulate friction.
That way collisions would be handled in the same way as the joints, and it should work physical correct even for dynamic - dynamic collision.
I don't know if 'recreating' joints for each frame is good idea at all, or if it would need to be done each substep to get accurate friction etc.

Can you describe what you mean with 'legal foot positions'?
would that be simply a position that is non-penetrating anything?
I don't think so - if doing that the ragdoll could not balance.
You need to try to push your feet inside the floor... otherwise you'll fall on your ass soon


However... the CCD assert seems still there.
It's very easy to get it in my app, but i've had no luck replicating it in sandbox so far.
I'll first change to your dll to ensure i don't have wrong settings compiling newton.
Maybe it's necessary to attach the pick-joint AFTER creation of the scene (assert happens as soon as i pick a body resting on floor in my app)
Will take some time...

[JoeJ]

Ah ,sorry - i didn't realize that i need to handle the zero timestep in my own joints as well
I use a slightly different kinematic joint, that caused the assert still happening.
I'll fix that everywhere and do some more testing...

[JoeJ]

Ok, after smashing stuff arounf like crazy in my app (lots of complex joined stuff, but only primitive collision shapes - boxes, spheres capsules),
i can say situation is much better now. CCD works for bodies with no joints attached. With force based interaction no problems.
My scene is not practical to proove it effectively prevents tunneling.

But if joints are involved i still get two kinds of different asserts - very rarely - maybe my fault (!)
I'll post screenshots.

[Julio Jerez]

I have seem that assert, yes this is a problem in the convex collision the I ne to resolve. I have being ignoring but I guess it is time to fix it.
can you reproduce that bug on any of the sandbox demos?

Ah ,sorry - i didn't realize that i need to handle the zero timestep in my own joints as well
I use a slightly different kinematic joint, that caused the assert still happening.
I'll fix that everywhere and do some more testing...

yes you need to deal with the time step when the argument passed in is zero.

basically for most joints it comes to : the values:
invTimestep = (timestep > 0.0) ? 1.0/timestep : 1.0;

[Julio Jerez]

Ok now I apply the fix to all custom joints.

[JoeJ]

This is second assert.
It does not happen again after i disabled your implementation of fast float to int:

Code: Select all

DG_INLINE dgFloat32 dgFloor(dgFloat32 x)
{
#ifdef _MSC_VER_OFF
   dgFloat32 ret = dgFloat32 (dgFastInt (x));
   _ASSERTE (ret == floor (x));
   return  ret;
#else
   return floor (x);
#endif
}


Very strange is - after my computer froze, restarted, clean & rebuild i get the old assert again.
Seems files where properly saved.
Also other strange things are happening: if i turn powering completely off and touch the ragdoll with another body - all bodies disappear completely without an assert.
I need to check all this and will try to reproduce minkovski in sandbox tomorrow.

[JoeJ]

Very strange is - after my computer froze, restarted, clean & rebuild i get the old assert again.

I understand little better now. The old assert came back bacause i disabled the custom floor, thus the error creeps further down.
What happens is that for some reason (explosion?) the AABB calculation recieves very large numbers that don't fit into 32 bit int.
It happens only if CCD is on and lots of joints are involved (ragdoll). I'll see if i can reproduce with a simple scene...

[Julio Jerez]

the new newton does no have world size. if a body is flown out of the world at a very high speed, is will eventually move to a position where the aabb will fail.
I suspect this is what tis happening to you now, either a body penetrated the ground and in falling, or some huge velocity propel a body outside the world.

In core 200 the was a callback for when a body leave the world, I will see if I can re enable that with some fake world size.
at least it could is good for debugging.

[JoeJ]

I thik this will reproduce. Add code to the same file as yesterday, uses the same ground box.
I create 5 bodies i a row and connect them with joint in air.
As soon as they touch the ground, they disappear (without assert and without CCD on).
Either i do something totaly wrong here, or that's a real bug.

Code: Select all

#elif 1

   // provoke explosion bug
   dVector p (0,3,3);
   NewtonBody* body[2] = {0, BuildBox (scene, 1.0f, p, dVector (4,4,4))};
   NewtonBodySetContinuousCollisionMode (body[1], 1);
   for (int i=0; i<4; i++)
   {
      p[0] += 6;
      body[i&1] = BuildBox (scene, 1.0f, p, dVector (4,4,4));
      //NewtonBodySetContinuousCollisionMode (body[i&1], 1);
      dMatrix matrix (GetIdentityMatrix());
      matrix.m_posit = p;// + dVector(-3,0,0);
      CustomBallAndSocket *j = new CustomBallAndSocket (matrix, body[!(i&1)], body[i&1]);
   }



EDIT: It is CCD bug - i forgot to disable CCD for first body when testing.
If you do so, the snake works.