Forces not applied

[zak]

I was on a July 2021 Newton4 version. Now I have upgraded to stable release and am having a strange bug. There is a simple ragdoll. When it walks a foot body - ground body joint is added to the world and then removed when the foot is lifted.
In the step where the joint is removed with ndWorld :: RemoveJoint ()
forces that I applied in ndBodyNotify are ignored by the solver. In the next step it works.
It seems to me that when the skeleton is assigned dirty for joint removal and then needs to be rebuilt, a bug cleans forces assigned in ndBodyNotify
If i use all IterativeSoft joints bug is not present.

[Julio Jerez]

Recent for uptimization reasons, three loop where fused together.
Build active body array, force and torque, and update aabb.
But I do not think that is the reason of the problem.

the dirty Skeletons, are re generated at the start of the update. The reason fir this is that is uses the same array the the solver uses.

The other point is the delay removed.
You say you removed the joint when the foot is lifted,
Again, I assume you removed it from a contact call back.
It make be possible that remove joint is not deactivating that joint, but if that is the case, them the same fix of rage eating the Skeleton after collision should fix it.

I take it you are creating the joint in a contact callback.

I will move the refenerator to the point afte the collision phase and before models update.

Fir that I will need to make the Skeleton builder use it own working buffer.

And I correct assuming you are making the joint in a contact callback?

[zak]

I'm not making the joint in a contact callback.
Foot/floor joint is added/removed in application before physical update. The floor is flat with no obstacles so if the y coordinate of the foot is below a value I add the joint, when it is above I remove it. Bug happens in the step where joint is added o removed. If foot joint is IterativeSoft bug is not present.
So:
STEP X
1) ndWorld::AddJoint()
2) ndWorld::Update()
BUG, forces in ndBodyNotify are ignored even for a simple body present in the simulation without joints and that is not part of the ragdoll

STEP X+1
1) ndWorld::Update()
OK

STEP X+2
1) ndWorld::RemoveJoint()
2) ndWorld::Update()
BUG, forces in ndBodyNotify are ignored even for a simple body present in the simulation without joints and that is not part of the ragdoll

STEP X+3
1) ndWorld::Update()
OK

[Julio Jerez]

oh wait, maybe is not that difficult. before I modify a demo to repro
in this test,

STEP X
1) ndWorld::AddJoint()
2) ndWorld::Update()
BUG, forces in ndBodyNotify are ignored even for a simple body present in the simulation without joints and that is not part of the ragdoll

STEP X+1
1) ndWorld::Update()
OK
.....
.....

Newton 4 runs async on its own thread,
to make run sync you should do something like

I think you should do something like this.

Code: Select all

ndWorld::Update()
ndWorld::Sync()


this will force the async tread to wait until the frame complete,
please try that and see if the bug goes away.


if that works, a more sophisticated method will be to sync when you are going to make the modification.

Code: Select all

STEP X
if (addorRemoveJoint)
{   
      0)world.Sync()
     1) ndWorld::AddJoint()
}
2) ndWorld::Update()
3) ndWorld::Update()
...
...

if (addorRemoveJoint)
{   
      world.Sync()
      ndWorld::RemoveJoint()
}
ndWorld::Update()


I can see how this can make more complex,
a better way is to add the joint in the callback, but for that you need to wait until I move the skeleton update to after the collisions.

and an even fat better way, if to just add the joint to your rag doll, and in the joint callback you check is the joint should be active or inactive. to make the joint inactive you just do not send any row to the solver. but let us try that after you try
0)world.Sync()

[zak]

Newton 4 runs async on its own thread

oh yes, I always call sync after an update.
So:
STEP X
1) ndWorld::AddJoint()
2) ndWorld::Update()
3) ndWorld::Sync()
BUG, forces in ndBodyNotify are ignored

STEP X+1
1) ndWorld::Update()
2) ndWorld::Sync()
OK

STEP X+2
1) ndWorld::RemoveJoint()
2) ndWorld::Update()
3) ndWorld::Sync()
BUG, forces in ndBodyNotify are ignored

STEP X+3
1) ndWorld::Update()
2) ndWorld::Sync()
OK

Note:
-forces in ndBodyNotify are ignored even for a body present in the simulation without joints and that is not part of the ragdoll
-the floor is the sentinel body, so no collisions at all are present
-if the joint is set IterativeSoft bug is not present
-bug is not present in the same code compiled with Newton4 July 2021 version

and an even fat better way, if to just add the joint to your rag doll, and in the joint callback you check is the joint should be active or inactive. to make the joint inactive you just do not send any row to the solver.

Yes this works, i have done this in the past, but it is 10% slower, so i add/remove foot joint when ragdolls walks or runs. However this only hides the bug which is still present and could cause different problems that we haven't noticed yet.

[Julio Jerez]

Note:
-forces in ndBodyNotify are ignored even for a body present in the simulation without joints and that is not part of the ragdoll
-the floor is the sentinel body, so no collisions at all are present
-if the joint is set IterativeSoft bug is not present
-bug is not present in the same code compiled with Newton4 July 2021 version

oh you mean even, when not joints, ok that's good info.
I will check it out now them.

[Julio Jerez]

to clarify, in one of the test demo, I generate just one body

the in the update I added this

Update(descreteStep);
Sync();

after the body is created when I step, over Update, I do not get any call to
void ndDemoEntityNotify::OnApplyExternalForce(ndInt32, ndFloat32)
but when I go over Sync(); I do get the call.

is you claim that the It gets the call but that the forces are ignored?
or that it does no gets the call at all.

[zak]

OnApplyExternalForce() is regularly called but forces are ignored

[Julio Jerez]

this is very strange, the behavior you are describing is what will happen when I add the deferred add body, joints, and models.
by I have not done that yet.

bodies are added immediate, what there are added form outside an update.

I modify the sandBox so that is play the rigid bod demo, it only add one capsule.

the I set a break point on these lines:
bool ndScene::AddBody(ndBodyKinematic* const body)
void ndDemoEntityNotify::OnApplyExternalForce(ndInt32, ndFloat32)
void ndBodyKinematic::IntegrateExternalForce(ndFloat32 timestep)

and I do get the call one after the other in that order when creation just that one capsule.

let us this:

first let us rule out async update so that it is easier to debug. in cmake set define
NEWTON_BUILD_SINGLE_THREADED
true and build the engine.

this will let you run the engine in the same thread that create it.
fir see if the bug goes away.

if not Sync to github, I set that test for one body and I do not see that problem, see if you can see it.

if you do not, then you are doing something that is causing it maybe you can, make a test that recrate the behavior using the Test build.
in cmake select NEWTON_BUILD_TEST and make the engine, this create the simpler demo IO can this

Code: Select all

int main (int, const char*)
{
   ndWorld world;
   world.SetSubSteps(2);

   ndVector size(0.5f, 0.25f, 0.8f, 0.0f);
   ndVector origin(0.0f, 0.0f, 0.0f, 0.0f);
   BuildFloorBox(world);
   BuildSphere(world, 1.0f, origin + ndVector(3.0f, 0.0f, 0.0f, 0.0f), 1.0f, 1, 1.0f);
   
   ndFloat32 totalTime = 0;
   for (ndInt32 i = 0; i < 10000; i ++)
   {
      world.Update(1.0f / 60.0f);
      totalTime += world.GetUpdateTime();
      newton.Sync();
   }
   return 0;
}

off, make you can recreate teh bug there

[zak]

The bad
bug is also present with NEWTON_BUILD_SINGLE_THREADED
The good
I think I've found the problem: when a joint is added or removed, with substep = 1, ndBodyNotify::OnApplyExternalForce() is called twice for each body !
At the first call my code assigns the forces to the body and initialize them to zero for the next step, but at the second call it assigns the forces setted to zero to the body and this made me think that the forces were ignored.
The bug can be easily reproduced with ndTest
1) Declare the two global

Code: Select all

ndBodyDynamic* bodyFloorBox;
ndBodyDynamic* bodySphere;

and assign them in the last line of BuildFloorBox()

Code: Select all

bodyFloorBox = body;

and BuildSphere()

Code: Select all

bodySphere = Body

Then modify main() creating a joint and add and remove it
Set also world substep to 1

Code: Select all

int main (int, const char*)
{
   ndWorld world;
   world.SetSubSteps(1);

   ndVector size(0.5f, 0.25f, 0.8f, 0.0f);
   ndVector origin(0.0f, 0.0f, 0.0f, 0.0f);
   BuildFloorBox(world);
   BuildSphere(world, 1.0f, origin + ndVector(3.0f, 0.0f, 0.0f, 0.0f), 1.0f, 1, 1.0f);

   ndJointBallAndSocket joint( bodySphere->GetMatrix(), bodySphere, bodyFloor );

   ndFloat32 totalTime = 0;
   for (ndInt32 i = 0; i < 10000; i ++)
   {
      if ( i == 0 ) world.AddJoint( &joint );
      if ( i == 2 ) world.RemoveJoint( &joint );

      world.Update(1.0f / 60.0f);
      totalTime += world.GetUpdateTime();
      world.Sync();
   }
   return 0;
}

with 1 substep ndDemoEntityNotify::OnApplyExternalForces() should be called 2 times for each step, one for bodyFloorBox and one for bodySphere but in step 0 and in step 2 it is called 4 times, 2 times for each body.
if you set a breakpoint on world.Update(1.0f / 60.0f) and another on ndDemoEntityNotify::OnApplyExternalForces() you will see it.
Note
If joint solver mode is set m_JointIterativeSoft bug is not present
i have setted world.SetSubSteps(1) but with 2 substep the bug is still present, ndDemoEntityNotify::OnApplyExternalForces() should be called 2 times for each sub step ( 2 + 2 ) but in step 0 e 2 again it is called 4 times on first substep e regularly 2 times on second substep ( 4 + 2 )

[Julio Jerez]

The bad
bug is also present with NEWTON_BUILD_SINGLE_THREADED

That's actually good considering the alternative of the kind of bug that could be and that mode is very good for debugging no multithread problems.

I will recreate the repro in the test.cpp later and debug it.

[Julio Jerez]

oh yes I see It, set up the repro and yes you are right, I am getting four calls.

It is check in, and I will debug it later today.

ok I see the bug, now.
is this in void ndWorld::UpdateSkeletons()
that function calls m_scene->InitBodyArray(); to determine the active bodies.
but them the update loop also call ndWorld::UpdateSkeletons()
but since get active bodies, calculate AABB and force and torque call back were fused in the same loop, we now get that double call.

I guess it is time to do what I said before, move the Skeleton update to after collision update, and have that function using its own memory so is does not have to call InitBodyArray()

it is easy to fix.

[Julio Jerez]

ok please try again.

[zak]

OK, it's fixed now