Simulation speed changes for unkown reason

[Julio Jerez]

Tel me something when you spawn a unit and the others slow down, do they stay slow if you do no spwn more, or do they speed up again by themself?

[agi_shi]

I've been trying to reproduce this bug in a minimal test case, however without success. In your game, OP, when at the SetFreezeState calls generally made? I want to get the same flow as your game in terms of calls. Is it something like:
- create body
- set freeze state 0
- create body
- set freeze state 0
or what?

[Julio Jerez]

I believe I found it. Mark calls SetFreezeState a lot,
Man this game really is a stress test for Newton I count more the ten worlds and a I stop counting many of them has 900 plus bodies,
and the all almost never go to slep because of al the call to Invalidate cache and SetFreezeState
After we find the Bug, we really need to do streamline some of that.
After all the goal is to make a good looking and fast game not to prove that the physics engine is very fast.

I am glad I am getting such a stress test because it give me the chance to fix bug that otherwise will be very hard to find.

The good news is that I found the Bug. It is a legacy form 1.5x
The function SetFreezeState is a recursive function that search for bodies in close proximity, and because the bodies are all on the terrain, the function check all 900 bodies every time a body in inactive

I believed I documented in 1.53 the it can severally affect the Engine performance,

But the good news is that I can fix that to because Static bodies do no need to be Unfreeze, and the will cut the search to only close body who are dynamics.

[Marc]

Yes, when I remove all calls to NewtonBodySetFreezeState() the simulation never slows down to slow motion. I also don't have this issue when I switch to some other engine. Practically I could live with removing all those calls. I thought they would make the simulation faster, by freezing bodies immediately when I know they have to stop, but it's hardly measureable. But it's odd anyways.

When I create a body which is dynamic but I know it's not moving initially, I immediately freeze it after creation, like agi_shi guessed. But I also freeze them when they are walkgin towards a given target position and got within 1m of it and their velocity is below some threshold. And I also unfreeze the body when nothing is touching it but the ai decides to start walking. I think this last unfreezing scenario is a bit problematic in combination with the practical solution to remove all calls to NewtonBodySetFreezeState because when a body is autofrozen, the ai would still have to unfreeze it? I'm not sure. Why it's working anyways is because I recreate the world from time to time or call NewtonInvalidateCache() which makes them active again I guess and so they start moving a little bit later.

The slowdown effect is exactly coupled to some bodies. When no such body is in the scene, everything is fast. As soon as a body gets in the scene which makes everything slows down, it stays slow until it gets removed from the scene or freezes. When it unfreezes the scene gets slow again. It doesn't just get faster again over time. And I have no idea what causes a body to be such a slow downer. I always create them similar. "All creatures are equal, but some are more equal than others!" ...

@Julio: I'm glad you like my game/stress test I'm also happy that you care about my issues like that and put so much effort into this. Thx

[Julio Jerez]

Ok I put the new version 2.08 with NewtonBodySetFreezeState fied, can you try again please.
I did not see the slow down and I get steady 52 fps, by I do no knwo hwo to play you game very well, all I do is hit n and m repeatly.

BTW that function now work in constant time like is should.
Let us see if that was teh Bug.

I also tesed tteh determonism an dit seem to work find as well. so we have tow three big bug fixed so far plus a big performance booth

[Marc]

Ok great. I'll test that.

I put the perfcounter code in there:



I'll make a toggleable graph in the game.

[edit]Oh the picture gets truncated on the ride side. Here is the full pic as a link: http://www.confrontation-unlimited.net/temp/newtonperfgraph.png [/edit]

[Julio Jerez]

Ha very good Theg counter are good for tunning the client inegration of the application and with teh engine, for example
see when a call back is taking too much time and stuff liek that.

we can see the spikes are in the collision system, it looks like the solver is fine.
That makes sence since Invalidate is waking the system up periodically,

The small spikes you see in the Dynamics update are the consecuence of Invalidation the Cache,
it cases the solver to regenerate all that data the is modified incrementanly, but the Invalidate cache throwe aways in one go.

Question I asume you are calling Invalidatecache form outside teh NetwonUpdate call, is that correct?
we need to find prescisetly what funtion is cause the spikes.
can you make another graph showing the the NarrawPhase and the BroadPhase only, with and witout Invaldate cache and maybe with and withoput NewtonBodySetFreezeState.

Oh I forget, did the last 2.08 solved the slow down bug?
we need to get the bug taken off before we deal with the optimizations.

[JernejL]

Marc: The pics get truncated only on lower resolutions, it is so it doesn't ruin the forum layout.

[Marc]

@Delfi: ic

I tested the new version of 2.08: Now it's always slow. Maybe I'm using NewtonBodySetFreezeState() wrong and the case when it run smoothly before was because of the bug?

[Julio Jerez]

My guess is that it is moving like it is support to, is the motion fluid?
can you very verify with and without the call, it should move more of less at the same velocity.

the Collision Update bar should be below 1000 at all time, for these scene, it is way to high.

[Marc]

Question I asume you are calling Invalidatecache form outside teh NetwonUpdate call, is that correct?

Yes.

[Marc]

Hmm I switched the parameter of NewtonSetFreezeBody everywhere from 1 to 0 and 0 to 1 and it seems like it's ok then. Can you verify that

NewtonBodySetFreezeState(body, 0) unfreezes the body
NewtonBodySetFreezeState(body, 1) freezes the body

instead of

NewtonBodySetFreezeState(body, 0) freezes the body
NewtonBodySetFreezeState(body, 1) unfreezes the body

?

Surprisingly I already did it the other way around in the deteminism code I showed you. I remember I toggled that value several times ...

What I don't get: even if that's true, my ai would have froozen a body whenever it actually wanted it to move. So why did it move at all? I update the ai with a fixed timestep equal to NewtonUpdate, so there is always newtonupdate, aiupdate, newtonupdate, aiupdate ... and every ai update I tried to unfreeze the body. So it should have been frozen all the time when it wanted to move and never move at all.

[Julio Jerez]

Hmm I switched the parameter of NewtonSetFreezeBody everywhere from 1 to 0 and 0 to 1 and it seems like it's ok then. Can you verify that

NewtonBodySetFreezeState(body, 0) unfreezes the body
NewtonBodySetFreezeState(body, 1) freezes the body

instead of

NewtonBodySetFreezeState(body, 0) freezes the body
NewtonBodySetFreezeState(body, 1) unfreezes the body

I do not remember I have to verify,

Basically the idea is that you can stop the body and all bodies conected to that body from moving even if they are not in equilibrium.
a way to find out is to drop a body in free fall, and call NewtonBodySetFreezeState when the body is in the air, it should stop in the air until another body touchs it.

[Marc]

Ah maybe the reason why it moved a little bit was that when I do a NewtonInvalidateCache() I do it at that position:

NewtonUpdate
AIUpdate
NewtonUpdate
AIUpdate
NewtonUpdate
AIUpdate
NewtonInvalidateCache()
NewtonUpdate
AIUpdate
NewtonUpdate
AIUpdate

And so 1/8th of the time, The ai froze the body, then InvalidateCache unfroze it and NewtonUpdate moved it.

[Marc]

With the flipped values, the graph now looks like this: There are some in between half high spikes, no idea why.



And here are only the collision graphs:



COLLISION_UPDATE_NARROW_PHASE actually is 0 all the time. I don't know if that's wrong or if it's actually that fast. But if it's ok, there is some other big thing that takes time in Collision than just boradphase and narrowphase

Here is the csv file: http://confrontation-unlimited.net/temp/newtonperfgraph1.csv