Simulation speed changes for unkown reason

[Marc]

Hi !

I'm not sure what causes this, but this happens for me from time to time and I think it's related to newton or how I use newton (maybe somehow wrong at some point I don't know yet): From time to time, my simulation slows down, meaning eveything behaves like normal, but much slower. This happens deterministically as well, when I rewind the simulation and run it again it slows down at the same occations. It's also not more or less cpu intensive.

Does anyone have any ideas what could cause this? I tried the different friction and solver models and it happens with all of them. I tried not to call NewtonInvalidateCache() and loosing determinism but the behavior still occurs.

Basically, my scene consists of a static heightfield, several static primitives and some moving spheres with an upvector joint. And it seems to be bound to some of the moving spheres: When this occurs, there is at least one body in the scene which, when it moves, causes this effect. When it rests, the simulation runs fast again. When it starts moving, the simulation slows down again - and not just the surrounding area/island, the whole simulation.
I create those moving spehers all with the same code, so I can't see what makes them different like that.

My code is somewhat old, I started when 1.53 was new. With that version, I didn't have this effect. It's new for me to 2.x. So I assume, it's somehow related to newton/the way I use it.

I don't really know how to create a testcase for this like I did for the determinism issues. I could try and make a video - haven't done that before though

[edit]
Maybe it's related to my calls to NewtonBodySetFreezeState(). I call that function when I create a new body and I know it's resting from the start. I also call it during my AI calcuations when I know/want the body to rest - basically when an object reached it's destination and has to stop there and the distance to that point is below some threshold and the bodies speed as well. I tried removing all my calls to NewtonBodySetFreezeState() and from my first try, it doesn't seem to happen anymore.

Besides I can't find docu for that function. I just guessed NewtonBodySetFreezeState(body, 0) is the replacement for NewtonWorldFreezeBody(world, body).
[/edit]

[JernejL]

Which version of 2.0 do you have? i know that 2.02 or 2.4 had this defect - it had to do with body material contacts, make sure you are up to date.

NewtonBodySetFreezeState is not really neccesary anymore because of autosleep, with autosleep stationary objects go to freeze and unfreeze automaticly if you apply forces, autosleep is on by default. (1.53 only called forcetorque callbacks when not sleeping, 2.0 always calls it and uses it as basis when to un-sleep the objects).

if 2.07 doesn't resolve the problem you should make a testcase for Julio.

[Julio Jerez]

Hi !
I'm not sure what causes this, but this happens for me from time to time and I think it's related to newton or how I use newton (maybe somehow wrong at some point I don't know yet): From time to time, my simulation slows down, meaning eveything behaves like normal, but much slower. This happens deterministically as well, when I rewind the simulation and run it again it slows down at the same occations. It's also not more or less cpu intensive.
[/edit]

After I mad the optimization of eth collsion system I saw that too, is becume slow fo one frame, but I have not bieng able to reproduce it.
In is no in teh collsiion system but I cna no fogure out wher ist is, Thsi was no in 2.04 and lower.

you say it happens deterministically, can you modify thedemo to reproduc it?

[Julio Jerez]

Which version of 2.0 do you have? i know that 2.02 or 2.4 had this defect - it had to do with body material contacts, make sure you are up to date.

Delfi had you have tah problem before?

[Julio Jerez]

Maybe it's related to my calls to NewtonBodySetFreezeState(). I call that function when I create a new body and I know it's resting from the start. I also call it during my AI calcuations when I know/want the body to rest - basically when an object reached it's destination and has to stop there and the distance to that point is below some threshold and the bodies speed as well. I tried removing all my calls to NewtonBodySetFreezeState() and from my first try, it doesn't seem to happen anymore.

Besides I can't find docu for that function. I just guessed NewtonBodySetFreezeState(body, 0) is the replacement for NewtonWorldFreezeBody(world, body).

In very rare ocation I seen that spike, and I had set many diffrent test trying to reproduce without luck, But all my test use autosleep off

were you able to verify is this is what NewtonBodySetFreezeState causes the slowdown?
could you do some more test to verify that NewtonBodySetFreezeState is causing this?
make sure you are using 2.07

[Marc]

I'm using 2.07. I've noticed this since I started using 2.0x, but I'm not exactly sure which was the first version I've tried, maybe 2.02 or 2.03.

Since I added the edit to my initial post that I removed my NewtonBodySetFreezeState, I haven't seen it again. I've been working and debugging on the project since then, so if it's still there, it occurs a lot less often. So it seem to me that it's related to these calls.

Btw. I don't touch the autosleep settings when setting up my scene, so I use it in its default setting, I think the default is on.

If that information isn't enough, I can try to do a testcase - but up to now, I have no idea how to do it in a small testcase code. Atm, I see it happening - with NewtonBodySetFreezeState calls enabled - in my rather large project. I doubt you want to debug this issue using that project. Maybe placing a single box in an empty scene with gravity, letting it fall and calling NewtonBodySetFreezeState from time to time and plotting a graph of the z position of that box does the trick - I don't know yet.

[Julio Jerez]

Since I added the edit to my initial post that I removed my NewtonBodySetFreezeState, I haven't seen it again. I've been working and debugging on the project since then, so if it's still there, it occurs a lot less often. So it seem to me that it's related to these calls.

I suspect that that might be the case, because I have seeing it in some occasions as a big spike in the profiles,
But each time I set a test to reproduce it I set auto sleep off and I do not see it happing again.
Now I will try not setting auto sleep off and see if I can reproduce more reliable that way.

It is almost like a external process is launched in windows and I thought it was Vista problem since no one else had say anything.
I started to see it after I installed Vista Ultima in my machine, but now I know it is not a Vista problem.

If you have a demo that can reproduce it, and you want to cooperate to fix it faster, I think I can debug it from a running program that uses the DLL.
To me this is too big to let if pass and I would like to have it fixed.

Also I am going to ask you a favor, Please when you are position a Bug can you not make reference to other engines, I don’t really want to have any references to any other technology in this forum, Good or Bad, The Internet scrollers use any keyword to another thing to make Google and Yahoo entries which some people out there use for other purposes.
I removed the reference to another library, since it does not add anything to the solution of the problem.

[Marc]

OK I'll keep that in mind next time when posting here. I didn't mean to create any trouble by doing it.

Is NewtonBodySetFreezeState(body, 0) supposed to freeze a body or NewtonBodySetFreezeState(body, 1) ?

I don't have a small demo program, but I have my app where this happens from time to time noticable. For sure within 5 min. Is it enough to give you the binary that uses newton.dll or do you need the whole src code and dependencies etc.?

[Julio Jerez]

I do not need teh source just a binary that uses the DLL

ah I have some good news, you are not going to beleive this, but I set the compiler setting option from Precise to Fast in visual studio, and the engine is 30% faster.
It is unbeleivable, This do not solves the problem by it is very good news specially for thoso using the x87 mode for mutiplayer.
I do not remember why I am using the precise option for math, by 30% speed up is not something to ignore.

I will make a video of the before and after.

[hpesoj]

ha I have some good new, you are no going to beleive this, but I set the compiler setting option from Precise to Fast in visual studio, and the engine is 30% faster.
It is unbeleivable, This do not solves the problem by it is very good news specially fo thoso using the x87 mode.
I do not remember why I am using the precise option for math, by 30% speed up is not something to ignore.

That's good news. I suppose this means that the Windows library is now more than twice as fast as the Linux library ?

[Julio Jerez]

And that is not all, the windews 64 bit version is literatly twice as fast as the same version when compiled with precise float.
This is the first time I see 64 bit code being measurably faster than 32 bit code, It really is faster by 50%

The same scene in 32 bit take 16.6 miliseconds, in 64 bit it only take 12 milisencond,
and with precise float they both were taking 24 milisiconds, never I have seen such a big bang like this by doing nothing.
Now there is a compeling reason to use the 64 bit version, because it is 50% faster.

In the linux front I guess the same thing will apply since I use options -ffloat-store that Guarantee IEEE compliance, maintly what that does in that prevent
the compiler from optimising code by holding intermediate floating point values in registers, instead it saves then to memory to truncate rounding error but that apparently makes it very slow.
In reality the precise version is less precice than the fast, but it is not IEEE since floats registers have more bits than memory,
anyway this in very good, and if we see some errors we can deal with them as we need it.

I will make a video to show the difference

[Julio Jerez]

Ok here is the video that show the differences, the 64 bit si not 50% faster it is about 20% to 30% faster but that's a very nice thing.
http://www.newtondynamics.com/downloads/NewtonOptimized.wmv

This goes to show that you should never believe what the self appointed experts of Game Developer net and some forum say.
Had you ever her these people saying that 64 bit code is slower, but when you think about there is absolutely no reason why 64 bit code to be slower that 32 bit code, why?
The only thing that is different in 64 bit code are pointers, everything else remain the same. Why do this people keep saying that there is no reason to use 64 bit code, does that seems like an idea that come from where the sun do no shine.
I would expect 64 bit code to be from equal to 20% faster than 32 bit code, but until know I never saw it myself, but now it we have it.
BTW this is all x87 code, SSE code is even better.

I will wait for Marc's demo to see if I can find the slowdown bug before I put 2.08

[Julio Jerez]

That's good news. I suppose this means that the Windows library is now more than twice as fast as the Linux library ?

AGGhh I just tried Linux without the option ffloat-store and it is the same speed as before.

The scene in the video whic is 240 staked blocks at full speed (sse, one core, 64 bit) now run in 8.5 milisecond per frames with Visual studio
using GCC 4.3.1 is run in 17.5 milisecund per frames, and with SSE is about 16.5, you are right GCC deliver about half the performance of Visual studio 2008
Plus GCC is atrociuos at optimising code with simd intrinsics.

I think the only one solution for Linux is using The Intel compiler. I will download the trial version tonight and see what I get.
We can not have Linux being so bad compares to the PC can we.
I am hoping the Intel compiler give it some more kick, but it is my expirience that Intel always promise more than what the can deliver, so we will see.

[hpesoj]

You have checked all of these options?

http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html

[Julio Jerez]

I tried all that stuff, any option other than -02 is slower.
In fact I do not really see any appreciable difference beween O0 and -O2 in GCC.
GCC is a run of the mill compiler that produces mediocre binaries, I never see it buiding decent code that can really compete with Visual Studio
or with the front runner compiler of the time, and I go back to 1995.
Back then Watcon was the best and GCC never came even close to half the speed of Watcom, but I guess for a free product it is OK.

I will try Intel and see what I can get. My guess is tha Intel now is amost as good as Visual Studio.
Last time I tryed was in 2003 with Intel 8.1 and Visual studio 2003 was better but not by much.
So my guess is that it will be much better than GCC for Linux and that can make it competive with the windows build.