A feature request

A place to discuss everything related to Newton Dynamics.

Moderators: Sascha Willems, walaber

A feature request

Postby misho » Thu Dec 19, 2019 4:11 pm

I'm not sure if I'll get flak for asking, and if this is the right section, but I would really, REALLY like to have the following feature implemented:

A proper time compression and time dilation functionality.

Right now, the demo sandbox has a function

Code: Select all
void DemoEntityManager::UpdatePhysics(dFloat timestep)


which does what it is supposed to, however, to me it is quite convoluted and hard to read and follow. The function parameter, timestep, isn't even used in the function. My project (a spaceflight simulation) has a requirement for time dilation factor of 1/4 and 1/2, and time compression factor of 2,4,8,16,32,64 and 128. I have implemented, to the best of my ability, the code that does this, however, I am getting inexplicable and distracting "stuttering" and pausing in the motion when I set time compression factor to 8 or larger.

So, would it be too much to ask for this feature - a proper, parametric implementation of time compression/dilation? This feature is present in many modern games, especially simulations, and would be very useful to many applications where there are stretches of time that need to be compressed while some "boring" simulation part is happening (like transit through empty space, or grass growing :mrgreen: )
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Re: A feature request

Postby Dave Gravel » Sun Dec 22, 2019 2:29 am

Hi misho,

It is just my personal thinking about the subjet.
This feature is present in many modern games

(like transit through empty space, or grass growing)

This exemples don't really need physics simulation for become done.
I don't think real modern games use this for doing it or they is surely wrong to use the physics in this way.
It can cost a lot cpu and it's not all users that having a nasa computer hehe.
Or maybe it is not really done the same way that you think.
Maybe they just execute some math calculations in background to reproduce the effect.
It can cause so much glitch like penetration, total collision lose, joint explode and surely a lot more precision problems.

If a scene use a lot physics it can quickly go wrong with freeze, stuttering, pausing and bad result.

I have already test this method with vehicle simulation for try to get faster vehicle sim like arcade.
It work good with low value like you say but after some times I have quickly understand that is not a magic solution.
With low value it appear to don't cause problem but I have realise it is not perfect and it introduce some little glitch hard to see.
With higher value it start to cause real problem with very bad effect and result.

I'm not a expert but i'm pretty sure that the modern games with similar feature skipping some parts about the physics for do it.
Or if it is done with the physics it is surely not done with all features enabled.

The only place where I have see similar feature for real, It is in simulation like AI life reprodution, Or planetary space simulation.
Or exemple meteo simulation.
All of this systems don't use real dynamics physics simulation, They only use some physics maths but it is not using real bodies or joints or physics components.

I'm pretty sure that you are better to search about a other method for do this if you use a real dynamics physics system.

It is just my personal thinking and suggestion about the subjet, You can do what you like with it.
Sorry about my english for explain thinks.
Good luck with your cool project.
You search a nice physics solution, if you can read this message you're at the good place :wink:
OrionX3D Projects & Demos:
https://orionx3d.sytes.net
https://www.facebook.com/dave.gravel1
https://www.youtube.com/user/EvadLevarg/videos
User avatar
Dave Gravel
 
Posts: 800
Joined: Sat Apr 01, 2006 9:31 pm
Location: Quebec in Canada.

Re: A feature request

Postby misho » Sun Dec 22, 2019 3:01 am

Thanks Dave! I appreciate your thoughts on this, I really do.

Sure - I understand the ramifications of what you're trying to explain - a large time step will miss some physics events, perhaps an entire collision event. In those cases (and I've seen it handled this way before), if there were 2 (or more) bodies that were some pre-set distance away and have possibility of interacting (colliding), I would "kick" the user out of accelerated time mode into a "real" mode and let physics do their thing.

I remember playing a game way back in the late 90's, a submarine simulator called "Silent Hunter". With submarines, there are days and weeks of patrolling the open seas where absolutely nothing would happen... and sure enough, there was a time acceleration factor, where days passed in seconds.

However, every time an enemy was sighted, the time acceleration factor would kick back into "1x" mode.
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Re: A feature request

Postby Dave Gravel » Sun Dec 22, 2019 3:24 am

Maybe they don't use real full dynamics physics engine and only a simple collision system. It's hard to say for sure.
Or maybe i'm wrong.
You search a nice physics solution, if you can read this message you're at the good place :wink:
OrionX3D Projects & Demos:
https://orionx3d.sytes.net
https://www.facebook.com/dave.gravel1
https://www.youtube.com/user/EvadLevarg/videos
User avatar
Dave Gravel
 
Posts: 800
Joined: Sat Apr 01, 2006 9:31 pm
Location: Quebec in Canada.

Re: A feature request

Postby misho » Sun Dec 22, 2019 3:31 am

Dave Gravel wrote:Maybe they don't use real full dynamics physics engine and only a simple collision system. It's hard to say for sure.
Or maybe i'm wrong.


Oh, they absolutely didn't have the collision system, or if they did, it was a very simple "sphere radius" system. In fact, the game was pseudo-3D, because it only dealt with surface targets, there was no sub-to-sub underwater combat.

The point I was trying to make is the method that would force-kick the player back to 1x mode if something dangerous or interesting has a potential to happen. In the case of the sub game, in the large time step between the updates, an enemy ship could have spotted the submarine, fired on it and sunk it before the next update, and the user would never know what happened to his sub :mrgreen:
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Re: A feature request

Postby JernejL » Mon Dec 23, 2019 4:08 am

Just a short writeup on what timestep is in newton:

Newtonupdate function takes timestep as input parameter, the value is a float time passed and expressed in seconds. This value goes to each callback so that you know how much time passed as newton can splice this value into substeps when things like continous collision is used, then you use the value to calculate your forces to apply in the callback.

If you always have a timestep that is fixed, you do not need to use this value in callbacks at all, but this mechanism is not useable for time dilation.

To simulate time dilation you can simulate this yourself, because in my experience newton is so fast, you could even call simulation multiple times in 1 frame, and / or use increased forces and simplified physics to get bodies moving "faster" and make larger jumps, ofcourse collision detection will not work as good with such huge jumps, but continous collision might help with this - to prevent bodies from going thru each other. Or as you mentioned in other game, you just slowdown simulation to normal when you detect something important might start to happen.

I am not aware of any physics engine to have built in time dilation, usually this is simulated.
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Re: A feature request

Postby misho » Mon Dec 23, 2019 5:32 pm

Thanks for the write up!

Just to clarify, with respect to timestep parameter, I was referring to the unused parameter in void DemoEntityManager::UpdatePhysics(dFloat timestep), not the (most definitely) used parameter in NewtonUpdate (const NewtonWorld* const newtonWorld, dFloat timestep);

And yes, I implemented the dilation AND compression using the multiplier parameter for the timestep in NewtonUpdate (...)

Code: Select all
NewtonUpdate(g_world, timestepInSecunds * SystemVars->fSimSpeed);


where SystemVars->fSimSpeed is 0.25, 0.5, 1, 2, 4, ... 128. It does work up until value of 4. Values of 8 to 128 stutter. Now, I remember a few years ago, I had this implemented the same way, and my sim ran without stuttering through all of the values of compression and dilation. I also remember that at that time, Newton demo sandbox ran the physics at a much higher rate, ~200 to ~400 FPS if I remember correctly, and that may be the reason time compression ran smoothly. The max FPS for physics is at the moment locked to 60 FPS, using
Code: Select all
 #define MAX_PHYSICS_FPS            60.0f

When I try setting this to 400 to see if the time compression will work, my whole simulation slows down, apparently, by a 60/400 factor... and that's not how this was working a few years ago...

Again, I am not exactly sure how this works because I find it a bit hard to follow, but what I am hoping for is for the NewtonUpdate (const NewtonWorld* const newtonWorld, dFloat timestep); function to be cleaned up a bit and made more flexible, allowing for parametrized MAX_PHYSICS_FPS, and compression/dilation functionality. Like I said, I have the dilation and part of compression working just by passing a timestep multiplier. This could (I think) easily be added, if not to the main Newton functionality, then to at least the demo sandbox functionality, to show users how it is done and how they can implement it in their applications.
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Re: A feature request

Postby misho » Mon Dec 23, 2019 8:04 pm

So - just to check if this is worth doing, I decided to play around with the demo sandbox. As a test case, I used scene 13 (height field Collision). In DemoEntityManager.cpp, I adjusted the values of MAX_PHYSICS_FPS and introduced dTimeFactor multiplier in

Code: Select all
NewtonUpdate(m_world, timestepInSecunds*dTimeFactor);


and sure enough - ALL cases of dTimeFactor, from 0.25 through 128, worked! However, for cases above 4, I had to adjust MAX_PHYSICS_FPS upward, otherwise object collisions with heightmap weren't registering, the objects disappeared through the ground! In order to get the 128x factor working, I had to adjust MAX_PHYSICS_FPS to 1500 but it worked!! (a value of 2000 crashed the demo).

This is IMPRESSIVE! :mrgreen:

And that's exactly what I've been hoping for: a parametrized dTimeFactor and a variable (instead of a define) for MAX_PHYSICS_FPS, which adjusts itself according to time compression/dilation factor so that it properly registers collisions at all time speeds.

This test also makes me think that stuttering is something on my side after all. I'll have to create a test case closer to what I have (orbital speeds, gravity, masses, etc.) to see why they are occurring.

To add some spice to this discussion, there is a space simulator called KSP (Kerbal Space Program, perhaps you've heard of it, it is very popular and has millions of users). However, its physics engine does have the very exact problem. Here is a link to an interesting forum discussion on the very same topic, where users discuss how hard it is for it to deal with time compression.

That's a testament to Newton's robustness and speed! :mrgreen:
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Re: A feature request

Postby JoeJ » Wed Dec 25, 2019 5:33 pm

Did you try to just call NewtonUpdate() 128 times in a loop? :)
I guess for a space game with not much bodies it should be fast enough easily.
There should be a better way i guess, avoiding some redunant work maybe...


I also had a situation where the change of MAX_PHYSICS_FPS define got in my way, but can't remember why.
Maybe it has no real purpose and could be removed?
User avatar
JoeJ
 
Posts: 1453
Joined: Tue Dec 21, 2010 6:18 pm

Re: A feature request

Postby Julio Jerez » Wed Dec 25, 2019 6:01 pm

I was going to say if this was not just playing the physics update in a loop?
You just do that instead of passing large dt to the simulation update.
As long as you loop several time, you can control how long of a time you update. And it does not has to be a power of two, only an integer multiple of the time step.

That define I do not even remember what purpose was, I guess was from the time went physics libraries were a novelty, but now every body knows how they work and those checks are not longer nessesary.

In that regard the latest Newton removed almost all of the correction checks that we have in the pass like clamping high angular velocity, clamping extreme inertia, a some other I don't remember now.
Now bad setup will yield bad physic simulation. It is the user responsibility to get it reasonably settings.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: A feature request

Postby misho » Fri Dec 27, 2019 2:40 pm

Julio Jerez wrote:That define I do not even remember what purpose was, I guess was from the time went physics libraries were a novelty, but now every body knows how they work and those checks are not longer nessesary.

In that regard the latest Newton removed almost all of the correction checks that we have in the pass like clamping high angular velocity, clamping extreme inertia, a some other I don't remember now.
Now bad setup will yield bad physic simulation. It is the user responsibility to get it reasonably settings.


Thanks Julio - yes, that is basically my point. I would hope that this area of code in the demo would be cleaned up and "de-cluttered" with unnecessary defines and undocumented parameters and constants. This is the only area in the whole library that shows the usage of NewtonUpdate()/NewtonUpdateAsync(), and as such, I think it would be pretty important to the user to clearly see how this should be done, and what exactly the code is accomplishing. Another example of the ambiguous parameter in void DemoEntityManager::UpdatePhysics(dFloat timestep) is:

Code: Select all
         if (m_physicsFramesCount >= 16) {


I'm guessing this code calculates Physics Frames Count, but what is the significance of value 16?
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Re: A feature request

Postby misho » Fri Dec 27, 2019 2:43 pm

JoeJ wrote:Did you try to just call NewtonUpdate() 128 times in a loop? :)
I guess for a space game with not much bodies it should be fast enough easily.
There should be a better way i guess, avoiding some redunant work maybe...


Thanks, yeah, I haven't tried that, I guess that would be the next step, but as I said, passing large (and small, for time dilation) time step values appears to work for the demoSandBox. Now I just have to figure out why do I have stutters in my case...

JoeJ wrote:I also had a situation where the change of MAX_PHYSICS_FPS define got in my way, but can't remember why.
Maybe it has no real purpose and could be removed?


Yes, exactly, if that could be cleaned up and/or simplified, that would be great!
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Re: A feature request

Postby JoeJ » Fri Dec 27, 2019 4:29 pm

Ahhm... you sound as being that person why such 'safety guard' defines exist, if i assume correctly :)

misho wrote:I guess that would be the next step, but as I said, passing large (and small, for time dilation) time step values appears to work for the demoSandBox.

Probably it works in the sandbox because somewhere is a mechanism that subdivides the timestep for you, using a constant (like from that define) as the maximum.
So you take a helping hand for something you could do easily yourself, but then knowing what really happens instead just trusting great Newton and making assumptions it will just work.

But Newton can't know what you intend - only you do.

Some background:
To simulate multiple bodies and constraints in realtime it's a big advantage to use constant timesteps, so any game physics engine assumes you do this. (You likely know that famous 'fix your timestep' article.)

If you use a variable timestep, it can't be guaranteed the simulation remains stable. If you use joints that connect your ship parts they could start to oszillate, explode etc. Or, if there is no external force (like gravity causing resting contact) and equal velocities for the parts, it could just work by luck. (So that's maybe an opportunity of optimization if necessary, but let's see if it is necessary first)

What you usually want to do is having this constant timestep all the time, so things just keep working like they should without risk. That's why i suggested the loop - it's the obvious and correct solution. It's not the 'next step', it should be the very first thing you try at all.

Now it could be this is slow, for example if Newton creates worker threads each time one calls NewtonUpdate(), or any other expensive stuff - i don't know. If that's the case your request is justified, and an additional function like NewtonUpdateNTimes() would make sense to avoid such overhead. So try the loop, see how it works, and report if performance is suspiciously bad.

Using this loop you do not even need to know what goes on internally, otherwise you must know. You could use the debugger and follow if and how newton subdivides your timestep. I assume you do not use DemoSandbox code in your project, so its irrelevant and only NewtonUpdate() matters.
User avatar
JoeJ
 
Posts: 1453
Joined: Tue Dec 21, 2010 6:18 pm

Re: A feature request

Postby misho » Fri Dec 27, 2019 9:21 pm

JoeJ wrote:Ahhm... you sound as being that person why such 'safety guard' defines exist, if i assume correctly :)


Yes, I am not ashamed to admit that I am a bit of an "end user", especially when code exists that apparently performs as intended (as it does for the sandbox demos). However, when I need to implement something that is not implemented by default, I start to poke and test, and break things, and when I hit a brick wall, I ask questions here :mrgreen: ... also

JoeJ wrote:I assume you do not use DemoSandbox code in your project


I actually do, verbatim, line for line, I use the same code as in void DemoEntityManager::UpdatePhysics(dFloat timestep) of the demo SandBox. I literally have no more than 30 objects at one time with very simple rigid links, so I would assume that the demo sandbox is a good testbed - I am not doing anything exotic (except perhaps my gravity vector points in different direction depending on where the object is, as opposed to constant downward vector in the case of all of the demos).

JoeJ wrote:You likely know that famous 'fix your timestep' article.


I do, but it has been a while since I needed to implement anything like that myself. And indeed, I see that UpdatePhysics(dFloat timestep) is a variant of the code suggested in the article.

So, sounds good, I'll implement the loop method and report back here on my findings!

Thanks!
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Re: A feature request

Postby misho » Sun Jan 19, 2020 1:36 pm

JoeJ wrote:What you usually want to do is having this constant timestep all the time, so things just keep working like they should without risk. That's why i suggested the loop - it's the obvious and correct solution. It's not the 'next step', it should be the very first thing you try at all.


Alright - I have revisited this. I have inserted the subdivision of NewtonUpdate() calls as such:

Before:

Code: Select all
NewtonUpdate(g_world, timestepInSecunds * SystemVars->fSimSpeed);


After: (this won't work with SystemVars->fSimSpeed < 1 but I'll deal with it later )

Code: Select all
for(int i = 0; i < SystemVars->fSimSpeed; i++)
     NewtonUpdate(g_world, timestepInSecunds);


The result is - no difference at all. Two pieces of code perform exactly the same: I get stutters at SystemVars->fSimSpeed >= 8.

Also, I re-read the fix your timestep article, and I can follow the logic behind it. The code in demoSandbox (which is the same code I use) is very close to the second last iteration of the code in the article (the final iteration deals with SLERP). I even implemented the code from the article just to test if it works in my case, and it does, except, again, I get stutters at 8x and above, just like the Newton code. Since both approaches give me identical results, and especially since the time compression/dilation works in demoSandbox, I have to assume that the code I am using for the time step is correct, and the problem I have is perhaps in the way I am calling it

So, unless something glaring and obvious is wrong with my approach above and someone can point it out, I'll go over my code and put together a graphical flowchart of how my app is structured. Perhaps that will shed some clarity on what is going on and where I might have gone wrong...
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 673
Joined: Tue May 04, 2010 10:13 am

Next

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 14 guests