Evaluating newton-4.00 for integration with Godot

[geekrelief]

Hi!
TLDR: I'm considering integration of ND with Godot. I'm comparing it against Rapier3D. Does ND have the ability to snapshot and replay deterministically?

I just discovered newton-dynamics and yesterday got a chance to play with the sandbox demos. Despite some issues, https://github.com/MADEAPPS/newton-dynamics/issues/234 I'm impressed so far!

I'm developing a physics based game using the Godot engine, which unfortunately / fortunately is undergoing a rewrite of its physics engine. https://godotengine.org/article/physics ... s-report-1 Bullet 2 has been integrated into Godot for 3D, but there's a whole host of issues related to it, and it's going to be replaced in Godot 4.0 by a custom one. I have limited visibility on when he'll complete the task. And given Godot's funding situation, it'd be terrible if the physics replacement is left in an unfinished state.

After a bit of research I found out that Demindiro had done some work integrating Rapier3D https://www.rapier.rs/ with Godot https://github.com/Demindiro/godot_rapier3d. https://youtu.be/yHeZfur1ivs Rapier has some great features over Bullet 2, better performance https://youtu.be/T4r_XUzX3Jc, better stability https://github.com/extrawurst/godot-vs-rapier, cross-platform determinism, snap shotting / replay which seems like that'd make be a great choice for debugging user reports or networked game. In my limited testing with a couple of friends, they've gotten in same results as me. Debug and release builds look identical except for performance.

Checkout the demos for the snapshotting https://rapier.rs/demos3d/index.html

But after using the godot_rapier3d binding I ran into several issues.
1. It's not a complete binding. There are missing features with regards to ray casting, kinematic control, interaction with godot. And recently I discovered a stuttering issue while running in the Godot editor. https://github.com/Demindiro/godot_rapier3d/issues
2. It's in Rust, and so I'd have to learn Rust to fix these issues / contribute. (not that I'm opposed to it, but it's another step). And to integrate with Godot it has to go through GDNative which is Godot's C interface plugin architecture. I'm familiar with GDNative too as I use Nim as my script binding, and debugging through it is not pleasant when trying to interact with Visual Studio since Godot is written in C++.
3. Rapier is relatively immature even though the author seems to have a good reputation as the author of nphysics and other physics and math related libraries.

Even despite these issues, Rapier3D is still a contender because of its performance, determinism, and some work has already been put into it.

But newton-dynamics was mentioned on the Nim discord, it looks like it has everything that Rapier offers except for the cross-platform determinism which doesn't seem to be a mentioned feature in any other physics engine. Is it possible to snapshot the state of the newton-dynamics engine and send it to someone else's machine which may have different hardware and replay it deterministically?

It's a long shot, but has anyone tried to integrate ND with Godot?

[Julio Jerez]

well by what you described, I am afraid you are not going to like what I am going to say.

first what makes you think that start from scratch with an unknow to you, library is going to solve those problems you are facing with the native solution in GODO, and even if it did, it will probably bring brand new problems.

on the determinism. Newton in the pass was deterministic. but very soon that claim went out of the windows the moment we start introducing multithread and different solvers that use different instructions set. and we have never claimed to be deterministic across different platforms.

on the performance claim, the Newton engine is not known for performance. In fact, in 18 years that we have been around, most people say it is the slower performing engine. this of course is a clam that I dispute, because on those bench people use to compare Newton to other popular engines, the test are misleading and right out dishonest. for example that put a pyramids of 30 x 30 and say engine A is 10 time faster than Newton, but after 10 or 20 second of simulation, the pyramid of engine A collapses to the ground while the Newton stay put. Or they show joints arrangements in which engine A is all jittering and exploding while the Newton one are coherent and goes on an on.
that dishonesty used to make me upset but after so many years I do not care anymore.

so, to summarize, Integrating a library into GODOT as a proper server is a very big enterprise.
so big that has the potential to give you fatigue alone the way.

I will be very flattered if you decide to go with the engine, but I do not want you to be mislead and going in for the wrong reasons. and certainly competing with other engines to see which one is the performing monkeys that knows more tricks, is not something I am in the mood to do again.

Beside in my experience most physics library probably do the same things, so if you already have some lead way with that engine, you should probably try to get the issues resolved as oppose to a fresh start.

[geekrelief]

Thanks for clarifying things. You're right, I don't know whether starting from scratch will solve my problems. I'm in the research phase. And I'm aware I could be trading one set of known problems for unknown problems. This is all new to me, but I'm exploring the possible avenues available to me before I make any commitments.

As I've mentioned elsewhere I've been impressed with what I've seen so far and lack of cross platform determinism is just one factor to weight against the alternatives. I don't dismiss the amount of work it would take to integrate a new physics engine into Godot. Supposedly Godot 4.0 will make new physics engine integration even easier. But I'm using 3.x, and I'm comfortable modifying the engine, also there are 3 examples of how engines have been integrated. So whether I go with Rapier or Newton I know there's a lot of work ahead of me.

[Julio Jerez]

I just do not want you to go in a route that will lead you to a wild goose chase of disappointment very quickly. I am just going to tell you that I can't even begin to think how to make anything thing as simply as

x = a + b * c

deterministic even in the same computer not to mention across different hardware.

for once if you get an intel cpu and the instruction was a executed with a multiply follow by an add, you get different result than if the operation was issued by a MulAdd because the second provides one extra bit precision.
in fact it is even worse than that, in the same computer with same compiler and same optimizations, an statement like

x = a + b + c

can yield different results if for example somewhere in the code there was an intermediate value that calculated b + c
and them later that value was reused.

there is one reason for this, and that is that in fix size floating point arithmetic, law of transitivity and associativity brakes down. so the expression

x = a + ( b + c)
can be different than
x = (a + b) + c

this is a fact for all floating point arithmetic in a completer.
There are ways to solve those problems by using special variable size floating point software emulation arithmetic. and we have that In Newton for some kind of problems that are known to have difficult rounding errors. In newton that library is called Googol you can see in the source, but it is from 1000 to 10000 time slower.

so if you have a physics library that solved the problem of transitivity, associativity and commutativity in float point arithmetic, them that's is not brainer, because I can tell you, I am not even going to try.

[geekrelief]

Thanks for bringing the difficulty of cross-platform floating point determinism to my attention. After doing so more research, it makes achieving it all the more impressive. After your comment, and reading this https://randomascii.wordpress.com/2013/ ... terminism/ I have high level understanding of the various causes of of floating point non-determinism.

The author of Rapier goes by the name "sebcrozet" on the dimforge Discord server has been very open as to how it's achieved. Searching the comments several users and those curious about how it Rapier does its magic, sebcrozet credits a few things to achieve it:

Target architecture compliance with IEEE 754-2008
"1. Make sure you use seeded randoms (including with data-structures like HashMap).
2. Make sure you use the same math implementations on all platforms for special functions (trigonometry, sqrt, etc.)"
- Rapier uses nalgebra for all its math. And nalgebra has the libm-force feature that will force simba (which abstracts real numbers) to use the libm crate instead of what's provided by libstd.

"sebcrozet — 03/27/2021
So far, I regularly check that the cross-platform determinism of the f32 version of Rapier holds between the following platforms:
- Apple mac mini with their ARM M1 processor.
- Windows laptop with a Core i7 gen 10 processor.
- Ubuntu desktop with an AMD Ryzen 9 gen 3 processor.
I never actively tested the determinism for the f64 version but I expect it to work deterministically too.
"

[Sweenie]

Hi.

May I ask what kind of game you're making that requires this level of determinism, even across platforms?
Just curious because I even add some randomness to the simulation sometimes just avoid a deterministic result.

I guess if it was some kind of game where you could time warp back a couple of seconds and want the simulation to repeat exactly what it did i guess you would want it to be deterministic but wouldn't it be easier to just store the bodies transform and playback as an animation.

But anyway, just curious, not trying to talk you out of it. I'm sure you have a valid reason for it.

[EDIT]
Just got a Déjà vu feeling after I wrote that and realised I asked someone the same question back in 2018. I guess I'm deterministic

[geekrelief]

MeltingPlastic's reply from your 2018 question, is the big obvious reason. I'm experimenting with mechanics similar to Phantom Brigade. Here are some reasons for determinism off the top of my head, though there may be alternative ways around this as you suggest with recording transforms:

- Debugging physics across platforms / devices. You can record data from the user and reproduce the issue on your machine. Someone on a mobile device can send a bug report to my PC.
- Multiplayer: The server doesn't have to simulate the physics. You could compute hashes on each player's system to verify everyone is simulating the same world.
- Puzzle or skill based mechanics where reproducibility / predictability matters. I've worked on a mobile, physics, action puzzler and designing levels without determinism was a testing headache. With determinism in physics you could develop more interesting mechanics. Imagine what an FPS or MOBA might be like if you had deterministic physics.

But in general, these are just guesses. Every new capability opens up possibilities you couldn't imagine until it's there for you to play with.

Update: I also found this. https://www.reddit.com/r/Unity3D/commen ... ith_unity/
and https://forum.unity.com/threads/burst-c ... st-7205884