questions regarding suitability of newton dynamics

[eabbink]

Hi,

I have a couple of questions regarding Newton Dynamics. We have our own vehicle model for which several new requirements (mainly joints and increased stability) make it preferable to incorporate an existing physics engine. Accuracy and stability valued above performance. To be able to (easily) integrate Newton (or any other engine) we need the following:

1. Double precision support (related to large worlds): This should be present in Newton, however according to some old posts it may not be complete/functional or even buildable. Enabling the option in cmake builds fine however, and the demos application runs fine as well. However, even if the appropriate define _NEWTON_USE_DOUBLE is declared in the using project there still seem to be places where a dFloat is a float instead of a double. Declaring _NEWTON_USE_DOUBLE in the newton CMakeLists.txt and recompiling *seems* to fix this..

2. Step the simulation a certain amount of time: Call NewtonUpdate ()

3. For a rigid body query position: Call NewtonBodyGetPosition or NewtonBodyGetMatrix

4. For a rigid body query orientation: Call NewtonBodyGetRotation or NewtonBodyGetMatrix

5. For a rigid body query linear velocity: Call NewtonBodyGetVelocity

6. For a rigid body query linear acceleration: No function present?

7. For a rigid body query angular velocity: Call NewtonBodyGetOmega

8. For a rigid body query anguler acceleration: No function present?

9. Multiple times between simulations steps, add a force at a position (force may generate resulting torque): Callback mechanism is inconvenient here, but could be worked around by storing a list of forces to sum in the callback. For NewtonBodyAddForce() the manual lists: "This function is only effective when called from NewtonApplyForceAndTorque callback". Why is this the case? Why couldn't one simply call this function a number of times adding forces before calling NewtonUpdate?

10. (Re)set the simulation to a certain old or new state (ie. repositioning of vehicle).

11. Dynamic joint deletion and creation between existing rigidbodies.

12. Live changing of parameters (mass, stiffness, etc).

So at first glance most of the basics seem ok except for 6 and 8. Is it really not possible to query accellerations for a rigidbody? Note that for our application simply deriving them from velocities is a nogo.

For 10 - 12 I have not been able yet to glean enough information from the manual.

Thanks!

Esger

[Julio Jerez]

ok the answer to almost all the questions is yes, so I will answer the one missing.

-double is supported in all modes.

-get linear and angular acceleration is not there but the is not reason why is should not be, in the pass
there was a function GetNetForce which I removed early in 3.xx, I can add that function if you feel like you need it.
In my experience I can't think of a physical process where the acceleration is used a state variable, that's why it is not exposed, but if you need it that's not a problem.

-all parameters can be modified at any time, and bodies, joint an collision can be created and destroyed when the engine is not on a physicsUpdate.

-reset the physics is answered by the above question since the matrix and velocity are state variables of a body.

-apply force and torque call back, you do not need to apply the force on that callback if you do not want to, that call back is more for field forces like gravity, drag, wind, stuff like forces that are properties of the environment.
For a vehicle, you can make your vehicle system and connect it to the physics engine by a pre and post listener call back.
pre listener get called after force and torque callback, there you can update all your vehicles adding forces or what ever you need to do. You can also plug a post listener which is called after the update giving you a change to post correct simulation time or what ever.

-the joints in newton 3.14 are mathematically and physically rigorous at all time, they provide stability within the bounds of numerical accuracy (meaning integration and numerical rounding precision)
what kind of simulation is this? academia project a game or a simulation?

[JoeJ]

Getting acceleration would be awesome.
For my ragdoll controller i tried to calculate from velocities but it did not work even with the new exact joints, probably the lag of one step is already too much.
So i use the predicted acceleartion from the controller instead, but things may drift appart.

I did not even get the idea to ask for this feature I'm curious how it will affect my work...

[eabbink]

thanks for the quick reply!

One of the main reasons we need clean linear and angular accelerations is that in some applications they are sent to a motion system for motion cueing. Deriving them from speed delta's introduces artefacts which need to be filtered out, degrading the cueing response.

But the pre/post listener callbacks would still be callbacks, right? Which is quite different from the way our stuff works. But having to use a callback should not be too much of a problem, we can gather forces and torques going through our calculations and store them for the callback to collect and apply them to Newton. Is there some information somewhere about the listeners though? The manual does not seem to mention them and a search of the forum did not turn up anything either.

(Note: for a first time user it is quite easy to get lost between the two wiki's and not find the much more up to date docs at http://newton-dynamics.readthedocs.io/en/latest/index.html I believe I found those using google(!))

We develop simulators, both high-end involving 360 degree visuals, actual cabin and motion as well as smaller lower-cost systems. Currently I am working on an articulated vehicle combination with heavy masses (between 7 and 10 metric tons total). For future projects this could also be 40 tons for instance, or the vehicle may need to operate at crawl speeds with high drive torques being present. For us it's important to be able to match the parameters of the simulated vehicle as closely to the actual vehicle as possible. Which is one of the reasons I have decided against for instance PhysX, Bullet or the built-in physics of our 3D engine as those seem to be more primarily focused on effect and whether it looks nice. Which is perfectly fine for games and many other types of visualisations, but does not suit our purpose. We need a 14 ton truck to weigh 14 tons, and not 400kg because otherwise the simulation blows up

Regarding the version of newton, I have cloned the git repository as I could not find a 3.14 download and the 3.13 that can be downloaded did not immediately compile (whereas the git repo did). Is there a stable 3.14 release already? (I believe you introduced some major changes/improvements regarding stability in 3.14?)

[Julio Jerez]

do not use 3.13 use 3.14, the big change is that in 3.13 robust joint had to be specified manually and they did not support friction, this was resolved in 3.14, there are also a bunch of changes that makes 3.14 by far the best version yet.

regarding the masses, you should be fine if your masses and with is the few order of magnitude, you use double precision and your time step is relatively small, you can easy deal with ratios of ten of thousands. you can't do ridicules stuff like 100 ton vs 1 mgram, because you will need more precision than what can be encoded in a double, so as long as the ratios are of the order of half the decimal places of the mantissa of in a double, that's about 7 or 8 digits, it should be ok.

The documentation is old, the best way to get familiar wit the engine is to compiled the sandbox and look at the demos. there are all very straightforward and if you get confuse you can post question here, there are people who actually know tricks better than I do.

I recommend compiling the sandbox demos, and play with them see if you like them before you start experimenting.

[eabbink]

which commit is version 3.14? github only tags 3.13 and 3.12. or where can I download a targz?

I initially used the head revision from github, but that crashed as there are asserts in some functions where the body of the function has been commented out. presumably because of WIP.

Also, should I use dCustomJoints instead of NewtonConstraintCreateXXX? manual/tutorial does not mention them, but sandbox standardjoints demo uses them exclusively?

[Julio Jerez]

the head revision project should not crash, I tested it.
../newton-dynamics-master\applications\demosSandbox\projects\visualStudio_2015/demosSandbox.sln

the joints are dCustomJoints, which is a cpp class library of different joint types.
standardjoints is a good demo to see what most of then do.

[misho]

Just my $0.02,

(1) Regarding double precision, I am simulating spaceflight engine, and I am using a whole-earth model, with the centre of the coordinate system at Earth centre, and orbital altitudes (above ground) of up to 30,000 km. My orbital docking manoeuvres are millimetre-precise, so yeah, double precision works. The only catch is, (and I think that's still the case) it only works in a VS10 project (which loads fine in my VS15).

(10) (Re)set the simulation to a certain old or new state (ie. repositioning of vehicle). I am kind of stuck right now on this very thing. Seems that (as I gleaned from the demoSandbox), NewtonWorld and all objects are best destroyed and new world created, new objects created/inserted (in one call, not asynchronously) and continue with NewtonUpdate(). I have a reuirement of loading visual models asynchronously and realised that as they are loaded, they get "left behind" by older objects that loaded a few cycles behind. I am inserting/removing new objects on the fly without problems.

11. Dynamic joint deletion and creation between existing rigidbodies. I am doing that without any issues

cheers,
Misho

[Julio Jerez]

Oh yes the double configuration is no set on vs2013 and vs2015 projects.
what I am going to do is that I will recreate the projects these weekend, those project are 14 year old legacy form VS6 that I have being upgrading each time a new version of VS come out.
some time I have edited then manually and I think they have some corruptions.

so is time to make the fresh from VS2010 and import then into 2013 and 2015. one of the biggest problem I have is that when I import a project not all the setting get converted and this is why some option are not present, and I am sick of having to edit them by hand.
I will also re organize the directory structure, to make simpler.

and I will add that new function to get the acceleration.

[eabbink]

We are using linux so the VS projects are not an issue for us.

the master/HEAD version worked fine for the demos, but my very simple first try in our application consisting of:

Code: Select all

   
        // transformation matrix
   float   position_matrix [16] = {
      1.0f, 0.0f, 0.0f, 0.0f,
      0.0f, 1.0f, 0.0f, 0.0f,
      0.0f, 0.0f, 1.0f, 0.0f,
      0.0f, 0.0f, 10.0f, 1.0f
   };

   NewtonWorld* const world = NewtonCreate();   
   NewtonCollision* const collision = NewtonCreateSphere(world, 1, 0, NULL);

   NewtonBody* const body1 = NewtonCreateDynamicBody(world, collision, position_matrix);
   NewtonBodySetMassMatrix(body1, 0.0f, 0, 0, 0);
   NewtonBodySetForceAndTorqueCallback(body1, cb_applyForce);
   
   position_matrix [12] = 1.0 ;
   NewtonBody* const body2 = NewtonCreateDynamicBody(world, collision, position_matrix);
   NewtonBodySetMassMatrix(body2, 1.0f, 1, 1, 1);
   NewtonBodySetForceAndTorqueCallback(body2, cb_applyForce);
   
   float pivot_point [3] = { 0, 0, 10 } ;
   NewtonConstraintCreateBall (world, pivot_point, body2, body1) ;

crashes on the joint creation in dgBilateralConstraint.cpp line 114 where there is a "dgAssert (0)". Version 3.13 runs fine with identical code. Again, demos work for both versions.

Are the above functions (partly) deprecated and should I be using the dCustomJoints instead?

Thanks for adding the acceleration functions, that's great support for a first time user!

Misho, thanks for your feedback, that's good to hear. Regarding your being stuck on 10), I do not quite understand which issue you are having? On the one hand it seems you are saying creating all in one go is better, on the other hand inserting/removing on the fly works without problems. What is the "left behind" effect you mention?

For our purposes it might not always be feasible to recreate the entire world when repositioning, so we really just want to (between simulation updates) reset one or more bodies to a specific state. But from Julio's answer this should be no problem.

[Julio Jerez]

I said before dCustomJoints superseded the build in joints. those assert are there for that reason.

also I am not sure why there is still a confusion with adding and deleting, this operation should work at all time and under any allowed mode.

[misho]

Misho, thanks for your feedback, that's good to hear. Regarding your being stuck on 10), I do not quite understand which issue you are having? On the one hand it seems you are saying creating all in one go is better, on the other hand inserting/removing on the fly works without problems. What is the "left behind" effect you mention?

For our purposes it might not always be feasible to recreate the entire world when re-positioning, so we really just want to (between simulation updates) reset one or more bodies to a specific state. But from Julio's answer this should be no problem.

In my case, I have a collection of bodies linked to each other (a space station), moving at orbital speeds (~7000m/s). I can save their instantaneous state (positions/orientations/linkages) in a file. Now, when I want to re-load this file, my visual display system (server) loads my visual objects asynchronously, so there are a few cycles between request and notification of loading, and in that time, previously loaded objects have "raced ahead" 10-15 meters and not where they're supposed to be.

On the other hand, while in-sim, I can add a body in precise location at the orbital speed because I determine its velocity/position/orientation at the time of server notification that it has been loaded (it is not read off from a file). So, to answer your question, the difference is between reading from file, and on-the-fly creation, or more to the point, the difference is a few cycles that are required between request and notification of visual object loaded.

Resetting one or more bodies to a specific state is no problem. I do it on single bodies (location/orientation/velocity) and there is a method to do it on a group of linked bodies (the name escapes me right now)... the method traverses up the linked chain and makes appropriate transformations.

[eabbink]

@julio

Ah, apologies, I missed that remark. Custom joints it is

@misho

Is your newton object created on visual object load? in that case I understand your problem, you are providing a position at time = 0 from file for that object which is not in sync anymore with the already existing objects and simulation which are at time = some t.

If the above is correct, a possible solution could be to decouple newton and visual objects. Ie. create all (meta)objects which are present at time = 0 including newton objects, and add the visual objects to their (meta)objects (which contain reference to newton and visual object) when they become available. However, this implies knowing which objects need to exist at time = 0, which may not be the case if object presence/creation is triggered by the graphics engine loading a scene file with unknown object contents.. Having a state file should eliminate that issue though?

[eabbink]

Looking at the StandardJoints demo body creation seems quite similar. But the (custom) joint creation was not quite clear to me. During writing this post, googling and looking at more newton code I think I understand, but to perhaps also help others, I'll post anyway.

Looking at below excerpt (StandardJoints.cpp line 202)

Code: Select all

dMatrix pinMatrix(dGrammSchmidt(dVector(0.0f, -1.0f, 0.0f, 0.0f)));

// connect first box to the world
dMatrix matrix;
NewtonBodyGetMatrix(box0, &matrix[0][0]);
pinMatrix.m_posit = matrix.m_posit + dVector(0.0f, size.m_y, 0.0f, 0.0f);

dCustomLimitBallAndSocket* const joint0 = new dCustomLimitBallAndSocket(pinMatrix, box0, base);


I assumed the pinMatrix specifies both the location of the joint in world coordinates and an axis? The m_posit part being the bottom row of the matrix where translation is stored.

But what is dGrammSchmidt(dVector(0.0f, -1.0f, 0.0f, 0.0f))?

I am not that brushed up in maths, but this creates not a rotation matrix but a matrix containing a coordinate system (ie. the upper three rows are axis vectors)? with the first axis specified in the first three components of the dVector as the pin axis?

So:

dGrammSchmidt(dVector(0.0f, -1.0f, 0.0f, 0.0f)) -> pin axis parallel to Y axis

dGrammSchmidt(dVector(1.0f, 0.0f, 0.0f, 0.0f)) -> pin axis parallel to X axis

dGrammSchmidt(dVector(1.0f, 1.0f, 0.0f, 0.0f)) -> pin axis parallel to 45 degree x = y line

dGrammSchmidt(dVector(0.0f, -1.0f, 2.0f, 0.0f)) -> pin axis parallel to z = -2y line

Based on above assumptions my initial test seems to work ok in 3.14. yay!

[misho]

If you are looking to connect 2 bodies via "hard link" (which I use for my space station objects as "hard dock" linkages), the code is straightforward:

Code: Select all

dMatrix mSourceMatrix = dGetIdentityMatrix();
mSourceMatrix.m_posit = targetEnt->m_curPosition;
sourceEnt->pHinge = new dCustomHinge(mSourceMatrix, sourceEnt->nBody, targetEnt->nBody);
sourceEnt->pHinge->EnableLimits(true);
sourceEnt->pHinge->SetLimits(0.0, 0.0);

pHinge is defined as dCustomHinge* (member of entity class), and targetEnt and sourceEnt are entity classes set as user data for 2 newton bodies being linked. They contain, well, user data , all kinds of properties pertaining to the visual object, in my case. The code is simplified, I actually have an array of pHinges, depending on how many docking hatches are defined on an object.

What I do is, I align 2 objects according to what attachpoints I am linking them to, and call the above code. The objects don't have to be in contact, or aligned, or anything - all this code does is locks the relationship between their retrospective position/rotation matrices. The objects can be far apart, but after this call, they will behave as if they have an invisible clamp joining them.