On-demand body creation breaks [SOLVED]

Report any bugs here and we'll post fixes

Moderators: Sascha Willems, Thomas

On-demand body creation breaks [SOLVED]

Postby misho » Fri Mar 29, 2019 9:52 pm

Hi everyone!

I have a setup where Newton bodies are created and linked in real time (on demand), rather than pre-creating them. While testing this in demoSandBox, I noticed different behavior for Debug and debug_double. If in a double precision build, the object I am adding comes out completely distorted. Here is a screen shot from demoSandBox:

Image

On the left, in single precision build, the bottom object is added during scene creation, and the top object (inverted cone) is added when I hit spacebar. In the double precision build, the added object comes out completely distorted.

I have to mention that my method for inserting this object is a bit "quick and dirty": I implemented it inside a ForceAndTorqueCallback function, and I made sure it ran only once. While there is probably a proper way of doing this, I did it this way for simplicity's sake. Also, I noticed that when adding objects this way, the specified materials weren't applied to them (in both builds). Also, same thing happens in Release and release_double builds.

Any idea why this is happening? Here is the code for the test case:

Code: Select all
#include <toolbox_stdafx.h>
#include "SkyBox.h"
#include "DemoEntityManager.h"
#include "DemoCamera.h"
#include "PhysicsUtils.h"
#include "DemoMesh.h"
#include "OpenGlUtil.h"

enum LinkType
{
   NONE,
   LINK_FIXED,
   LINK_BALLSOCKET
};
static void ParachuteLoad(const NewtonBody* body, dFloat timestep, int threadIndex);
static void Nothing(const NewtonBody* body, dFloat timestep, int threadIndex) { return; }

NewtonBody* GetObject(NewtonWorld* const world, dString strName)
{
   NewtonBody* object = NULL;
   for (NewtonBody* bodyZ = NewtonWorldGetFirstBody(world); bodyZ; bodyZ = NewtonWorldGetNextBody(world, bodyZ))
   {
      DemoEntity* ent = (DemoEntity*)NewtonBodyGetUserData(bodyZ);
      dString name = ent->GetName();
      if (name == strName) {
         object = bodyZ;
         return object;
      }
   }
   return NULL;
}

void   CreateCapsule(NewtonWorld* const world)
{
   dMatrix offset(dGetIdentityMatrix());
   dVector damp(0.0f, 0.0f, 0.0f);
   dFloat CapsuleMass = 10387.0f;

   DemoEntityManager* const scene = (DemoEntityManager*)NewtonWorldGetUserData(world);

   // Make Orion Capsule body...
   NewtonCollision* const CapsuleShape = NewtonCreateCylinder(world, 20.0f, 6.0f, 20.0f, 1, &offset[0][0]);
   NewtonCollision* const CapsuleCompound = NewtonCreateCompoundCollision(world, 0);
   NewtonCompoundCollisionBeginAddRemove(CapsuleCompound);
   NewtonCompoundCollisionAddSubCollision(CapsuleCompound, CapsuleShape);
   NewtonCompoundCollisionEndAddRemove(CapsuleCompound);
   dMatrix matrixCapsule(0.0f, 0.0f, 1.5708f, dVector(0.0f, 0.0f, 0.0f));

   DemoMesh* const geometryCapsule = new DemoMesh("Capsule Mesh", scene->GetShaderCache(), CapsuleCompound, "metal_30.tga", "metal_30.tga", "metal_30.tga");
   NewtonBody* const CapsuleBody = CreateSimpleSolid(scene, geometryCapsule, CapsuleMass, matrixCapsule, CapsuleCompound, 0);
   NewtonBodySetForceAndTorqueCallback(CapsuleBody, ParachuteLoad);
   DemoEntity* const entityCapsule = (DemoEntity*)NewtonBodyGetUserData(CapsuleBody);
   entityCapsule->SetNameID("Capsule");
   NewtonBodySetLinearDamping(CapsuleBody, 0.0f);
   NewtonBodySetAngularDamping(CapsuleBody, &damp[0]);
   geometryCapsule->Release();
   NewtonDestroyCollision(CapsuleCompound);
   NewtonDestroyCollision(CapsuleShape);

   dVector origin(matrixCapsule.m_posit);
   origin.m_x -= 120.0f;
   origin.m_y += 30.0f;
   dQuaternion rot;
   scene->SetCameraMatrix(rot, origin);
}

void   CreateParachute(NewtonWorld* const world, dVector location)
{
   dMatrix offset(dGetIdentityMatrix());
   dVector damp(0.0f, 0.0f, 0.0f);
   dFloat ParachuteMass = 20.0f;
   
   DemoEntityManager* const scene = (DemoEntityManager*)NewtonWorldGetUserData(world);

   // Make Parachute ... 
   NewtonCollision* const ParachuteShape = NewtonCreateCylinder(world, 0.2f, 40.0f, 40.0f, 1, &offset[0][0]);
   NewtonCollision* const ParachuteCompound = NewtonCreateCompoundCollision(world, 0);
   NewtonCompoundCollisionBeginAddRemove(ParachuteCompound);
   NewtonCompoundCollisionAddSubCollision(ParachuteCompound, ParachuteShape);
   NewtonCompoundCollisionEndAddRemove(ParachuteCompound);

   dMatrix matrixParachute(0.0f, 0.0f, 1.5708f, location + dVector(0.0f, 30.0f, 0.0f));
   DemoMesh* const geometryParachute = new DemoMesh("Parachute Stage", scene->GetShaderCache(), ParachuteCompound, "metalblu.tga", "metalblu.tga", "metalblu.tga");
   NewtonBody* const ParachuteBody = CreateSimpleSolid(scene, geometryParachute, ParachuteMass, matrixParachute, ParachuteCompound, 0);
   NewtonBodySetForceAndTorqueCallback(ParachuteBody, Nothing);
   DemoEntity* const entityParachute = (DemoEntity*)NewtonBodyGetUserData(ParachuteBody);
   entityParachute->SetNameID("Parachute");
   NewtonBodySetLinearDamping(ParachuteBody, 0.0f);
   NewtonBodySetAngularDamping(ParachuteBody, &damp[0]);
   geometryParachute->Release();
   NewtonDestroyCollision(ParachuteCompound);
   NewtonDestroyCollision(ParachuteShape);
}
void   DeployParachute(NewtonWorld* const world, dString strParent, dString strChild, int nLinkType = NONE)
{
   static bool bOnce = false;

   if (bOnce)
      return;

   bOnce = true;

   NewtonBody* parent = GetObject(world, strParent);

   dMatrix  mParentMatrix;
   NewtonBodyGetMatrix(parent, &mParentMatrix[0][0]);
   CreateParachute(world, mParentMatrix.m_posit);

   NewtonBody* child = GetObject(world, strChild);

   switch (nLinkType)
   {
      case LINK_FIXED:
      {
         dCustomHinge* const pHinge = new dCustomHinge(mParentMatrix, child, parent);
         if (pHinge)
         {
            pHinge->EnableMotor(false, 0.0);
            pHinge->SetAsSpringDamper(false, 0.0, 0.0, 0.0);
            pHinge->EnableLimits(true);
            pHinge->SetLimits(0.0, 0.0);
         }
         else
            printf("\nFAILED TO LINK WITH FIXED JOINT\n");
      }
      break;
      case LINK_BALLSOCKET:
      {
         dMatrix pinMatrix(dGrammSchmidt(dVector(0.0f, -1.0f, 0.0f, 0.0f)));
         pinMatrix.m_posit = mParentMatrix.m_posit + dVector(0.0f, 10.0f, 0.0f, 0.0f);
         dCustomBallAndSocket* const pBASHinge = new dCustomBallAndSocket(pinMatrix, child, parent);
         if (pBASHinge)
         {
            pBASHinge->EnableCone(true);
            pBASHinge->EnableTwist(true);
            pBASHinge->SetConeLimits(30.0f * dDegreeToRad);
         }
         else
            printf("\nFAILED TO LINK THROUGH BALL AND SOCKET\n");
      }
      break;
      default:
         break;
   }
}

static void ParachuteLoad(const NewtonBody* body, dFloat timestep, int threadIndex)
{
   NewtonWorld* const world = NewtonBodyGetWorld(body);
   DemoEntityManager* const scene = (DemoEntityManager*)NewtonWorldGetUserData(world);
   bool bSpaceBar = false;

   if (scene->GetKeyState(' ')) {
      bSpaceBar = true;
   }

   if (bSpaceBar)
      DeployParachute(world, "Capsule", "Parachute", LINK_BALLSOCKET);
}
void MishosHingeTest(DemoEntityManager* const scene)
{
   scene->CreateSkyBox();
   NewtonWorld* const world = scene->GetNewton();
   CreateCapsule(world);
}
Last edited by misho on Thu Apr 04, 2019 3:38 pm, edited 1 time in total.
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 549
Joined: Tue May 04, 2010 10:13 am

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby Julio Jerez » Sat Mar 30, 2019 12:45 pm

that demo will not run on any configuration for many reasons.

1-from graphic standpoint in the sandbox, you can't make graphic geometry from outside the scope of a opengl context.
2-from the newton side it is not allowed to create any newton object from a callback.

I just pasted it and It get many errors. with so many broken rules I am not going to even debug it.
my guess is that you do no get the errors because you build in release. But the problem you see if because try to create open gl objects without a gl context. it has nothing to do with the engine.
Julio Jerez
Moderator
Moderator
 
Posts: 10984
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby misho » Sat Mar 30, 2019 4:36 pm

Julio,

I'm not sure what you mean - the code, as I pasted it, compiles without a single warning or error. (I completely replaced content in MishoRocketTest.cpp) I just tried it again, and as I paste it, I do get a lot of "red" in Visual Studio, but if I perform "Rescan Solution" is VS, it all goes away (as it sorts out references).

So, would you mind trying to compile it again?

And, when I run it, the Debug and Release single precision works, but Debug and Release double precision does not.

In any case, if I am calling the object creation the wrong way, how is it done the right way? Is there any demo in the demoSandBox that loads the object on demand?
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 549
Joined: Tue May 04, 2010 10:13 am

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby Dave Gravel » Sun Mar 31, 2019 8:19 pm

If you remove NewtonBodySetForceAndTorqueCallback(CapsuleBody, ParachuteLoad);
If you just use DeployParachute at the creation it seen to work good.
//
It is bad idea to create object inside callback NewtonBodySetForceAndTorqueCallback
If i'm not wrong it is a bad idea to use any system command inside the callback too.
Exemple it is not good to call keyboard command or any win system command inside the callback.
It is not working 100% like a normal callback because it is multithreaded, If you start to lock and unlock anywere you have big chances to get lag and big slowdown or problems.

The best is to always let's the callback's the most free possible for the use of newton commands only..

Normally the keyboard commands is call from the update,
From it you can create the objects.
If i'm not wrong when you don't use a listener and when you create a object in runtime.
You need to call NewtonWaitForUpdateToFinish before.

I think you can find how to use keyboard correctly from vehicle demo.

This working good in double.
CreateCapsule(world);
DeployParachute(world, "Capsule", "Parachute", LINK_BALLSOCKET);
You search a nice physics solution, if you can read this message you're at the good place :wink:
OrionX3D Projects & Demos:
https://www.facebook.com/dave.gravel1
http://orionx3d.googlepages.com/
https://www.youtube.com/user/EvadLevarg/videos
User avatar
Dave Gravel
 
Posts: 712
Joined: Sat Apr 01, 2006 9:31 pm
Location: Quebec in Canada.

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby Julio Jerez » Sun Mar 31, 2019 10:04 pm

so did you get it to work Dave, I could not.
Misho if you try what Dave said and it works can you made a video, I am curios as to see how the parachute works.

as for how you crate bodies outside a call back, you make a queue where you push the command generated by any call back. the before a newton update, you check If the queue is not empty

if the queues is not empty, then you call
NewtonWaitForUpdateToFinish (m_world);

and you iterate the queue don any change to the Newton world.
this is how graphic system work for years, not one make vertex buffer from a shader, but teh expect to do it form a physics library.
there are ton of reason whey not you create resources form a fall back, and that date form newton 1.0, with the multicores and multithreaded these reason are just more extract than before.

for example you may create an object, by the memory resource is in some thread local memory buffer and the parent are not in a different local thread memory.
this just does not work in newton.
Julio Jerez
Moderator
Moderator
 
Posts: 10984
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby Dave Gravel » Sun Mar 31, 2019 10:15 pm

Yes it work with the change.
You search a nice physics solution, if you can read this message you're at the good place :wink:
OrionX3D Projects & Demos:
https://www.facebook.com/dave.gravel1
http://orionx3d.googlepages.com/
https://www.youtube.com/user/EvadLevarg/videos
User avatar
Dave Gravel
 
Posts: 712
Joined: Sat Apr 01, 2006 9:31 pm
Location: Quebec in Canada.

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby misho » Sun Mar 31, 2019 10:31 pm

Dave Gravel wrote:You need to call NewtonWaitForUpdateToFinish before.

I think you can find how to use keyboard correctly from vehicle demo.

This working good in double.
CreateCapsule(world);
DeployParachute(world, "Capsule", "Parachute", LINK_BALLSOCKET);


Thanks Dave!

Yeah, that's why I mentioned I was doing it the "quick and dirty" way, and that's what Julio was saying, I just wasn't sure how to properly create body "on demand", so I was asking. Yes, the parachute is created correctly at the creation but obviously fails when invoked from the callback.

Thanks for the tip, I'll try creating it from the NewtonWaitForUpdateToFinish

Misho
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 549
Joined: Tue May 04, 2010 10:13 am

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby misho » Sun Mar 31, 2019 10:47 pm

Julio Jerez wrote:Misho if you try what Dave said and it works can you made a video, I am curios as to see how the parachute works.


For sure - I have it almost working, I'll make a short video, it looks great! I just have some instabilities in the links, but that might be fixed if I implement body creation properly, as you suggest below.

Julio Jerez wrote:as for how you crate bodies outside a call back, you make a queue where you push the command generated by any call back. the before a newton update, you check If the queue is not empty

if the queues is not empty, then you call
NewtonWaitForUpdateToFinish (m_world);

and you iterate the queue don any change to the Newton world.
this is how graphic system work for years, not one make vertex buffer from a shader, but teh expect to do it form a physics library.
there are ton of reason whey not you create resources form a fall back, and that date form newton 1.0, with the multicores and multithreaded these reason are just more extract than before.

for example you may create an object, by the memory resource is in some thread local memory buffer and the parent are not in a different local thread memory.
this just does not work in newton.


So, is that done that way because of OpenGL? or Newton itself? Or both? Because I have a different graphics layer - it is independent of Newton. I've started adding objects on demand just recently and I haven't noticed problems in body creation, just some linking instabilities. I should add that I'm not adding my objects from the ForceAndTorque callback, I simply create a newton body within my code and add it to the Newton scene. For example, here is a capsule re-entering atmosphere, and the 3D animated re-entry plasma effect is a Newton object with NULL collision that gets loaded at a certain G level and attached to the capsule:

Image
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 549
Joined: Tue May 04, 2010 10:13 am

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby Julio Jerez » Mon Apr 01, 2019 10:48 am

misho wrote:So, is that done that way because of OpenGL? or Newton itself?


it no just that, is about how teh app was designed. I physio library is designed for high perforce, you are no expected to do allocation, of stuff that has to do with context on call backs.

for example in teh demo that is commited you do stuff liek this
Code: Select all
         dString name = ent->GetName();
...
   if (name == "LAS")
      NewtonBodyAddForce(body, &force.m_x);


this is not jsut bad in newton is bad all around, the operation == will cerate an object on the stack and destroy aft all, this will work on stuff like STL but fail bad in newton because ether newton string class is about 10 time faster than STL, but is design for realtime usage, is does not do any of teh thread safe checks.
the reason is works in because you are ruing on single thread so all allocations and destruction are sequential.

if you jsut simple set the engine to run on multithread the code crashes really bas, because is possible that while one thread is creating one string, the another is deleting some other and the poll system goes out of sync.

you secudn demo try to create objects form teh main tread but newton is running on a another thread, so teh cause for the crash I do not knwo what is was but Dave found it.
I have to fix the code by doing this

Code: Select all
         const  dString& name = ent->GetName();
...
   static dString las("LAS");
   if (name == las)
      NewtonBodyAddForce(body, &force.m_x);


and that make it work all around.
these is no just for newton, these are just good programing practices.
In fact these days I made my leaving optimizing code by doing that kind of stuff to code
written by inexperienced programmers.
Julio Jerez
Moderator
Moderator
 
Posts: 10984
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby misho » Mon Apr 01, 2019 1:36 pm

Thanks Julio!

Yes, I knew I wasn't doing it properly, I was mentioning I did it "quick and dirty" way :roll: but I thought I'd get away with it just for the brevity's sake. I since found a bug in my sim - it was a misplaced "pivot" point when I was linking, so that demo I was posting is now moot. I still have some odd instability in the linked objects, (capsule and parachute) as they fall to the ground, but I think it has to do with how I apply forces to the objects (drag, dynamic pressure & restoring torque...)

I am actually myself an inexperienced programmer (but I'm learning fast), and I do appreciate you and others pointing out all the mistakes I might be making! I'm glad to hear you make a career out of optimizing code - perhaps one day, if I make some decent cash with the "first pass" version of this sim, I'd love to retain you to go through my code and optimize it, I'm sure it needs it!
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 549
Joined: Tue May 04, 2010 10:13 am

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby misho » Mon Apr 01, 2019 1:55 pm

Now, with regards to properly inserting Newton bodies on demand, here's what I have at the moment:

My main event loop is calling PhysicsUpdate() as such:

Code: Select all
void CALLBACK MyDispatchProcSD(SIMULATOR_RECV* pData, DWORD cbData, void *pContext)
{
    HRESULT hr;

    switch(pData->dwID)
    {
        ... (events)
    }

    if (NewtonGetEntityCount() && !SystemVars->bSuspendPhysics)
    {
        dFloat timestep = dGetElapsedSeconds();
        UpdatePhysics(timestep);
    }
}


And now, instead of just creating Newton body and inserting it into Newton world at any point in execution, I will create an aggregate (container) of objects to add, and add to it as I need:

Code: Select all
vector<ObjectContainer> ObjectsToAdd;
...
ObjectContainer RocketPart;
ObjectsToAdd.push_back(RocketPart);
...
ObjectContainer AnotherRocketPart;
ObjectsToAdd.push_back(AnotherRocketPart);
...


And then, in my event loop:

Code: Select all
void CALLBACK MyDispatchProcSD(SIMULATOR_RECV* pData, DWORD cbData, void *pContext)
{
    HRESULT hr;

    switch(pData->dwID)
    {
        ... (events)
    }

    if (NewtonGetEntityCount() && !SystemVars->bSuspendPhysics)
    {
        if(ObjectsToAdd.size())
        {
                if (g_world) {
                        NewtonWaitForUpdateToFinish(g_world);
                }
                AddObjects(ObjectsToAdd);
                ObjectsToAdd.clear();
        }
        dFloat timestep = dGetElapsedSeconds();
        UpdatePhysics(timestep);
    }
}


If I understood it, that's the proper way of adding objects "on the fly", correct?

Thanks,
Misho
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 549
Joined: Tue May 04, 2010 10:13 am

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby Julio Jerez » Mon Apr 01, 2019 3:04 pm

The container accesor function should have a critical section, so that only one operation can be added or removed at a time.
Julio Jerez
Moderator
Moderator
 
Posts: 10984
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby misho » Mon Apr 01, 2019 3:08 pm

Julio Jerez wrote:The container accesor function should have a critical section, so that only one operation can be added or removed at a time.


Oh, ok - so only one body can be added or removed per UpdatePhysics() cycle?
Misho Katulic
CTO, FSX SpacePort
TerraBuilder
www.terrabuilder.com
misho
 
Posts: 549
Joined: Tue May 04, 2010 10:13 am

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby Dave Gravel » Mon Apr 01, 2019 3:19 pm

Just a quick suggest.
I think the best demo exemple for show how create multiple bodies in runtime is SimpleConvexFracturing.cpp
This exemple show how to create many bodies in one update pass.
You search a nice physics solution, if you can read this message you're at the good place :wink:
OrionX3D Projects & Demos:
https://www.facebook.com/dave.gravel1
http://orionx3d.googlepages.com/
https://www.youtube.com/user/EvadLevarg/videos
User avatar
Dave Gravel
 
Posts: 712
Joined: Sat Apr 01, 2006 9:31 pm
Location: Quebec in Canada.

Re: On-demand body creation breaks in _NEWTON_USE_DOUBLE bui

Postby Julio Jerez » Mon Apr 01, 2019 3:23 pm

No, one operation, adding of removing can be executed at a time on the entire container.
a common mistake people make of locking only the accesor and the get into extrematlly hard bug.

Imagine link list in which you lock because you are adding a node, meanwhile some other thread is deleting some other node, if the lock is not locking the entire container, they will both succeeded, but then you may end up with a node created on one thread that is linked to a node that was deleted on another thread because at the time they were lock they were both alive.

Making thread safe containers is an unsolved problem in software ingeniring.
I will try to write an example how to do it tonight.

Does with the second demo you posted that Dave fixed.
Julio Jerez
Moderator
Moderator
 
Posts: 10984
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Next

Return to Bugs and Fixes

Who is online

Users browsing this forum: No registered users and 1 guest