[SOLVED] Crash on destroying bodies

[JernejL]

Yes, i think this was fixed once in the past already, i can make a workaround by setting userdata to null and ignore such bodies during material callback but i think this is probably a memory leak in the end and is probably something that can be fixed, if a deleted body is making material contact callback calls, it's still in the system somehow..

I am making a demo for you, the demo will crash at an unrelated location, but if you inspect debug log you will see a callback will be called for a body which was deleted.

DEMO: https://gtamp.com/tdc/tdc_demo_error.7z

The demo will not crash outside debugger - run with debugger, it will crash very fast without you having to do anything else.



All you need to do is run it in windbg or debug it in visual studio directly.
You can put custom newton.dll into the Libraries folder - game soft loads newton dll at runtime, it needs a 32 bit build - debug or release.

This is the same crash in visual studio with full call stack:


A deleted body had callback called:


There are other problems with the trunk newton too - it looks like optimizer deleted some parts of map and player can fall thru ground at random.. but let's first fix this as it could be the cause for other issues.

[Julio Jerez]

not do not do any work around, we have to get it fixed.
I can see images or download stuff from where I am, I will check it tonight

in eh collision tree, there was bad bug that I fixed, by it should make better optimizations,
or a least that was my goal.
anyway let us check it tomorrow.

[JernejL]

not do not do any work around, we have to get it fixed.
I can see images or download stuff from where I am, I will check it tonight

in eh collision tree, there was bad bug that I fixed, by it should make better optimizations,
or a least that was my goal.
anyway let us check it tomorrow.

The optimizer seems to excessibely remove a lot of faces, here is an example:

Roofs on left side of the picture go missing:

https://i.imgur.com/xQSSnhm.png

https://i.imgur.com/NDf2bro.png

You can see and turn this on in game by pressing F1 going to debug controls and check both draw newton map + draw newton materials (to render debug shapes solid)

This seems to happen mostly with faces that are sloped, but nearly identical scenarios sometimes fail and sometimes not. let me show this spot:

This has 2 identical structures:

https://i.imgur.com/dpGZ7v7.png

The one on left is optimized ok and the one on right is not, it removes all those faces.

https://i.imgur.com/36t2Oom.png

[Julio Jerez]

when I run the demo I get this error

>> Prefetching Map textures
>> Newton Init...
'tdc.exe' (Win32): Loaded 'C:\Users\julio\Downloads\TDC DEMO\Libraries\newton.dll'. Symbols loaded.
'tdc.exe' (Win32): Unloaded 'C:\Users\julio\Downloads\TDC DEMO\Libraries\newton.dll'
'tdc.exe' (Win32): Loaded 'C:\Users\julio\Downloads\TDC DEMO\Libraries\newton.dll'. Symbols loaded.
[Newton] Initialized version 3.14 with 4 threads.
[Newton] World size: From -10.00000, -265.00000, 0.00000 to 265.00000, 10.00000, 8.00000
[Newton] Converting Map to physical world
Cache file C:\Users\julio\Downloads\TDC DEMO\data\maps\cache\tstd(d64_0_0_64_64_Slice_0)_floor.ngd newton checksum changed, rebuilding Is: 64D7C69C - Should be: BF7E06C4
tdc.exe has triggered a breakpoint.

and it can't go on form there

[Julio Jerez]

Oh I think I figure out why thsi bug is so weird. whe I though I fixed I made a big mistake.
I added the code the flag the contact as dead in teh same funtion that garbage collect the contact

Thsi explain whe all these call back keep getting call on dead contacts.
It is this function here.

Code: Select all

void dgDeadBodies::DestroyBodies(dgWorld& world)
{
//   dgScopeSpinLock lock(&m_lock);
   if (GetCount()) {
/*
      Iterator iter(*this);
      for (iter.Begin(); iter; iter++) {
         dgTreeNode* const bodyNode = iter.GetNode();
         dgBody* const body = bodyNode->GetInfo();

         for (dgBodyMasterListRow::dgListNode* node = body->GetMasterList()->GetInfo().GetFirst(); node; node = node->GetNext()) {
            dgConstraint* const joint = node->GetInfo().m_joint;
            dgAssert(joint);
            if (joint && (joint->GetId() == dgConstraint::m_contactConstraint)) {
               dgContact* const contactJoint = (dgContact*)joint;
               contactJoint->m_killContact = 1;
            }
         }
      }
*/
      world.m_broadPhase->DeleteDeadContact(0.0f);



thepart tha is commented out does not belong there. It belong to function

Code: Select all

void dgDeadBodies::DestroyBody(dgBody* const body)
{
   dgScopeSpinLock lock(&m_lock);

   if (body->m_destructor) {
....


I am fixing and testing this now.

[Julio Jerez]

ok JernejL

I added this code to the optimizer.

Code: Select all

void dgCollisionBVH::EndBuild(dgInt32 optimize)
{
   dgVector p0;
   dgVector p1;

   bool state = optimize ? true : false;
#ifdef _DEBUG
   if (state && (optimize >> 1)) {
      char debugMesh[256];
      sprintf (debugMesh, "debugMesh_%d.ply", optimize-1);
      m_builder->SavePLY(debugMesh);
   }
#endif


thsi will let us save the mesh as a PLY file, than we can use for debugging without having to send apps.
If you pass a value larger than 1 to the optimizer, and onle in debug mode. I will produce a ply file
Later I will add a menu option that will let us load tshi file to the SDK demos and for debugging.
but you can also check out the file with this tool.
http://www.meshlab.net/

this way is far easier to debug these problems, I think.

[Julio Jerez]

ok now if you sync and in you code you add a line liek this

NewtonTreeCollisionEndBuild(collision, optimize ? n : 0);

where n is a large that one value, this will save a PLY mesh, the you can open it in he engine and see whi one has the hole, I hope this make is eassy to find the mesh with bugs, so that we can debug the problem.
in the engine it will show in wire frame, by you can use the menu to see if solid faces

also if you sync, the new commit should have the contact crash bug fixed.

[JernejL]

i've synced to trunk and had some problem with record alignment that i described there: viewtopic.php?f=9&t=9649&p=65210#p65210

I added some workaround for that, but now i get a crash in newton - UpdateRigidBodyContacts:

First-chance exception at 0x7C819493 (newton.dll) in TDC.exe: 0xC0000005: Access violation reading location 0x00000010.

Call stack:

Code: Select all

>   newton.dll!dgBroadPhase::UpdateRigidBodyContacts(descriptor=0x180cfe14, timeStep=3.77288724e-017, threadID=2) Line 1570   C++
    newton.dll!dgBroadPhase::UpdateRigidBodyContactKernel(context=0x180cfe14, __formal=0x00000000, threadID=2) Line 1488   C++
    newton.dll!dgThreadHive::dgWorkerThread::ConcurrentWork(threadId=2088467053) Line 242   C++
    newton.dll!dgThreadHive::dgWorkerThread::Execute(threadId=2) Line 260   C++
    newton.dll!dgThread::dgThreadSystemCallback(threadData=0x1938be94) Line 202   C++
    newton.dll!std::_LaunchPad<std::_Bind<1,void *,void * (__cdecl*const)(void *),dgThread *> >::_Go() Line 187   C++
    newton.dll!_callthreadstartex() Line 376   C
    newton.dll!_threadstartex(ptd) Line 354   C
    kernel32.dll!@BaseThreadInitThunk@12()   Unknown
    ntdll.dll!__RtlUserThreadStart()   Unknown
    ntdll.dll!__RtlUserThreadStart@8()   Unknown



Local variables:

Code: Select all

+      this   0x00000000 <NULL>   dgBroadPhase *
+      descriptor   0x180cfe14 {m_world=0x192f1a60 {m_destructor=0x00000000 } m_timestep=0.0166666675 m_atomicIndex=0 ...}   dgBroadPhase::dgBroadphaseSyncDescriptor * const
      timeStep   3.77288724e-017   float
      threadID   2   int



Screenshot:
https://i.imgur.com/TdRPYjv.png

You can run the game demo in visual studio to test the crashes - just compile newton as a .dll and put it into game \libraries\ directory then open the tdc.exe in visual studio and run it to debug it directly.
If the game sees that newton dll changed, it will also re-create serialized map meshes automaticly with new dll (it checks checksum).

https://gtamp.com/tdc/tdc_demo_error.7z

[Julio Jerez]

I still do not get how is that the branch get executed when a the deleted body flags all its contacts as dead. if you look at the line above

Code: Select all

      if (!(contact->m_killContact | (body0->m_equilibrium & body1->m_equilibrium))) {

that bracket is conditioned on contact->m_killContact been false.

before I get the repro demo, can you please sync to get the alignment issue fixed first?

[JernejL]

I made a reply about alignment, but the records are identical now, the demo should work with union and older variant of struct properly.

[JernejL]

Also, it seems like there are no more map holes in optimization - the debug routines work and render proper map meshes now. - still crashes tho.

[JernejL]

Ok, HERE is a new compiled with new structs- demo showing the crash:

https://gtamp.com/tdc/tdc_demo_error.7z

newton.dll is in Libraries folder (32 bit single), let me know if you need any help reproducing crash or debugging

[Julio Jerez]

ok fine, I will download and try tonight when I get home.

mean time can you run this test, I added this assert, to see if somehow a dead contact is processed

Code: Select all

      if (!(contact->m_killContact | (body0->m_equilibrium & body1->m_equilibrium))) {
         dgAssert(!contact->m_killContact);


for what i can see this crash defies the laws of logic.

[JernejL]

Ok, i made a pull of this, but the assert didn't trigger, it did now crash here in dgBroadPhase::SanityCheck - this is debug build (previous it was crashing in release build):

Code: Select all

First-chance exception at 0x7A3CA836 (newton.dll) in TDC.exe: 0xC0000005: Access violation reading location 0x000001D0.
Unhandled exception at 0x7A3CA836 (newton.dll) in TDC.exe: 0xC0000005: Access violation reading location 0x000001D0.


Code: Select all

>   newton.dll!`dgBroadPhase::SanityCheck'::`2'::dgKey::dgKey(contact=0x188d24e0) Line 1606   C++
    newton.dll!dgBroadPhase::SanityCheck() Line 1634   C++
    newton.dll!dgBroadPhase::AttachNewContact(startCount=126) Line 1659   C++
    newton.dll!dgBroadPhase::UpdateContacts(timestep=0.0166666675) Line 1753   C++
    newton.dll!dgWorld::UpdateBroadphase(timestep=0.0166666675) Line 1229   C++
    newton.dll!dgWorld::StepDynamics(timestep=0.0166666675) Line 916   C++
    newton.dll!dgWorld::RunStep() Line 977   C++
    newton.dll!dgWorld::TickCallback(threadID=1) Line 1013   C++
    newton.dll!dgMutexThread::Execute(threadID=1) Line 57   C++
    newton.dll!dgThread::dgThreadSystemCallback(threadData=0x18d71900) Line 201   C++
    newton.dll!std::_Bind<1,void *,void * (__cdecl*const)(void *),dgThread *>::_Do_call<,0>(_Myfargs={...}, __formal={...}) Line 1149   C++
    newton.dll!std::_Bind<1,void *,void * (__cdecl*const)(void *),dgThread *>::operator()<>() Line 1138   C++
    newton.dll!std::_LaunchPad<std::_Bind<1,void *,void * (__cdecl*const)(void *),dgThread *> >::_Run(_Ln=0x01f5e850) Line 196   C++
    newton.dll!std::_LaunchPad<std::_Bind<1,void *,void * (__cdecl*const)(void *),dgThread *> >::_Go() Line 187   C++
    newton.dll!_Call_func(_Data=0x01f5e850) Line 28   C++
    newton.dll!_callthreadstartex() Line 376   C
    newton.dll!_threadstartex(ptd=0x18db16e8) Line 359   C
    kernel32.dll!@BaseThreadInitThunk@12()   Unknown
    ntdll.dll!__RtlUserThreadStart()   Unknown
    ntdll.dll!__RtlUserThreadStart@8()   Unknown



Since this probably has to do with contact.. my game relies a lot on NewtonContactJointRemoveContact - if this is in any way related?

[JernejL]

Adding additional piece of info that is probably relevant - NewtonWorldForEachBodyInAABBDo is reporting same issue aswell - it reports bodies that were deleted and are no longer valid inside the iterator callback.