what pofiler people use?

A place to discuss everything related to Newton Dynamics.

Moderators: Sascha Willems, walaber

Re: what pofiler people use?

Postby Julio Jerez » Mon Jul 23, 2018 2:25 pm

it is all committed to GitHub, to set the plugin is you can some like
Code: Select all
void* preferedPlugin = NewtonGetPreferedPlugin(world);
NewtonSelectPlugin(world, preferedPlugin);


the engine will load all plugin found in folder ../newtonPlugins relative to the executable.
../newtonPlugins/debug
../newtonPlugins/release
you may need to copy the by hand. or make a post build rule to do it your project.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Mon Jul 23, 2018 3:01 pm

ha an BTW I just try it on my compere at work which is an older E5 -1650 XEON processor.
is not suppose to be faster that mine at home which is a Icore7 7700
by boy I have never seen such a dramatic performance jump in newton.
that seem in the video takes approximately 24 ms using one core, and about 9 ms with four cores.

the difference is that with the optimization newton is now a Multicore engine rather than a multithreaded engine.
since this xeon has 6 cores, it leaves two extra core for the OS, while mine at home using 4 cores, only leave hyper threaded cores to the system.

The moral is that to get optimal perforce the application should leave at least one core for the OS to responded. I tested by setting 8 or 16 cores and the solve is become unresponsive and goes at about 1 fps or less.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby JoeJ » Mon Jul 23, 2018 3:41 pm

Julio Jerez wrote:The moral is that to get optimal perforce the application should leave at least one core for the OS to responded. I tested by setting 8 or 16 cores and the solve is become unresponsive and goes at about 1 fps or less.


Maybe you could prevent this by still using some yields so the OS gets some CPU in any case?
User avatar
JoeJ
 
Posts: 1453
Joined: Tue Dec 21, 2010 6:18 pm

Re: what pofiler people use?

Postby Julio Jerez » Mon Jul 23, 2018 3:57 pm

yes I actually do that, this is the function

Code: Select all
inline void dgParallelBodySolver::Sync()
{
   if (m_threadCounts > 1)
   {
      //DG_TRACKTIME(__FUNCTION__);
      dgInt32* const ptr = (dgAtomicExchangeAndAdd(&m_syncIndex, 1) / m_threadCounts) & 1 ? &m_sync1 : &m_sync0;
      dgAtomicExchangeAndAdd(ptr, 1);
      dgInt32 count = 0;
      while (*ptr % m_threadCounts) {
         count++;
         dgThreadPause();
         if (count >= 1024 * 64) {
            count = 0;
            dgThreadYield();
         }
      }
   }
}


as you can see the lock spin to 64000 loop, of that does not respond the is call thread yield
this fixed the lock when using more threads that core available in the sense that let the application to be responsive.
The is more of less what mutex do, but the lock is no personally a Mutex, is more the equivalent of a Sync function that we see in langyages like CUDA or OpenCL.
I hate to have to re invent a special Mutex but I can not trust OS Mutexes to get the functionality that I am seeking, this is because a Mutex will do a Task switch the movement is call from different thread and that is precisely what I am try to avoid.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Mon Jul 23, 2018 4:07 pm

This is the new set of optimizations.
profiler.png
profiler.png (38.14 KiB) Viewed 8861 times
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Tue Jul 24, 2018 9:37 am

ok I now reorganized the collsion update, and the three tidy update and remove old contact is at the end of the update, this was all the work that is done by send job to teh tread pool have happen one aft the other which mean we can now combined.

here is the capture of one frame, and you can see the time spend in task switching is about the same as teh time in the update, and that scene has 1000 bodies, thsi cost is fix whether there are 1 or 1000 bodies, its after 1000 bodies that the cost start to be small tha the cost per body.
this is approximately 0.5 ms of dead time. I will optimize this the same way than teh solve and wee where we are.
Untitled1.png
Untitled1.png (19.75 KiB) Viewed 8847 times
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Fri Jul 27, 2018 9:42 am

ok I know fuse teh majority of the job done by the collision system,
Untitled.png
Untitled.png (29.21 KiB) Viewed 8803 times



as you can see there are fewer task switches, and teh entire collsion cost about the same as a task switch, I am very disappointed of c++ 11 native thread support,
they do no allow for setting the thread prioristy as a result the respon of a worker thread is a random vamdon time for 100, top abpout 500,000 clock cycles.

threfore if you have many workld thread is very much gurantee that it will be abpout 500, 000 clock wast.

I kind of what to re enable pthread support, is lower level and and work thread respond is almos instantanes for hiegher priorithread.

afte I am done wit ethe final toaches I will try to re enable pthread, many make it optional onle I can find out a way to set a thread priority. becaus eoteh that that a newton update will continue to be 0.5 ms even when doing nothing.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Fri Jul 27, 2018 10:58 am

apparently is possible to get the native thread handle wit this m_handle.native_handle()
so I added this code to othe worker thread initialization
Code: Select all
#if (defined (_WIN_32_VER) || defined (_WIN_64_VER))
   SetThreadPriority(m_handle.native_handle(), THREAD_PRIORITY_ABOVE_NORMAL);
#endif


but it does not seem to have any effect.
I need to check is the call back to see if it is no overwritten
anyway if we can get the native handle than no nee to re enable pthread, so that's good.

this is how the update look now, it is abut 400 million ticks faster, by is dominated by force and rtrque and pre listener.
Untitled.png
Untitled.png (17.65 KiB) Viewed 8802 times
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Sat Jul 28, 2018 12:33 pm

this is a capture with all teh optimization active.
is a scene with 1024 ball is a tight container when teh are all colliding, the scene does not goes to rest
so collsion as solver are always working.
as you can see ther si oen weak point whi is buidl teh island every frame.
it is about 25% of total time.
I have thought a lot about coming up with an incremental island generator but that has always been difficult, however as we can see forming island should has a cost similar to solving or doing the collisions.
Untitled.png
Untitled.png (14.15 KiB) Viewed 8785 times
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Sat Jul 28, 2018 2:08 pm

The spawn tree algorithm, has k * n log ( n)
Time complexity, but if k is large the when forming a large set of bodies that all form a single island, you can see the large effect of k * log(n)
Before the these large island were sent to one core so the effect of making the island was not a favor.
Not that the solver can do large island in parallel, build the island is a factor.
It is time to replace the spanning forest with disjoint set algorithm.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Sat Sep 08, 2018 11:42 pm

ok guys here is the result of all the optimizations combined.
all optimizations on


when optimization started


I believe it is plenty fast for now and it not longer suffer from the drastic slows down when large stacks became activated. Now I will move to polish the engine features than has been neglected for a while.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby JernejL » Sun Sep 09, 2018 1:22 pm

THIS IS EXCELLENT WORK!!

I'm really looking forward to this, with these timing optimizations and eventual polishing of engine features, things are looking really good for the future :)
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Previous

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 12 guests