what pofiler people use?

A place to discuss everything related to Newton Dynamics.

Moderators: Sascha Willems, walaber

Re: what pofiler people use?

Postby Julio Jerez » Tue Mar 15, 2016 7:52 pm

Here is a graph that compere the new optimization to the original. It is almost too god to be believed. :mrgreen: :mrgreen: and there is more to come. :shock: :shock:
Untitled.png
Untitled.png (87.79 KiB) Viewed 9588 times
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby JoeJ » Wed Mar 16, 2016 2:18 am

Whoooo! :mrgreen:
User avatar
JoeJ
 
Posts: 1453
Joined: Tue Dec 21, 2010 6:18 pm

Re: what pofiler people use?

Postby godlike » Wed Mar 16, 2016 5:07 am

That's awesome! It seems lots of things happening at the moment in the repository and that's good.
User avatar
godlike
 
Posts: 58
Joined: Sun Mar 16, 2014 3:48 am

Re: what pofiler people use?

Postby Julio Jerez » Wed Mar 16, 2016 7:35 am

yes I am getting to the last planned changed for 3.14 before making is official and start 3.15
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Mon Jul 09, 2018 1:16 pm

Ok I decided to put some time of making the profiler,
I found that the google visualizer is too limited, the moment you take a capture if a few hundred frame is simple bug down to a crawl and it even crashes.

so I decided to make a visualize to present the sat similar to what the google visualize does,
I am trying window form, but wow, I only have one world to describe windows form in C++, it absolutely *. what a wasted for time that was.

no wonder Microsoft abandoned, what a terrible wasted of time that was.
I am thinking to use either imGui, or MFC.
I am no spending more time on window form. is like pulling teeth without Novocain.

The good knew is that I already discovered a weak point in the thread system and I fixed already.
anyway I will try this viewer with a different GUI again.

My inclination is using imGui
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby JoeJ » Mon Jul 09, 2018 5:54 pm

Julio Jerez wrote:My inclination is using imGui


Should be fine, i do it this way by drawing progress bars.

But it can be done better, i think MeltingPots link (https://bitbucket.org/wolfpld/tracy), that profiler uses imGui as well.
User avatar
JoeJ
 
Posts: 1453
Joined: Tue Dec 21, 2010 6:18 pm

Re: what pofiler people use?

Postby Julio Jerez » Tue Jul 10, 2018 9:47 am

yes that profiler seen very cool, I download and try to compile it but it uses feature of c++ 11 and higher that are no covered by visual studio 15 and lower version.

I already made the framework using imGui, and right out of the box I forgot why I was no using in teh first place, ImGui does not have a file browser,
there are some demos ou there, but there are lot of code, but since this is an stand alone application I thin I can download on of those. I am hoping it still work with the version I have, because teh later imGUI they try to go fancy with multi thread with thread_local graphic context and the result is that no long van yo render form a diffrent thread.

I would probably write my own file browser it is become too difficult.
on the bright side teh profile can be multiplatform.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Fri Jul 13, 2018 4:06 pm

I made some progress on the profiler visualizer,
if anybody want to check it out, is it project ../../newton-dynamics\sdk\dProfiler

there are two project the visualize and the server, the serve si the one the application link to the application, and the visualize is the one the show the data.

still does now do anything, but is enough to show what the final product will be.

if people want to check it you can do this.
1-Run the sandbox demo for a few second so that I take a trace.
2-Build the profiler visualizer, and open the capture.

is does not show the capture yet I will do that this weekend, but is show the navigation interface.

I have to give to the people who wrote imGui, it is quite impressive. the more I use it the more I like it. I will have to go to their forum and give the some prop and thanks, maybe even a donation.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Mon Jul 16, 2018 2:00 pm

I now have the profiler data structure completed, this is how is look like
there is no navigation yet, I will do that and the udp control this week,
profiler.png
profiler.png (14.79 KiB) Viewed 8778 times
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Wed Jul 18, 2018 10:26 am

I was try to use the profiler for teh first time and I already found some very good candidate for very straight forward optimizations. Tha make me think the time put in making the simple profiler was well spend.

It still need lot of work, but this this is only for internal use and maybe for anyone who wnat to try it, since it is a stand alone tool.

there are onle three major points to add until a resume work on teh engine
1- adding zoom and fix to screen
2- prune empty spaces, (optional)
3- udp socket so that traces can be taken form the tool at will.
4-optional, removed zlib, because the newton core soudl not depend on any external library
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby JernejL » Sat Jul 21, 2018 3:01 am

Julio Jerez wrote:I was try to use the profiler for teh first time and I already found some very good candidate for very straight forward optimizations. Tha make me think the time put in making the simple profiler was well spend.

It still need lot of work, but this this is only for internal use and maybe for anyone who wnat to try it, since it is a stand alone tool.

there are onle three major points to add until a resume work on teh engine
1- adding zoom and fix to screen
2- prune empty spaces, (optional)
3- udp socket so that traces can be taken form the tool at will.
4-optional, removed zlib, because the newton core soudl not depend on any external library


New straightforward optimization candidates sound very good, i'm looking forward to simulate more bodies per core :D
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Re: what pofiler people use?

Postby Julio Jerez » Sat Jul 21, 2018 6:57 pm

I have good news in tha area.
here is a picture of the late optimization I made to the paraller solver, it gained 40% doing the same work.
Basically the optimization is simple no issuing many small job to teh thread pool, instead place the job together and have each job block waiting for all the other thread to complete teh small task, no yielding, instead doing a special spin lock.
The reason that since the task are so small adding the as job has a cost similar to the overhead of the
thread support on teh operation system. in some case the parallel operation has a negative net gain.
This prevent the system form scaling linearly with teh core count, because teh more thread the more time the OS take to displace the cores.
and this is in window, in system liek Linux and OSX the operation system is more than twice slower, some time even 5 time, basically you can not using fine granularity on a PC.

My solution is to pack mane small task into a large tack so eth teh overread for the Task switch is not a major factor. The result seems very, very good so far

SolverOptimization.png
SolverOptimization.png (59.68 KiB) Viewed 8731 times
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Sat Jul 21, 2018 7:49 pm

The next optimization in to do teh same for the collsion system, if you at this scene
Untitled1.png
Untitled1.png (237.97 KiB) Viewed 8729 times


as you can see the scene it a rest, but is takes form 0.9 to 1.5 ms to update.
I expect to be not more than 0.5 ms at most.

if you look at the profiler trace, you can see than teh solve take almost no time at all, by teh collision system spend a lot of time for nothing.
Untitled.png
Untitled.png (21.38 KiB) Viewed 8729 times


there are two optimizations that can e apply here,
1-as you can see most the missing time is in task switching executing small job, I believe those can be combined.
2-The second part is that collsion system is a bottom up pruning system similar to a sweep and prune.
these system can not take advantage of island that are already a rest.
if teh algorithm is modified so that is a top down, then large island tha are no moving outside there OBB can no possible make pair, so tha can cut a significant amount of time for scene with many bodies that are in fact at rest, but that the system has to visit in a brute fore mode.

I will make sure the current otpmization is robust so that I can commit it, and then I will do the other two in reverse order.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby Julio Jerez » Sun Jul 22, 2018 7:50 pm

I now added the same optimization to the plugin solver, and for the first time I can verify that indeed
AVX code is about 35% faster than SSE, it seem intel claim are true, but you need to work really hard to achieve it, basically any kind of memory or OS overhead kill that gain.
here is a capture of the two solver doing the same thing, it is no cherry picking, the result are consistent across the entire capture of 600 frames.
Untitled.png
Untitled.png (27.63 KiB) Viewed 8711 times


I have not check teh AVX2 yet but I expect less for an impact, this is exciting because I was no able to see these result in practice, not even using the Intel vtune profiler whi in my options is useless BS.
why Intel has not make an instrumented frame capture profiler in 20 year beats me.

edit:
I just trace the avx2 and indee avx2 is about 10% faster tha avx and about 40% faster than plain sse.
this is along what I was expecting, and I am guessing in 64 bit it will be another few % faster because 0the register presure is a big issue in 32 bit mode with only 8 register both float and interferes. here is an AVX2 trace.
Untitled1.png
Untitled1.png (18.81 KiB) Viewed 8709 times
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: what pofiler people use?

Postby JernejL » Mon Jul 23, 2018 1:05 pm

This is amazing, this goes all into next stable release? which version is to be released?
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

PreviousNext

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 13 guests