Newton on Windows Store and XBOX

A place to discuss everything related to Newton Dynamics.

Moderators: Sascha Willems, walaber

Newton on Windows Store and XBOX

Postby zak » Tue Jan 03, 2017 2:47 pm

I ported my engine to Universal Windows Platform (UWP), because i plan to publish the soccer game i'm developing on Windows Store and XBOX.
I am one of the first users adopted Newton more than 10 years ago and as i can see the first compiled it for UWP.
Well, Newton passes UWP certification when compiled in single threaded (defined DG_USE_THREAD_EMULATION) but failed in multi-threading because of pThread.

UWP is the new Windows platform, shared between Windows 10, Windows Mobile, XBOX, Hololens, Surface HUB, Windows IOT. It is sandboxed, based on COM, and not allows to call old WIN32 API ( with some exceptions). And this is the problem with pThread, it calls Win32 Api such as GetProcessAffinityMask().

So if Newton wants to jump on Windows Store and XBOX, pThread is to be replaced by a more standard multithreading api (c++11?)
zak
 
Posts: 87
Joined: Mon Dec 06, 2004 9:30 am

Re: Newton on Windows Store and XBOX

Postby Julio Jerez » Tue Jan 03, 2017 3:37 pm

Ha thank for reminding me about that.
Yes is about time with upgrade to standard thread provided by c++ 11;

are you in the last newton?
I have to do two thing for Newton 3.14
1-Implnem support for stardard Thread of C++ 11. and maybe remove pthread or make optional like the option Thread Emulation.

2-make the memory manage thread safe, as it is only thread safe when allocation or free new memory chunks, by not when allocating new items form a chunk. This work well when doing thread and doing micro thread by fail miserable when running asynchronous. One of my goal is to make the asynchronous update the standard more and get rid of NewtonWaitFor Frame to complete.

I will do 1 this weekend. it should be very easy since all thread support is well encapsulated.
Thank for the reminder.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Newton on Windows Store and XBOX

Postby Julio Jerez » Wed Jan 04, 2017 2:07 am

al right I now committed the support for native c++ 11 threads.
It was quiet eassy to add it, no more that a dozen line of code changes.

we only lost the ability to set the priority of the worker threads one level higher that the parent thread. This is a good idea to remove some of the thread eviction that some time happens when a thread is executing a task, but is no really critical.

I am hopping that native thread are much better in OS like Linux and OSx,
I have notice that when using pthread is OSX and Linux the overheard of waking up a worker thread is so huge that some time multithread code run slower I fteh task do no have enought work to do.
I windows despite whet people say threading overhead is of the order of ten of time faster that in OS like Linux and OSX.

anyway C++ tread si no the default mode for thread, and ther is a

#define DG_USE_PTHREADS
for enabling pThreads. In time we will probable get rid of that as we test all the OSs.

please try again and let me know what else you need.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Newton on Windows Store and XBOX

Postby zak » Wed Jan 04, 2017 6:08 am

oh, well. I will try it.
Is it so now?
- DG_USE_THREAD_EMULATION --> single thread
- DG_USE_PTHREADS --> multithreading with pThread
- default, no define --> multithreading with C++11

I am on 3.14 of some months ago and is stable. Is the current version ready for production?
zak
 
Posts: 87
Joined: Mon Dec 06, 2004 9:30 am

Re: Newton on Windows Store and XBOX

Postby JoeJ » Wed Jan 04, 2017 7:27 am

I noticed a floating point trap in the standart joints demo, this did not happen a month ago.
Uncommenting all the AddJoesRagdoll setups should help to reproduce it.
User avatar
JoeJ
 
Posts: 1453
Joined: Tue Dec 21, 2010 6:18 pm

Re: Newton on Windows Store and XBOX

Postby Julio Jerez » Wed Jan 04, 2017 11:45 am

oh yes thank joe, I keep leaving demos commented out and I when I make these changes I forget to uncomment them for test. Fortunately this was something I knew was there and this was a good time to fix. :D

Zak yes those are the define now.
- DG_USE_PTHREADS will be there only until I test all other OS with C++ 11 thread.
then pthread will be removed altogether and I will set platforms not supporting C++11 thread like VS 2010 to run in emulation mode only. So thank for the tip.

to make the final stable 3.14 I still have two more pending things.

1-bring the softbody demo to a stable state, and have some usable demo.
2-make the fixed size pool base manager manage thread safe, so that I can remove so of the locks that I have is some place and this start to get forgettable and tedious.
but for all purpose the current version can be consider release candidate.

question. what kind of high performance computing does UWP supports, directx 12 I assume.
but does it support others?
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Newton on Windows Store and XBOX

Postby Julio Jerez » Wed Jan 04, 2017 4:05 pm

actually I was so impressed with the simplicity of use c++ 11 threads that I went ahead ad removed pthread dependency.
now the set is like it was before, thread is the default and platforms that do not support C++ 11 threads or do not support threads can define macro DG_USE_THREAD_EMULATION in the command line

this is getting simpler and simpler as time progresses. :mrgreen:
Thank you for the suggestion.
if you sync the VS 2015 and 2010 project do not link to pthread I will fix 2013 projects tonight.

please let me know if you need any more requirement to be compliance.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Newton on Windows Store and XBOX

Postby JoeJ » Wed Jan 04, 2017 4:26 pm

Julio Jerez wrote:I keep leaving demos commented out and I when I make these changes I forget to uncomment them for test.


Yep, but this is something that may look very unprofessional to someone downloading for the first time.
You should keep my problem test cases commented out - why showing them anyone.
But mainly you should comment out crashing demos from the drop down list. Please :wink:

Maybe you can use a define that is set only on your machine?
User avatar
JoeJ
 
Posts: 1453
Joined: Tue Dec 21, 2010 6:18 pm

Re: Newton on Windows Store and XBOX

Postby zak » Wed Jan 04, 2017 7:27 pm

OK, Newton with C++11 multithread passes UWP certification. :D

JoeJ wrote:question. what kind of high performance computing does UWP supports, directx 12 I assume.
but does it support others?

For what I know OpenCL is not supported.
The choice for parallel programming in UWP is
- compute shader ( DirectX 11 or 12 )
- C++ AMP ( but i have some doubts that it is more or less dead )
zak
 
Posts: 87
Joined: Mon Dec 06, 2004 9:30 am

Re: Newton on Windows Store and XBOX

Postby Bird » Wed Jan 04, 2017 7:48 pm

JoeJ wrote:
Julio Jerez wrote:I keep leaving demos commented out and I when I make these changes I forget to uncomment them for test.


Yep, but this is something that may look very unprofessional to someone downloading for the first time.
You should keep my problem test cases commented out - why showing them anyone.
But mainly you should comment out crashing demos from the drop down list. Please :wink:

Maybe you can use a define that is set only on your machine?


Hi Julio,

One of the big reasons why Git is so popular is that it's so easy to make branches to work on new and experimental code while leaving the stable "master" branch intact for people to download and use in production. I think it would be helpful to all of us if you could use that approach for Newton.
https://www.youtube.com/watch?v=H5GJfcp3p4Q

Thanks for adding c++11 thread support and the new soft body works looks fantastic.

-Bird
Bird
 
Posts: 623
Joined: Tue Nov 22, 2011 1:27 am

Re: Newton on Windows Store and XBOX

Postby Julio Jerez » Wed Jan 04, 2017 8:11 pm

point taken , I need to be more careful yes.
Bird wrote:... new soft body works looks fantastic.
-Bird


hey Bird I can no with to see some of eth soft body stuff in you plugin.
I am not bring the cloth up to the same level.

this framework is for all kind of particle based simulation, but my biggest emphasis for 3.14 is getting cloth that is capable to solve a fully clothed person with semi tight fitting cloth that run at real time.

this module will with rigid bodies, but it will also be standalone, meaning it can be attached to an animated mesh and it will carry its own collision shapes.

for example if you have a model of a woman wearing that can play animations,
you will be able to attack cloth mesh, and them select the bone you want and attach newton collision shapes to approximate the body of the person.

then in a call back you will be abler to animate these collision shape to follow the bones, as the character animates the cloth should follow the model with proper friction and elasticity.
I am talking extreme high quality physically based simulation, not the Varlet or Position based or shape matching mombo jumbo people are doing these days.

Zak
- C++ AMP ( but i have some doubts that it is more or less dead )

yes C++AMP is just a wasted of time, incomplete. they try to make so transparent that it became useless. each time to do an operation on a buffer you do not have a way to lock or unlock, they try to second guess the programmer, so it end up coping data between GPU and CPU many time over.

other that nBody demo that the use to demonstrated very difficult to do anything practical with it.
I deleted all the AMP I had too hard to master for high performance. I rather spend the time on direct X if I have to.
anyway I was just gathering info.
My guess is that when it is the time, I will have to do two drivers one for OpenGL where version the have and one for direct compute.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: Newton on Windows Store and XBOX

Postby Julio Jerez » Fri Jan 06, 2017 3:51 pm

um one of the problem that I can see is that, c++ thread do no have semaphore objects as native primitive, you have to emulate the with a mutex and a conditional variable.

This is function and ON for the main thread by really disastrous for the thread pool.
Threard pool when the take a tack the has to sync waiting for the other thread to finish, and the do that by waiting in a common semaphore.

a semaphose is an in between a spin lock and a mutex in performance cost.

a mutes is an interpose that can take form several hundred thousand for several mullion instructions.
a semaphores in window is implemented with a spin lock and a mutex, where the code spins for a while and if it do not get release it call the mutex. the make so that mut the time the muster is almost never executes.
use a spin lock is also disastrous because the spin lock pin the bus with a atomic other cores can not get assess has to way for the but to get free, but the is only eve other instruction because the spin is a tight loop.

I implement the semephare in a naïve way using only a mutex, but the is terrible for the three poll an did actually run slower. I will have to do a better just, ether get rid of the mutex or see how I can emulate it like windows does.

I still like better thought because al least is cross platform, OSX does not sup[ort semaphores and you have to use a trick to get a equivalent which is a mutex anyway.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 18 guests