Official Pascal-Header (SDK 1.53), last update 26.05.2006

A place to discuss everything related to Newton Dynamics.

Moderators: Sascha Willems, walaber

Postby JernejL » Mon Jan 02, 2006 12:57 pm

Sascha Willems wrote:See line 35 :
Code: Select all
{$I delphinewton.inc}

It's used for compatibility with other Pascal compilers like FPC.


i saw that, otherwise i wouldn't know it is missing the inc file, just that i can't see any use of its definitions..
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Postby Sascha Willems » Mon Jan 02, 2006 1:01 pm

Well, I must admit that I very rarely use any .inc files in my Delphi projects at all, but the one that comes with the newton headers is mainly there for compatibility reasons with other Pacal compilers. And besides this it's a standard JEDI-File so that the header can also be included in the JEDI-Package.
User avatar
Sascha Willems
Moderator
Moderator
 
Posts: 346
Joined: Fri Aug 27, 2004 10:18 am
Location: Germany

Postby JernejL » Mon Jan 02, 2006 3:13 pm

i come across a problem, in delphi newton 1.5 works great, but in freepascal win32 it doesn't, i get a run-time error 202, which developers in #fpc say it is stack overflow, this happens in newton.dll, i tried mine and sacha's header and i get the same error, BUT a week older version of BETA newton.dll works in freepascal, so there is definetly something going on in the new newton.dll, the first newton call fails:

Code: Select all
   

        NewtonAllocMemory = procedure (sizeInBytes: integer ); cdecl;
        PNewtonAllocMemory = ^NewtonAllocMemory;

        NewtonFreeMemory = procedure (ptr: Pointer; sizeInBytes: integer ); cdecl;
        PNewtonFreeMemory = ^NewtonFreeMemory;     

function NewtonCreate (malloc: PNewtonAllocMemory; mfree: PNewtonFreeMemory): PNewtonWorld; cdecl; external newtondll;
 
NWorld := NewtonCreate(nil, nil);
 


the very same code works with 1 week older newton beta but not with 1.5 release - NewtonCreate causes the stack to blow up. whats going on? is anyone else having this problem?
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Postby Julio Jerez » Mon Jan 02, 2006 3:35 pm

yes there new engine used a local variable of 756 kbyte of the stack.
This is used for all of the computational geometry stuff (convex hull calculation and optimizer and other things)

This is absolute necessary for multithreading, each function must be practically re entrant, and before I was using the solve memory pool for that kind of things. But there was causing old kind of hard to track bug does to memory been trump by a call back calling some collision function or creation and objects or thing like that.

The problem is that now the engine trust that the stack space will be at least that long, in MSV the stack is ground incrementally on demand, for that the compiler issue some dead code at the beginning of each function that is using more than 4k bytes of local variable.
While reading the documentation on Visual Studio 8 I learned that this option called stack probe could be compiled out and I found out that this could be done in VS 2003 from the command line

The problem is that if a function try to assess none committed stack it will crash.

I could be that in Pascal the stack is not committed and that is why are crashes in the DLL.
You could do few thing.
- see if you can coming stack space (ugly but it may work)
- create a dummy functon that will commint the stack by creating a local variable of 800k and callong menset. (also ugly)
- If that dopes not work send me a demo for me to test and see when is the best pace to coming the stack from the dll internally.


I do not really understand what is the goal of Microsoft of allocating 1 mega Byte by default of stack fro each program, and yet the stack is not committed and they commite it 4 k at a time but calling and expensive function and adding that to each function of a program. (another of those solution in search of a problem).
Why not commit the stack at the beginning f the CRT?


in you sample try this

Code: Select all
void stackProbe()
{
char pool[756* 1024]
memset(pool, 0, sizeof pool)
}


        stackProbe()
        NewtonAllocMemory = procedure (sizeInBytes: integer ); cdecl;
        PNewtonAllocMemory = ^NewtonAllocMemory;

        NewtonFreeMemory = procedure (ptr: Pointer; sizeInBytes: integer ); cdecl;
        PNewtonFreeMemory = ^NewtonFreeMemory;     

function NewtonCreate (malloc: PNewtonAllocMemory; mfree: PNewtonFreeMemory): PNewtonWorld; cdecl; external newtondll;
 
NWorld := NewtonCreate(nil, nil);
 
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Postby Sascha Willems » Mon Jan 02, 2006 3:49 pm

Just set the stacksize in FPC to something above 1 MByte. That did the trick for me. Go to Options->Memory and just add a zero at the end of the stack size and then recompile.
User avatar
Sascha Willems
Moderator
Moderator
 
Posts: 346
Joined: Fri Aug 27, 2004 10:18 am
Location: Germany

Postby JernejL » Mon Jan 02, 2006 5:02 pm

Sascha Willems wrote:Just set the stacksize in FPC to something above 1 MByte. That did the trick for me. Go to Options->Memory and just add a zero at the end of the stack size and then recompile.


but do you think this is proper way doing it? i can only threat this as a bug in newton, this will IMHO confuse many people.
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Postby Julio Jerez » Mon Jan 02, 2006 5:27 pm

Delfi wrote:but do you think this is proper way doing it? i can only threat this as a bug in newton, this will IMHO confuse many people.

I agree with you.
However the stack size is define by the application. It just happens that visual studio and in each function a function to expand the stack. It really hampers the performance of tight loops.
The engine expands the stack on initialization but maybe for some reason when working in Pascal the committed stack is not sufficient.
If you have time please make and exe that I can test to see what can be done about that.

I did it is a naive way by setting a small funtion like teh one I showed abobe but is is possible teh the optimization comopled the funtion out and the stack is not been expanded at all.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Postby JernejL » Mon Jan 02, 2006 5:58 pm

ok, here is example app, first download this archive that has the data files as well (the exe i compiled with delphi 4):
http://www.gtatools.com/TDC/DEMO.rar

then download this EXE, which is compiled with FreePascal:
http://www.gtatools.com/TDC/tdcFPC.rar

when the FPC compiled version calls NewtonCreate the stack will blow up causing a runtime error 202.
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Postby Julio Jerez » Mon Jan 02, 2006 7:32 pm

It looks like FPC only commits less than 256 kilo byte of stack space.
I do not think I can change that so easily unless I change eth code to do runtime allocation and de-allocation, which is not good because eventually lead to memory fragmentation and gradual slow downs.

Here is my function for probing the stack at the from NewtonCreate

Code: Select all
#ifdef _MSC_VER
#pragma optimize ("", off)
#pragma check_stack (on)
#endif
void dgApi dgStackProbe()
{
   dgInt32 i;
   dgInt8 buffer[1024 * 100 * sizeof (dgFloat32)];
   memset (buffer, 0, sizeof (buffer);
}

#ifdef _MSC_VER
#pragma check_stack (off)
#endif

I thought it was optimized out but that is not the problem because the crash happen on debug too and it is a stack overflow.
I reduced the size to see what is the default size created by the application and it is about 250k which is not enough for the engine.
The reason this was working before I was using a global memory buffer for the calculations but with the call back been able to call the collision system and do many other thing there are memory clash.
Another thing I noticed is that the double precision version became significantly slow by using arrays on the stack instead of global pointer.
Why it is a problem to set the stack size to 1 mega byte is the compiler?

Edit:
I was playing your game a litle, itis really cool.
The camera is so high that the people look tiny. also the guy is so strong that he can push the car sideways.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Postby Aphex » Tue Jan 03, 2006 7:27 am

Could try using EDITBIN to change the exe stack size for verification..?
Aphex
 
Posts: 144
Joined: Fri Jun 18, 2004 6:08 am
Location: UK

Postby Sascha Willems » Tue Jan 03, 2006 9:36 am

Well, I just took a look at the documentation for FPC and I saw that there is a global compiler directive for setting the memory sizes, including the stack size. And since the stacksize under Windows and Linux (dunno for Mac) is only dependant on the main memory, it shouldn't be any problem to just override the default value by putting that directive into the header and setting the stacksize to something above 1 MBytes. But then again that should be documented, cause maybe people do weird stuff with the stack and set their own size.

So what do you FPC-users think of this "solution"? Should I put that directive into the header?
User avatar
Sascha Willems
Moderator
Moderator
 
Posts: 346
Joined: Fri Aug 27, 2004 10:18 am
Location: Germany

Postby JernejL » Tue Jan 03, 2006 10:02 am

Julio Jerez wrote:It looks like FPC only commits less than 256 kilo byte of stack space.
I do not think I can change that so easily unless I change eth code to do runtime allocation and de-allocation, which is not good because eventually lead to memory fragmentation and gradual slow downs.

Here is my function for probing the stack at the from NewtonCreate

Code: Select all
#ifdef _MSC_VER
#pragma optimize ("", off)
#pragma check_stack (on)
#endif
void dgApi dgStackProbe()
{
   dgInt32 i;
   dgInt8 buffer[1024 * 100 * sizeof (dgFloat32)];
   memset (buffer, 0, sizeof (buffer);
}

#ifdef _MSC_VER
#pragma check_stack (off)
#endif

I thought it was optimized out but that is not the problem because the crash happen on debug too and it is a stack overflow.
I reduced the size to see what is the default size created by the application and it is about 250k which is not enough for the engine.
The reason this was working before I was using a global memory buffer for the calculations but with the call back been able to call the collision system and do many other thing there are memory clash.
Another thing I noticed is that the double precision version became significantly slow by using arrays on the stack instead of global pointer.
Why it is a problem to set the stack size to 1 mega byte is the compiler?

Edit:
I was playing your game a litle, itis really cool.
The camera is so high that the people look tiny. also the guy is so strong that he can push the car sideways.


a possible workaround would be if dll could detect if the stack is too small and show a messagebox with details on the cause, this way programmer just adjusts stack space and it would all work again :)

glad you liked the little game i am working on, there are ofcourse some things i'm working on to improve - like the actor behavour, etc.. but i really forgot to include readme.txt in that demo.rar, since there is more you can do than just walk around, i added the readme.txt into the archive: http://www.gtatools.com/TDC/DEMO.rar :)
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Postby Sascha Willems » Thu Jan 05, 2006 9:35 am

Important Note :
By accident I included an older version of the custom joints unit which was using an old function name and therefore gave errors on compiling it. A user just pointed that out and I uploaded a fixed version. So if you want to use the custom joints with Pascal, please redownload the headers.
User avatar
Sascha Willems
Moderator
Moderator
 
Posts: 346
Joined: Fri Aug 27, 2004 10:18 am
Location: Germany

Postby aidave2 » Mon Jan 09, 2006 8:56 am

thanks for these Delphi headers Sascha!
great work.

and the demos are very helpful too!
User avatar
aidave2
 
Posts: 56
Joined: Sun Jan 08, 2006 7:09 pm

Postby rick_dodd » Mon Jan 09, 2006 12:39 pm

Is it possible to get a copy of the 1.32 header? Some code I have is behaving funny in 1.5, but worked in 1.32 (or at least seemed to be). I suspect I doing something I shouldn't be :oops:. 1.32 was OK with it, but 1.5 doesn't like it. Having the 1.32 headers (I have the .dll) might help me pin point the problem.

Thanks.
rick_dodd
 
Posts: 4
Joined: Mon Jul 18, 2005 2:50 pm

PreviousNext

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 12 guests

cron