making wrappers for other languages

A place to discuss everything related to Newton Dynamics.

Moderators: Sascha Willems, walaber

making wrappers for other languages

Postby Julio Jerez » Fri Sep 03, 2021 3:40 pm

I am using swig for making a python wrapper for a Blender addon.
I used swig in the pass and I really like it.
I already have the wrapper generated and convert almost the complete engine, but this has forced me to make some changes.
the first is that in all garbage collection languages like Python and csharps, objects memory allocations.

this is a problem because in the engine side, I decided to simply memory alignment by making it 32 bytes.
I found this a reasonably compromise for a physics library since objects are not really that small, so assuming just one alignment for every thing is fine to me.

this means if something alloc 1 byte object, it the allocation will pad it with 32 bytes.

this immediately created a problem with the wrapper, because if it make something like a vector
the wrapper does not make an instance, instead it keeps a pointer to an allocated object.
but since those object do not have memory allocators, the wrapped call the generic new and delete, but the generic new and delete does not allocate 32 bytes aligned memory chunks.

of course this can eassy be fix by adding a global operator new like this.
Code: Select all
void *operator new (size_t size)
{
   return dMemory::Malloc(size);
}


but I consider that a very hacky and cheap solution

the proper solution is to make all class derived from the dClassAlloc calls so that the yget the operator new and delete.
so that fixed that problem.

however this brings a new problem.

this changes make visual studio 2015 and lower ignore inline and __inline operator and make all the math functions call with very catastrophic performance problems.

fortunately Visual studio 2017 does not have that problem. so I guess it is time to abandone VS 2015 compatibility.

to summarized this class

Code: Select all
class dVector : public dClassAlloc
{
   #define PERMUTE_MASK(w, z, y, x) _MM_SHUFFLE (w, z, y, x)
   public:
   D_INLINE dVector()
   {
   }
}

makes dVector() a function call in vs2015
but this class below doesn't
Code: Select all
class dVector
{
   #define PERMUTE_MASK(w, z, y, x) _MM_SHUFFLE (w, z, y, x)
   public:
   D_INLINE dVector()
   {
   }
}



but with VS2017 but classes result on inline functions.

anyway I went over the code and made most classes subclasses of dClassAlloc so that the wrapper are happy with the expected alignment

now it is time to test it with the blender addon project.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: making wrappers for other languages

Postby Julio Jerez » Fri Sep 03, 2021 9:13 pm

on a secund though another way is to just use inline instead of __force inline. and let the compiler decide what to or to not inline.

I did it already and I can't see any big difference at a macros level. I am sure that a profile will find some.
but what I like is that using inline, we are more cross platform and beside new compilers are very good at deciding what function are good candidate for inlining so is probably better to delegate that decision to the compiler.
Plus we can still use VS 2015
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: making wrappers for other languages

Postby JernejL » Tue Sep 14, 2021 1:54 am

Is it possible to create a wrapper which looks and functions similar to old c api with same methods?
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Re: making wrappers for other languages

Postby Julio Jerez » Tue Sep 14, 2021 6:21 am

Do you mean Makin a c interface?

The two things that swig lack is languages translation. To C or Delphi pascal.
In general I think the state away from languages that are not garbage collected.

However I know that some people had made those translator modules. I while back I was searching for the pascal and I found some in Google code, but them I lost interest since there where wrappers already.

Would a conversion to pascal be more useful to you than a translation to C?

I bet there are some translators in github that do that.
If there aren't, the swig doc explain the rules for how to make one.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: making wrappers for other languages

Postby Julio Jerez » Tue Sep 14, 2021 6:35 am

I enter c++ to pascal in Google. And I found this link
https://wiki.freepascal.org/SWIG

It seems that since swig 3.xx they support translation to pascal.
I look like some revision had some bug but I think it is possible to make automatic wrapper for pascal.

The coll thing about swig, is that it is a one fix all methods

Once there is a working script, all it need to do is change the command like to make a wrapper for a different language.

Is that what you mean.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: making wrappers for other languages

Postby JernejL » Thu Sep 16, 2021 5:53 am

if it was possible to create a identical api like original C api dll that would work well already, swig for delphi / pascal seems incomplete.
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia

Re: making wrappers for other languages

Postby Julio Jerez » Thu Sep 16, 2021 12:29 pm

I think we need to find a way to make automatic translators.

Until 3.xx updating the c app was alway major time consuming part of the development time.
It was so a sort of bugs, and by the mid dev became a bottle neck that inhibit that explosion of many internal features.

You can see that with the custom joint library many people use the buil in from 1.xx and I alway had to keep inviting on them to use the c++ version.

I will take a look at swig to see how reliably is the Delphi or the c translation.

The worse part is that is probably not well supported, but the good think is that swig can be extended.

If this is the case, I see if I can make a c translation unit.
Since I am not versed on Delphi pascal, but from there it should be easy to make one fir pascal.

But maybe the one they have for pascal is sufficient, I will check it out.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: making wrappers for other languages

Postby Julio Jerez » Thu Sep 16, 2021 12:59 pm

I look at the latest github swig https://github.com/swig/swig

and you are right they does no seem to be any kind of support for Pascal.
I read a lot of people of experimental work, like it is in eversion 2.0.8 but as usual this is all misinformation.
so it seems swig does not support translation to c or pascal. which is a shame.
and to make even worse, they do not have cmake or visual studio support either.
that means that adding that functionality to swig is a project outside the scope of newton.

so it seem the onle way would be makin a manual translation, or see if they are some other tool that convert c++ to C. ther has to be some out there.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: making wrappers for other languages

Postby Julio Jerez » Fri Sep 17, 2021 10:40 am

I look some more, and I found a project named gccxml

This convert cpp headers to a c format in xml.
Maybe we can used that to make a pure c interface.
And maybe even pascal.
I will check the gighub to see how fixable it is for us.

the newer version is called https://github.com/CastXML/CastXML

It seem very small, but that could be misleading, they say tshi nee lot of special LLMV stiff, so I am no sure if is work with Visual studio, but if it does, then we can use this to make automation c wrapper that can be use as the base for for the language not supported by swig.
maybe we can even make direct conversion to pascal.

I will try the download this weekend.
It is not a zlib license so I can't put it in the download.

edit: as I suspected it is a linux / mingwg thing
it serach to ton of preinstall stuff and generate these error, even when using VS2019 whis support Clang


Detecting CXX compile features - done
CMake Error at CMakeLists.txt:83 (find_package):
By not providing "FindLLVM.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "LLVM", but
CMake did not find one.

Could not find a package configuration file provided by "LLVM" with any of
the following names:

LLVMConfig.cmake
llvm-config.cmake

Add the installation prefix of "LLVM" to CMAKE_PREFIX_PATH or set
"LLVM_DIR" to a directory containing one of the above files. If "LLVM"
provides a separate development package or SDK, be sure it has been
installed.

so, so much for that.
is is such a shame how the open source comminute is far, far more restrictive with these moronic rules that the close source. the open source community has become a cabal of thugs and bullies.
I truly despite them.
Julio Jerez
Moderator
Moderator
 
Posts: 12249
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: making wrappers for other languages

Postby JernejL » Mon Oct 04, 2021 1:30 am

A simple plain-C would work in pascal, there is no need for it to be an OO wrapper.
Help improving the Newton Game Dynamics WIKI
User avatar
JernejL
 
Posts: 1578
Joined: Mon Dec 06, 2004 2:00 pm
Location: Slovenia


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 10 guests