How to make mouse picking to work correctly?

[Jedive]

I am using Newton 2.0 beta on the iPhone, basically to perform mouse picking on some objects. I used the mouse picking code provided with the samples as reference, but I don't seem to be able to get the values correctly.

After receiving a touch event on the screen, I am using gluUnProject (throught the iGLU library) to get the ray coordinates in the near and far clipping planes. Once I have the ray, I use NewtonWorldRayCast to get the picked body which is closest to the camera.

If I get it right, the way in which the RayCastFilter callback works is that, the closest point of the ray is assigned a value of 0.0, and the far point of the ray is given a value of 1.0. The intersectParam value of the function receives a value between 0 and 1 which is the normalized distance from the origin at which the collision occured.

So, following the sample code, I have the closest ray point in a dVector named p0, and the disntant point in a vector named p1. By doing dVector p( p0 + (p1 - p0).Scale( intersectParam ) ), I should get the exact point of collision, isn't it? The problem is that the resulting value is not at the point of collision as it is expected, but closer to the camera instead.

[Julio Jerez]

WorldRaycast deal with point in global space.
if you are using point in Camera space from glUnproject you nee to transform the to global space before you pass then to worldRaycast.
check teh doc on glUnproject for the iphone

[Jedive]

The results given by gluUnProject seems to be correctly in world space. I made two tests and logged the results to be where the start and end points for the ray are, and they are correct.

In my engine, I use a right-handed coordinate system, with the Z axis pointing upwards. Since Newton uses a left-handed system with Y pointing upwards, I am just swaping the Y and Z coords before sending them to Newton, and swap the result again before giving it back to the engine, so the problem is not there.

In the tests I made, the position picked is the center of the screen. The first test has a viewer in the position (0, 0, 0), with an euler angle of (0, 0, 0). The results given are:

Viewer pos: 0.000000, 0.000000, 0.000000
Viewer rot: 0.000000, 0.000000, 0.000000
Ray start: 0.000000, 1.000000, 0.000000
Ray end: 0.000000, 24.999987, 0.000000

The ray starts one point in front of the camera (because that the near clipping range), and moves forward to 24.999987 (the far clipping range is 25). Since the viewer is placed at the origin and unrotated, there is no difference between the camera and the world space here, so let's make another test:

For the second test, I moved the viewer 8 points backwards, and made it look 35 degrees upwards (across the X axis). These are the results:

Viewer pos: 0.000000, -8.000000, 0.000000
Viewer rot: 35.000000, 0.000000, 0.000000
Ray start: 0.000000, -7.180848, 0.573576
Ray end: 0.000000, 12.478789, 14.339405

It seems that the coordinates given are in the global space, isn't it?

The strange thing is that, to make a comparison, I tried the picking with another physics library (Bullet), and gives the same problem, with the picked result not placed at the same position as the triangle picked, but more like in the middle of the ray between the camera and the collision point. This obviously means that there's something wrong in my engine not related to the physics lib at all. I'll probably port the application to PC to be able to use a full-fledged OpenGL with the built-in GL picking functions, and compare the results with the Newton ones...

[Julio Jerez]

why do flip values? that migh be what is wrong. you do not have to do any flipping at all. The laws of algebra are no right or left handed,

as to pick a point from the screen do:
pick the coordenate for the screen using any of the API function, basically this is the relative mouse cursor.
the point value of the point in the viewport window space, you need to convete it to the homogeneos viewport space by untransforming by the viewport scales and origin.
the depth value will be zero at the front clip plane and 1 at the far clip plane, with this information you can make two points
p0 (xs, yx, 0, 1)
p1 (xs, ys, 1, 1)

using the projection matrix, untranform these two points to view space
pv0 = untrasform (Projectionmatrix * p0);
pv1 = untrasform (Projectionmatrix * p1);

now you can unproject the by dividing by the w value from the preview trasform

p0 = (pv0.x / pv0.w, pvx.y / pv0.w, pvx.z / pv0.w, 1.0);
p1 = (pv1.x / pv1.w, pvx.y / pv1.w, pvx.z / pv1.w, 1.0);

now you have a 3d point in view space,
you need to trasform them back to global space by untraforming by the GlobalViwMatrix

p0 = untrasform (GlobalVeiwCamnera * p0);
p1 = untrasform (GlobalVeiwCamnera * p1);

now you have two points in global space,
with these two point you can do a raycast and the parametric value, is the the hit point bewten p0 and p1
OpenGL glUnPrect is a utility function that encapsulate that sequence of operation, but you can do it yourself by using the OGL functions Get,
My guess is the some how you are getting the screen coordenate incorrently.

Please give me a favor do not make more references as to how other technologis do stuff
If you are going to use newton and simple book in basic newtonian physics and basic algebra is more the enought.
I am not invention brand new laws, so I will no know how they do it, you have to ask them.