i'm currently trying to work out some 3d picking code that I started years ago, but then lost interested the assignment was completed (this part wasn't actually part of the assignment).

I am not using the mouse coords for picking, i'm just using the position in 3d space and a ray directly out from there. A small hitch though is that I want to use a cone and not a ray.

Here are the variables i'm using:

float iReticleSlope = 95/3000; //inverse reticle slope
float baseReticle = 1; //radius of the reticle at z = 0
float maxRange = 3000; //max range to target
Quaternion orientation; //the cameras orientation
Vector3d position; //the cameras position

Then I loop through each object in the world:

Vector3d transformed; //object position after transformations
float d, r; //holder variables

for(i = 0; i < objects.length; i++) {

    transformed = position - objects[i].position; //transform the position relative to camera
    orientation.multiply(transformed); //orient the object relative to the camera

    if(transformed.z < 0) {

        d = sqrt(transformed[0] * transformed[0] + transformed[1] * transformed[1]);
        r = -transformed[2] * iReticleSlope + objects[i].radius;

        if(d < r && -transformed[2] - objects[i].radius <= maxRange) {

            //the object is under the reticle

        } else {

            //the object is not under the reticle


    } else {
    //the object is not under the reticle

Now this all works fine and dandy until the window ratio doesn't match the resolution ratio. Is there any simple way to account for that

  • \$\begingroup\$ I am not exactly catching what coordinate are you using, but if we turn everything into world coordinate, then screen change won't matter, right ? \$\endgroup\$
    – zinking
    Jun 13, 2012 at 6:30
  • \$\begingroup\$ Yes, everything is being transformed into world coordinates. The effect I am getting is an illusion due to the reticle not changing size when the window size changes. So it looks as though something outside the reticle is being targeted in some cases or that something isn't that should be. So I guess I'm asking how to distort the reticle to match, what I have right now is (window.width - resolution.width) / resolution as a percentage to distort it by and i'm on the right track I think. I guess I just needed to write it all out some where... \$\endgroup\$
    – Wolftousen
    Jun 13, 2012 at 16:04

1 Answer 1


I ended up changing the reticle to be rendered by the game instead of the GUI. All is well now and glad my math was solid from the beginning.

  • 2
    \$\begingroup\$ Please accept your own answer, and, if possible, explain a bit more why the change you made fixed the problem. \$\endgroup\$
    – Eric
    Oct 12, 2012 at 13:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .