Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to rotate a line to point to the mouse's world coordinates (rather where the mouse lies on the plane where y=0).

I am running into a couple of problems though:

  • Whilst it follows the mouse the line doesn't always lie directly beneath the cursor, which is what I would expect to see. It seems to be perfect when the position is at the origin and less accurate the further away from the origin I get.
  • The rotation doesn't go from 0 - 360, rather 0 - 180. I think the solution here is to use a dot product but the first problem is preventing me from fixing this problem.

Here's where I calculate the angle by which to rotate:

Calculating mouse position

 Vector3 near, far, dir;
 unproject(*mouse_, near, far);
 dir = near - far;

 float t = -near.y / dir.y;
 Vector3 point = near + (t * dir); //where the mouse lies on the plane y=0

I'm almost certain the above is correct as when I draw a point at point the rendered point is directly beneath the cursor on the plane y=0

Calculating angle between the line and the point.

 rotation_.y = angleBetween(position_ - dir_, position_ - point);
 rotation_.y = radToDeg(rotation_.y);

position_ is where I want the line to start from. Which is the player's position at the moment.
dir_ is the direction of the line, set to and never changed Vector3(0.0f, 0.0f, 5.0f);

 float angleBetween(const Vector3& a, const Vector3& b) {
     float cosTheta = dot(a, b) / (length(a) * length(b));

     return acos(cosTheta);

Here's how I am drawing the line:

 glTranslatef(position_.x, position_.y, position_.z);
 glRotatef(rotation_.x, 1.0f, 0.0f, 0.0f);
 glRotatef(rotation_.y, 0.0f, 1.0f, 0.0f);
 glRotatef(rotation_.z, 0.0f, 0.0f, 1.0f);

 glColor3f(0.0f, 1.0f, 1.0f);

 glVertex3f(0.0f, 0.0f, 0.0f);
 glVertex3fv(end_.toArray().data()); //toArray() returns and std::array of the vector


And just in case it matters, for the camera I do:

void Camera::transform() {
    gluLookAt(position_.x, position_.y, position_.z,
              target_.x, target_.y, target_.z,
              0.0f, 1.0f, 0.0f);

target_ is also the player's position.

I hope this is enough information.

share|improve this question
up vote 1 down vote accepted

Regarding this bit of code:

rotation_.y = angleBetween(position_ - dir_, position_ - point);
rotation_.y = radToDeg(rotation_.y);

First of all, if dir_ is truly a direction vector, it shouldn't be subtracted from the position, so you'd just call angleBetween(dir_, point - position_);. (I reversed the subtraction in the second argument so that it's calculating the vector from position_ to point, which I think is what you wanted.)

Second, the way you want to calculate the angle between these vectors is probably something like atan2(b.x, b.z) - atan2(a.x, a.z) rather than the method you used. As you noted, the acos(dot product) method will always pick the shortest distance around the circle and give a positive angle; the atan2 method should give a correctly-signed result so that you'll rotate either clockwise or anticlockwise as appropriate. Also, this will automatically project things into the XZ plane, which is appropriate since you're always going to be rotating around the Y axis. (The dot product version of angleBetween will give an angle around whatever axis is perpendicular to the two vectors, which won't be the Y axis if either vector has a nonzero Y component.)

share|improve this answer
Aha, thank you. I was getting close to solution with my scribbling. I realise what I was doing now, thank you. I will try using atan2 for the angle now. – Lerp Jul 31 '12 at 21:40
Sorry to bother you but does atan2 give the rotation around whatever axis is ommited from the calculation? I.e in your example y was ommited and it gave the angle on the y axis? – Lerp Jul 31 '12 at 21:49
@Lerp Yes, that's right. In 2D, atan2(y, x) gives the angle from the +X axis, going counterclockwise around the origin. In 3D, I just changed y, x to x, z to account for the fact you're using Y as up and XZ is the "ground" plane. – Nathan Reed Jul 31 '12 at 21:51
Brilliant, thank you! – Lerp Jul 31 '12 at 21:53

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.