# How to calculate rotation needed to cancel out perspective shift

I have a direction vector that's pointing to (0,0,1) in local object space. How can I calculate rotation needed to rotate it on Y axis so that from camera's point of view it would look as if it's pointing up, while in world space it would stay projected on (0, 1, 0) plane.

When object is directly in front of camera, it behaves as I want to, however, moving it elsewhere changes the appearance due to perspective effects. It seems like I need to cancel out the perspective shift by rotating the direction vector.

In the attached screenshot, orange lines show the desired behaviour.

Edit 1: Some context on why I'm trying to do this. I'm working on a puzzle game and want to add support for joystick controls. Each controllable object has handles above that should get activated dependent on the joystick input. I found that because of the perspective shift, selecting appropriate handle is a bit awkward. For example, in the screenshot, input is (0, 1) - directly pointing up, however, input translated to object space (yellow ray) is not hitting the expected handle (highligted in orange)

• Why do you need to do this? I ask because it might help us determine the best way to solve the problem for your specific needs. Commented Feb 23 at 1:47
• @Kevin thank you for suggestion, I've added context to the main post. Commented Feb 23 at 17:17
• Would using a larger collider on your controls mitigate this issue? It looks like the ray is close. It would probably save you some complex math. Commented Feb 24 at 12:41

There's probably a neat mathematical way to do this, but here's the kludge I've used when I needed this in the past:

1. Project the object position to a screen position using Camera.WorldToScreenPoint

2. Increase the y coordinate to get a point on screen above the object

3. Turn this point into a ray with Camera.ScreenPointToRay

4. Use Plane.Raycast to see where this ray hits the object's ground plane

5. Subtract the object's position from the hit point to get a vector in the ground plane that's vertical from the perspective of the camera - you can use Quaternion.LookRotation to rotate in that direction.

(If the ray misses the plane, you can fall back on facing directly away from the camera instead - this only happens if you're looking up from under the object, or exactly edge-on along the plane)

Note that this rotation cancels out the perspective shift and the opposite only. The horizontal direction should not be shifted at all - so really you're dealing with a shear, not a rotation. If you need to map all joystick directions to "on-screen" directions, you can use a shear matrix, or use the current joystick direction as the screen-space shift in step 2 before firing the ray.

• Thank you for the answer, this did the trick. Commented Feb 29 at 22:34