# Drag 3D camera with mouse cursor

I have a camera that is moved by changing its X and Z coordinates. However, the camera is rotated, so when an object moves by 1 unit in the world, the number of pixels it moves by isn't trivial to determine.

How could I move the camera so that if the pointer is at a set point, that point will be under the cursor after the cursor moves?

I have a PerspectiveCamera and the number of pixels that the pointer was dragged in each direction (X and Y) on the screen.

• Just to validate my understanding: the camera a mostly top-down camera but with a tilted perspective, and you want to always keep it at the same height. Is that correct? Jun 18, 2019 at 21:04
• @chaosed0 yes except that I’d also like to add two finger pinching/zoom but I figure that once I understand how to do panning I’ll understand how to do that too Jun 19, 2019 at 11:57
• I think it is already being answered here stackoverflow.com/questions/26185074/… Jun 19, 2019 at 14:41
• Thanks @SaadAnees, the last answer is useful and I've almost got it (just need to figure out why a Raycaster won't work on a Plane). Still, it would be nice to have a definitive up to date answer so that people in the future don't struggle for hours like I have- if I figure it out on my own I will write it Jun 19, 2019 at 16:01

Here is a fiddle hopefully demonstrating the effect you want to achieve. It can get quite jittery on different machines - likely because of javascript nuances I'm not familiar with - but I think it's close to what you're looking for.

https://jsfiddle.net/chaosed0/b4cqktan/37/

Here is an explanation of the math involved. We have two constraints:

• Keep the camera at a constant height.
• While the mouse cursor is down, "move" the ground to keep the same location under the cursor at all times.

"Move" is in quotes because we don't want to move the ground, we want to move the camera. Thing is, moving the camera in the opposite direction we would move the ground has the exact same effect, and I find it more intuitive to picture it as if the ground were moving.

With this knowledge, we move on to solving problem #1: what ground position is the mouse is over? First, we get the location of the mouse in world-space:

var mouseScreenPos = new THREE.Vector3();
mouseScreenPos.x = ( event.clientX / window.innerWidth ) * 2 - 1;
mouseScreenPos.y = - ( event.clientY / window.innerHeight ) * 2 + 1;

var rayOrigin = (new THREE.Vector3(mouseScreenPos.x, mouseScreenPos.y, 0.0)).unproject(camera);


Then we obtain the direction of the ray under the mouse by adding a bit of depth to get a point "more under" the mouse, and subtracting that from the world position:

var rayPos1 = (new THREE.Vector3(mouseScreenPos.x, mouseScreenPos.y, 1.0)).unproject(camera);
var rayDirection = rayPos1.sub(rayOrigin);


Finally, we can use these two points to get the location in space we want to hold constant. For my purposes I'm calculating where the ray intersects the XZ plane at y = -5, but this may differ for you depending on where your objects are located.

var ySlope = (planeY - rayOrigin.y) / rayDirection.y;
var xIntersect = rayDirection.x * ySlope + rayOrigin.x;
var zIntersect = rayDirection.z * ySlope + rayOrigin.z;

return new THREE.Vector3(xIntersect, planeY, zIntersect);


Now, we solve problem #2: how much to move the camera? This ends up being fairly easy now that we have the above formula for obtaining the ground position under the mouse. The algorithm goes like this:

• When mouse down is detected:
• Record mouseDownPosition, the current ground position under the mouse
• Record mouseDownCameraPosition, the current position of the camera
• When a mouse move is detected:
• Obtain the new ground position under the mouse and subtract it from mouseDownPosition.
• Subtract that from mouseDownCameraPosition.
• Set camera.position to this final result.

As a final note, I think the Raycaster can aid in the case where there's variable-height terrain, i.e. where we don't know what planeY above should be upon mousedown. However, we still need to calculate the ray/plane intersection when the mouse moves, with planeY set to the Y of the point the raycaster found. Otherwise, our first constraint (keep camera at constant height) will be broken.

• Thank you for this, I will give you the bounty and mark the answer as accepted when I implement it Jun 22, 2019 at 16:09
• Can someone please help me solve the same problem in Unity. Nov 24, 2020 at 21:23

What you want to achieve is called camera panning.

When you use your mouse you are working with a 2d vector in screen space pixels. In order to correctly transform mouse 2d to world 3d coordinates you can use the well known viewport Project method (should be available at your camera object) or you can even use it the other way around and Unproject your 3d position to 2d screen space position. That's also how raycasting for object selection works. Unproject is used for mouse rectangle selection in RTS games. The 3d coordinates get unprojected into screen coordinates and checked if inside the mouse 2d rect.

A 3d camera usually consists of a viewmatrix and a projectionmatrix. The cameras orientation and position is stored inside the viewmatrix. In order to move your camera to the right or left independent of it's current rotation you have to do the translation relative to the cameras forward vector.

So for panning the camera relative to mouse drag works as follows:

1. Save click position as initialMousePos.
2. Ondrag save mouse position as currentMousePos
3. Project both positions via camera.screenpointtoray to 3d vectors
4. Calculate delta = currentMousePos-initialMousePos
5. Translate camera with delta vector. Your camera is rotated which means you want to translate relative to it's forward vector by calculating the cross product of camera forward with delta first. Correct me if I'm wrong on this one.
6. Set initialMousePos=currentMousePos
7. Repeat on next update
• Could you include more detail please? 3. I'm not sure what screenpointtoray is- I have Vector3#unproject(camera) and am I doing that for both vectors? If so, how? The pointer only has an X and Y and it's a Vector3 Jun 18, 2019 at 14:50
• I recommend you to read the first Chapter of the book game eengine architecture. It explains the different spaces and maths very detailed. Screenpointtoray is a method from unity game engine that will calculate a ray from mouse coordinates. The ray starts at the near plane and ends on the farplane.its basically a ray as long as the camera can "see" and it can be used to perform intersection tests with 3d objects in the 3d world, for example ray plane intersection. Jun 18, 2019 at 18:06
• @lolums unproject will have 3 coordinates, the X and Y of the screen and Z depth in the screen between 0 - 1. When you project the mouse location onto screen. You will need to supply the X and Y, sometimes the Z which in some functions assumes to be 0 . The accurate solution is a bit more complex. If you want the exact world location of mouse pointer, you will need to project a ray into your world from the mouse/camera position using camera direction. Then use a ray/plane collision test. This type of collision test will give you the most location the object should be at under your mouse. Jun 19, 2019 at 4:08