# Hybrid perspective / orthographic projection

I'm trying to figure out how to setup camera projections such that they fulfill following requirements:

• main scene is rendered with perspective projection
• camera / viewport controls allow only zoom (scale) and pan (translate)
• when zooming and panning, the image "does not change", as if we were scaling/panning orthorgraphic projection
• the image does not loose quality when zoomed in

So a naive solution would be to render the main scene to FBO, then use that as a texture for a quad, that translates and scales depending on zoom/pan interaction. This however results in low-quality, as I have to pre-set resolution for that FBO texture. That's not what I want.

What else can I try?

• Do you also want to be able to rotate the camera? Zooming and panning can't be made equivalent to rotating. Commented Jul 7, 2023 at 19:12

To change the zoom without changing the image, modify your field of view angle. This effectively scales the image up or down, without changing the geometric relationships in the image plane — if two lines cross at a particular point, they'll continue crossing at that same point, with no parallax effects that move the apparent intersection relative to other image features.

To translate side to side or up and down (truck and boom/jib movements — perpendicular to the view direction), you can use an oblique camera frustum. This effectively shifts the image after the projection has been applied, so again you'll see no parallax. The effect can be strange though, if you use a large shift: you'll see an image projected from a viewpoint distant from the location in the center of the image, and you'll see more dramatic perspective distortion the farther you go from the camera's axis.

Taking the classic OpenGL projection matrix:

$$\left[\begin{array}{cccc} { \dfrac{2n}{ r-l } } & 0 & { \dfrac{r + l} { r-l } + x_{shift}} & 0 \\ 0 & { \dfrac{2n}{ t-b } } & { \dfrac{t + b}{ t-b } + y_{shift}} & 0 \\ 0 & 0 & -{\dfrac{f+n}{f-n}} & -{\dfrac{2fn}{f-n}}\\ 0 & 0 & -1& 0\\ \end{array}\right]$$

The $$\x_{shift}\$$ and $$\y_{shift}\$$ variables there control the oblique frustum. When they're zero, you get the usual projection, with the camera axis passing through the center of the image. As you adjust these values, you'll slide the image side to side without altering the perspective.

• Thanks for this, you seem to understand the perspective projection matrix more than I do. Commented Jul 7, 2023 at 8:17

So the solution at the end is trivial. Simply premultiplying the perspective projection matrix, with a translation matrix, moves the rendered image around.

• for panning, premultiply the perspective projection matrix with a translation matrix
• for zooming, apply zoom factor while calculating effective FOV and sides of the perspective projection matrix

With THREE.js this is simply:

class HybridCameraImpl extends THREE.PerspectiveCamera {

controlMatrix: THREE.Matrix4;

constructor(fov?: number, aspect?: number, near?: number, far?: number) {
super(fov, aspect, near, far);
this.controlMatrix = new THREE.Matrix4();
}

updateProjectionMatrix() {
super.updateProjectionMatrix();
if (this.controlMatrix) {
this.projectionMatrix.premultiply(this.controlMatrix);
this.projectionMatrixInverse.copy(this.projectionMatrix).invert();
}
}
}

// Using it
cameraRef.current.zoom = s;
cameraRef.current.controlMatrix.makeTranslation(-tx, -ty * aspect, 0);
cameraRef.current.updateProjectionMatrix();
$$$$
`