When moving the "camera" should one move the projection matrix or move the world?

Question

I'm making my own game engine as an exercise and I've gotten really confused with what I've read so far.

When I want to move my camera do I simply just move the world or move the position of the camera?

OpenGl tutorials and scratchapixel state that you move the world and not the camera. But that really confuses me wouldn't it be simpler to just move the camera instead of for example applying millions of additions in worlds as big as minecraft? What about VR headsets? Those have 2 cameras how do they do it?

Is there something I'm missing or it just works like this?

Answer 1

OpenGL and Scratchapixel do not state that you need to move the world. The editors of the documents you are referring to spent a great deal of time explaining the process in detail and you are simplifying it down to a statement that doesn't make much sense out of its context.

• you have objects say in world space. I am not even talking about the object-to-world transform here.
• then you have the camera. You need to move the camera around in world space to frame part of the set. This gives you the cam to world matrix.

When you are ready to render, what you do is this:

1. you transform the vertices of the scene objects using the world to cam matrix, which as you have noted, is the invert of the cam to world matrix. This has for effect of moving the vertices in front of the camera (what you refer to as moving the world), as if the camera had never been moved from its default position (centered around the world origin looking down the z-axis). We say that the vertices are then in camera space.

   Think of it that way: "you move the entire block (the geometry + the camera), so the camera returns to its default position. Because the scene and the camera moved together, what you see through the camera isn't changing. But the benefit of this transformation is that the maths for projecting the vertices onto the camera screen (step 2) are greatly simplified (mostly a matter of dividing the x and y coordinates of the vertices by their z-coordinate. But we use the perspective projection matrices because it does a little bit more than that for us: it takes into account the frame aspect ration, field of view and remap the depth of the vertices to 0-1 in a non-linear fashion. That's why we use the perspective projection instead of a simple z division).

2. then you project the vertices that are in camera space at this stage, into screen space using the perspective projection matrix. This leads to transforming points that are original 3D coordinates into screen coordinates (2D points, aka position on the flat screen + eventually a depth value if you need it).

In a real-time graphics pipeline (in the shader), this generally looks like this:

vec3 vertex_in_screen = 
    perspective_mat * 
    world_to_camera_mat * 
    vertex_in_world_space;

When you mean that we move the world in front of the camera you assume item 1. in the process described above.

In this order. I hope this can help you understand the process. I think this is very well defined and explained in the references you are pointing to.

Answer 2

First, the matrix in question is the "view matrix," which yields the scene coordinates relative to the camera. The "projection matrix" is responsible for the actual warping of the objects as a function of distance to give the perspective projection from which we see the world with our eyes/cameras.

In my mind, the "move the world, not the camera" is a false dichotomy. Ultimately, the job of the graphics pipeline is to get arbitrary scene coordinates mapped onto pixel coordinates.

Consider an orthographic projection that maps the screen to a region that is 100 meters wide (say, for a top-down 2D game). If this camera is at x = 0m, an object at x = -50m will be at the left edge of the screen, and an object at x = +50m will be at the right edge of the screen. The orthographic projection matrix takes care of actually turning the range of [-50, +50] meters into (X, Y) pixel coordinates.

If you have an object at x=1500m, and a camera at x=1550m, you need to map the position of the object to the left edge of the screen regardless of which way you slice the problem. The simplest way to achieve this is to simply subtract the camera position from the object position, which leaves you with the object's position relative to the camera (in this case, x = -50m). In that sense, there really isn't such a thing as "moving the camera," because whatever overall transformation you use still have to map the scene to the screen in an equivalent manner. You could encode the transformation in the projection matrix instead of the view matrix, but they're still multiplied together to yield the same final result. Thus, whether you're moving the world or moving the camera is more a matter of semantics than a practical one.

And yes, every single one of the millions of vertices on-screen need to be subject to this transformation. GPUs are designed to take advantage of the fact that transforming all these vertices is a highly parallelizable workload, and are able to do that enormous amount of math in the very short time in between frames.

Answer 3

You move the world because there is no camera.

The camera is a useful analogy in tutorials, but it's important to understand that an analogy is all that it is: it doesn't actually exist in any 3D API.

When people talk about "moving the camera" what they're really doing is taking that camera transform and applying the inverse of it to everything in the world. Instead of, for example, moving a camera forward by 10 units, what really happens is that the world moves backwards by 10.

This isn't a lot of work for a GPU. Each object you draw will typically have a single concatenated MVP (model-view-projection) which contains everything needed to position and orient the object in the world and on the screen. The per-vertex operation is just a simple vector/matrix multiplication.

Answer 4

If your aim is to see how things look relative to a camera perhaps you could think of it like splitting the task into two steps, first being computing how things are positioned relative to it and its orientation and the second being computing how objects look relative to the camera in this standardized coordinate space. Then the second part could be solved once for all positions so to speak.

If your camera is really weird and works differently in different positions of your world, I guess you can't split it up like this and need some kind of nonlinear function which takes both the camera and object to compute how things would look.

Answer 5

You always move the world.

As stated above, the camera itself is just an idea that helps you abstract the concept of view, movement, etc. Therefore sometimes you may have multiple "cameras" (or vectors in space for that matter) to help you switch your view of the world.