glm::perspective() with wide field of view

Question

I'm still pretty new to this GL stuff, but I've managed to put together some C and C++ code that allows me to construct simple 3D objects consisting of line segments and display them on my laptop using GLFW and the glm library. I've hooked in enough keyboard callbacks to allow me to navigate the position and orientation of an observer via glm::lookAt(). A couple of the keys allow me to grow or shrink the field of view value that gets passed to glm::perspective(), like so:

float field_of_view; // degrees, not radians

        glm::mat4 proj = glm::perspective(
            glm::radians(field_of_view),    // field of view
            1920.0f / 1080.0f,      // aspect ratio
            0.0f, 100.0f            // near and far clipping planes
        );

From my experience as an amateur photographer, I know that if one takes a photo with a camera with a wide angle lens of a scene that contains straight line near the periphery of the field of view, the physically straight lines will appear bent in the photo. They bow outward so as to create a more circular shape around the center of view.

But when I modify the field of view that gets passed to glm::perspective() such that it is a high value (approaching 180 degrees), with an orientation such that the straight line segments appear near the edges of the field, the rendered line segments never bend like I'd see with a real camera and wide angle lens. Instead, the lines remain straight but appear to get very long.

This creates very unnatural looking images.

Is there some other glm function that I should call instead in order to make the rendered lines curved like I'd expect to see in real life?

Thanks

Answer 1

The distortion you see with a real lens isn't from the wide field of view itself, but from the the way the lens warps the light. A wide field of view makes this effect much more noticeable, but it's present at narrower angles too. If you used a pinhole camera, the results would be much closer to the straight lines you see in the computer rendering.

That's because we're using linear perspective, which models the camera's aperture as an infinitely small point - similar to the hole of a pinhole camera. All the transformations the view and perspective matrices do are linear transformations - they can only map straight lines to straight lines. Mathematically, a matrix can't do anything else. So no value that glm::perspective() or any drop-in replacement could produce would give you the curved lines you're looking for.

Similarly, the rasterizer in your GPU will only rasterize straight-sided polygons joining the vertices. So even if we add a non-linearity to the transformation of the vertices, you still won't get a rounded curve unless you subdivide the meshes finely (or use tessellation on the fly).

The most common way to implement these curving effects in games is to render to an off-screen buffer using conventional linear perspective, then apply a post-processing pass where we read that rendered image as a texture, applying a barrel or pincushion distortion to the sampling coordinates to bend the image that's presented to the screen.

Answer 2

Excellent explanation DMGregory. I'd only add that, for nearly 180° no barrel or cushion is enough, so I'd suggest this general way. Draw the scene six times to the six textures of a cube map, each one with proper modelview of the six cardinal views and FoV of 90°x90° - the eye is at the center of the cube. Then, implement a fragment shader picking up the right texel of the cube map depending on the ray direction along the two directions of a full-viewport quad. As an example, that's mine:

uniform samplerCube cubemapTexture;
uniform vec2 viewport_res;
uniform float y_stereo_offset;

void main()
{
    vec2 coord = (gl_FragCoord.xy - vec2(0., y_stereo_offset)) / viewport_res.xy;
    vec2 thetaphi = ((coord * 2.0) - vec2(1.0)) * vec2(3.1415926535897932384626433832795, 1.5707963267948966192313216916398) * vec2(-1,-1);
    vec2 c = cos(thetaphi), s = sin(thetaphi);
    vec3 rayDirection = vec3(c.y * c.x, s.y, c.y * s.x);
    gl_FragColor = textureCube(cubemapTexture, rayDirection);
    gl_FragColor.a = 1.0;
}

All the magic is done by textureCube - polar to cartesian conversion, and pick the texel from the right map among the six. y_stereo_offset can be 0 for non up-below rendering of stereo pairs.