Perspective projection and ultrawide resolution

Question

The problem I'm going to describe did not occur to me until very recently, when I got a super ultra-wide monitor (32:9 aspect ratio) and tried playing videogames on it.

TL;DR: the scenery is stretched near the left and right edges of the screen due to how projection onto a plane works. Question: why isn't the standard to use "spherical" canvas instead of a plane?

I tried several games and I noticed that the objects displayed far left and far right on my screen are stretched and appear way too closer than they actually are. I could not figure out any general setting (field of view, focal distance...) to mitigate this uniformly, until I realized how projection works.

The way 3D objects are displayed on a 2D canvas is as follows: the camera consists of a point (focal point) and an infinite plane (canvas). To calculate where another point in front of the camera will be displayed, this point is connected to the focal point and the point drawn on the canvas corresponds to the intersection of this line with the canvas.

No wonder the far left and far right parts of the screen are so stretched! The rays from these vertices intersect the canvas at a very small angle.

This problem would be eliminated if we chose the canvas to be a sphere, ideally with the same curvature as the monitor (in case the monitor is flat, the sphere degenerates into a plane and we have the same situation as before). That way, if we choose a correct position for the focal point (depends on our screen setup, curvature, aspect ratio etc.), no stretching should occur, since the focal point is literally "surrounded" by the canvas, instead of the canvas running away from it in all directions.

So my question is, why isn't the standard to implement this "spherical" projection instead of "planar"? Is it because it happens on such a low level of coding that developers don't care, or the code with spherical projection would be a serious bottleneck compared to what is done now? Or the market with super ultra-wide resolution is so niche that they don't care, but we should expect to see it sometime in the future? Or the standards are so baked in that nobody cares about implementing something new, even if it would make more sense?

Sorry in case this question doesn't quite fit in here, I actually don't know anything about making videogames, but this strikes me as it's more about geometry than actual game-making.

Answer 1

why isn't the standard to implement this "spherical" projection instead of "planar"?

It looks to me like you answered this yourself here:

"in case the monitor is flat, the sphere degenerates into a plane"

That is, if most people are using flat screens, then the planar projection is geometrically correct (from the intended viewpoint).

Yes, even if the flat monitor is ultra-wide. If the projection axis and field of view are chosen correctly for where the viewer is seated, then the stretching of the objects at the edge of the screen exactly compensates for the way the edge of the screen itself foreshortens and recedes into the distance away from the viewer, resulting in correct perspective when viewed from that viewpoint. But it definitely looks strange if you stand back far enough that you can see the whole monitor without significant foreshortening to balance-out the stretch in the image.

So as you say, the right way to render is to use a projection surface with "the same curvature as the monitor." And most monitors are flat, so a flat projection plane and linear perspective makes sense the vast majority of the time.

If spherically-curved monitors became a large segment of the market, then doing specialized projections for them would likely get correspondingly more popular.

As things stand, only a few types of games are routinely played on curved/wraparound screens - mainly racing and flight sims - so these types of games may sometimes implement special projections and curvature configuration to serve the segment of their players who own these specialized cockpit rigs. Often these projections will be cylindrical, rather than spherical, to match widescreen monitors that curve in one axis but not both.

Just like with the flat case, these projections are correct only from one specially-chosen viewpoint, and will look similarly distorted if you look at them from further away.

There are also other strategies for rendering to ultra-wide displays, such as:

• The Panini Projection (which you can find integrated into the Unreal Engine as an optional feature)

• Barrel Distortion

These projections aren't technically geometrically correct for any viewpoint - they apply a kind of compromise distortion so that the image looks plausible enough from a range of viewpoints, which can help if you know your player has a super wide display, but you don't know exactly how it is curved or exactly where their head is in relation to it (as you would with a cockpit simulator).