Why are gaming graphics not as beautiful as animated movies?

Question

I remember watching the Tomb Raider pre-rendered trailer, and wished those graphics could be in the game itself.

Why there is such a big difference between the trailer and the actual game? I understand the game is a completely different concept, it has different pipeline, it has to go through different kinds of player interactions etc. I want to know what it is about games that makes rendering them so difficult compared to animated films.

So far I know that making a game and an animated movie share some basic workload, e.g. making 3d models, rendering them (only in-game it happens live). Animated movies render for a long time and we only see pre-rendered scenes. That's all I know so I hope you'll answer from that perspective!

What about animated movies rendering for hours and hours makes them so beautiful while in-game live rendering is less beautiful (from a general point of view)?

Answer 1

You already mentioned one of the central points: Time.

In the process behind rendering a high fidelity animation, multiple different approaches and algorithms are used (all usually combined under the term "Global Illumination"), with Ray-Tracing being one of the most common ones (others include for example Radiosity and Ambient Occlusion).

Ray-Tracing involves simulating a (usually high) number of light rays going through the scene and calculating their paths, their reflections and refractions when they hit objects with different materials. Different materials in return then have different physical properties that result in specific reactions for the rays (the amount of light that bounces from one objects for example is higher for a shiny objects compared to a glossy one).

Another point is Physics: Simulating thousands of hair strains in a physically correct way is time-consuming. This is why in older games, hairs are often approximated with a very rough mesh that is then textured to give the impression of hair, perhaps with some additional moving objects to make it look a bit more realistic.

Also to be considered: Memory & Bandwidth. The higher the quality of a texture applied to an object in a scene should be, the more memory you need to load and use it in a game. But not only does the system need to have enough memory to hold the data, but this data also needs to be transferred around which uses up the available bandwidth. Since memory and bandwidth are limited, there is a maximum what can be achieved.

Games often cheat a bit by only using the high-resolution textures for near objects, and use lower resolution images for the far away objects (term: MipMapping) thus reducing the needed bandwidth since less texels need to be fetched which in return increases the performance (see section about MipMapping in Apple's OpenGLES Programming Guide).

Similarly, games also often use different meshes for objects depending how far away they are with far away objects being less detailed (term: LoD = Level of Detail).

Conclusion: In real-time graphics (such as games and simulations), this detailed and complex rendering process will of course not work to produce fluid/smooth scenes. You need at least 20 rendered frames per second to achieve that fluid animation/movement effect for the human eye. On the other side, rendering a single frame (!) in an animation movie can easily take up anything from a few hours to multiple days depending on many factors such as the number of used rays in Ray-Tracing or the number of samples for Ambient Occlusion (see these Pixar page 1 / Pixar page 2 for screenshots of 16 vs. 256 samples) as well as the desired movie resolution (more pixels = more information to be calculated). Also see this article about the process behind Pixar's Monsters University animation movie, giving some interesting insights and also mentioning 29 hours render time per frame.

In general: The higher the fidelity / realism that is to be achieved, the more iterations / bounces / samples are usually needed which in return then requires more resources (both time and/or computation power / memory). To visualize the difference see the resulting render based on the number of bounces for the refraction calculation in this example: Diamond Bounces from Keyshot

But of course, the quality in real-time applications increases all the time for two reasons:

1. Stronger hardware: As the (gaming) computers get better (= more [parallel] computing power, higher data transmission between computer components, more and faster memory, etc.), the visual fidelity increases as well, because more time-consuming computations actually become feasible in a real-time system.

2. More clever methods/formulas are developed and implemented that are able to create rather photo-realistic effects without needing raytracing. This often involves approximations and sometimes pre-calculated data. Some examples:

   • Different types of anti-aliasing such as FXAA
   • Approximation to sub-surface scattering
   • Physically Based Shading (PBS) / Physically Based Rendering (PBR), e. g. in Unreal Engine 4.

Answer 2

Besides the time factor, it's worth noting that in a movie, the artist has complete control over what the viewer will and won't look at.

In your typical movie scene, the camera won't spend much time pointed at the ceiling, or pointed into a dark corner of the room, or aimed at someone's ankle, so the polygon and texturing budget for those elements will be fairly low.

If the entire scene takes place from one vantage point, the virtual set (like a real film set) doesn't need to include the parts that are behind the camera. In most games, the player is free to look anywhere at any time. That means the quality budget can be focused on what's actually seen. (Some third-person games, e.g. the God Of War series, use a restricted camera; their visuals tend to be notably better than those of more free-camera games.)

Answer 3

What about animated movies rendering for hours and hours makes them so beautiful while in-game live rendering is less beautiful (from a general point of view)?

You're assuming that the difference is simply in the render -- in an animated movie, there's also a chance for editing after the fact. They might have effects composited in that would have been difficult to have achieved in the original engine, or they may tweak things a little bit (eg, remove or repeat every 10th frame to speed up / slow down the animation).

If you get a chance, dig up one of the 'Roughnecks: The Starship Trooper Chronicles' DVDs, as they have commentary tracks by the editor & animators about tricks that they had to do when they started falling behind their delivery schedule -- things like recycling shots but flipping the axis so it wasn't as obvious, color corrections, masking off things they didn't like, adding in explosions, etc.

Answer 4

You kind of answered your own question already. Animated movies generally usually have a higher level of detail which causes a long render time for each individual frame.

Games on the other hand don't have quite as much detail in them, because the scene has to be rendered 30 or more times per second. That's also why developers try to reuse as much assets (textures, models, etc.) as possible, because rendering the same object in two positions is a lot faster than if everything would be unique. They also have to watch out not to use too many polygons in their models, and instead try to achieve an impression of depth by using lighting, textures, bump mapping and other techniques.

Movies don't have that problem; they make the scene the way they want and use as many and as detailed textures, models and geometry as they need to to achieve the scene they're looking for.

One might argue though that games are catching up. If you look at some of the best looking games in recent times, they're not as far away from movie quality as they used to be. Of course you'll always be able to cram more detail into a pre-rendered scene than a game, but I think the difference will not be as noticeable in a few years time.

Answer 5

The other answers cover the raw graphics issues in good detail, but don't mention an important part of the realism of games vs. movies and trailers: the animations and camera movements.

In a movie or trailer, each movement of people and cameras can be carefully coordinated to show just the right emotion for the moment, and they never need to be repeated. In video games, the game must be able to react instantly to the player's movements, and must reuse a small pool of stock animations in order to fill the unlimited hours of game time that can be possible.

Examples of this from the trailer in question are when the therapist nods and gives a "not bad" smile, and when Lara grips the armrest, jiggles her leg, or looks on the cavern in awe. These small things (or the lack of them, and the "plastic" characters involved) affect the perceived realism far more than minor graphics improvements.

Unlike the other graphics differences, this is not a problem that can potentially be solved by throwing more computing power at it: it's a fundamental difference between a world that's scripted and a world that responds to your every action. In this particular regard, I expect that the realism of today's trailers will exceed all gameplay for the foreseeable future.

Answer 6

Adding to the other great answers that were already posted, it is worth noting that in order to achieve the fast processing times that games need, game developers need to bake many of their visual effects as simple textures. This means that great care must be taken in avoiding effect that won't bake well.

One important effect that's hard to bake for video games is Subsurface Scattering (SSS). Unfortunately, this effect is really important in generating realistic-looking human skin. That's why many "realistic" video game characters come out looking plastic.

One way developers avoid this issue is by deliberately making the characters brightly colored to detract from the plastic look, or by adding a lot of shadows and textural details to the face (like beards etc.) to break up the otherwise large, continuous sections of skin.

Answer 7

To answer one of the questions the OP asked in a comment:

"So who should be the main culprit? Polygon level or ray tracing?"

This question is more difficult than it looks. I think a good rule of thumb is the following equation (which I made up by the way):

number of calculations = {polygons} * {light sources} * {effects}

Basically, this means that for models with relatively few special materials (i.e. no mirror, subsurf, etc.), the computation time will be large determined by the number of poligons. This is usually the case for very simple video game graphics.

In practice, however, in newer high-end games and especially movies, the culprit is "usually" ray tracing. Why? Well for two reasons. I'll give the mathy reason first, and then my opinion of the real reason at the end.

Mathy reason:

Update: This mathy explanation may not be completely accurate. Please see CrazyCasta's explanation in the comments for more details.

Suppose you have 1000 polygons and 3 light sources. The smallest number of ray traces you need to perform is 3*1000.

If we over simplify the situation and assume that 1 ray trace = 1 calculation (a gross underestimate) then we will need 3000 calculations.

But now lets suppose you want have reflections too.

The default settings for reflections in the freeware program Blender are:

max reflections  = 2
ray length limit = None
...

For these settings we can guesstimate that, in the best case scenario, this super-simple reflection will double the computational cost of your model.

But like I said before, this example is very-oversimplified, and you can find a great many effects (besides the ones I mentioned already) that will shoot your rendering time through the roof.

Case and point: try to render reflections with gloss=1 (the default in Blender), then turn the gloss down to 0.01 and compare the two rendering times. You'll find that the one with 0.01 gloss will be much slower, but the model complexity hadn't been changed at all.

My qualitative but more realistic reason:

Increasing the mesh complexity will only improve the model quality to a point. After going past a couple million faces, there really isn't a whole lot that adding more faces will do. I'll even go so far as that say that if you use smoothing, you'll be able to get away with just a couple hundred faces for most general purposes.

But the things that almost always make a difference are the lighting, material effects, and ray tracing. That's why movies will tend to use a great number of these in an attempt to approximate the beautiful complexities of the real world.

A great way to get a feel for all this stuff is to look at the cast list at the end of the latest Disney feature-length animations. You'll probably be surprised by just how many lighting and texture people they have.

Answer 8

It's worth adding that movie animation usually does a good deal of visual trickery to make perceived motion more fluid.

Animators may, for instance, use traditional hand-animation techniques which are not typically used in realtime rendering, such as smears, multiples or warping to produce more fluid-looking motion despite the lower framerate which movies are (were at any rate, until recently) displayed at. The use of smears and multiples in particular are complicated by the use of meshes - you need to produce mesh deformations for that sort of mesh distortion, and I don't believe I've seen that done in any 3d video games.

Frames with fast-moving objects passing through them may be rendered at a different framerate and then combined again to produce motion blur through the scene. Motion blur is a fairly common technique in realtime 3d graphics these days. The effect is usually not of the high quality that an animation house with dozens of processors at their disposal will obtain (see the "time" answers above) largely due to the fact that faking motion blur really well requires multiple post processing steps per layer, and a large number of intermediate frames to be really fluid.

Doing this sort of visual trickery to enhance the perceived quality with realtime graphics effectively requires limiting the output framerate to rates well below the maximum available, background-rendering intermediate frames, then doing compositing to produce the final frame. There are probably multiple schools of thought on whether the slight gains in visual fidelity for using techniques like this is worth the loss of frame budget, particularly if it's going to be extremely hard to get right.

To address one aspect of your question:

What about animated movies rendering for hours and hours makes them so beautiful while in-game live rendering is less beautiful (from a general point of view)?

Artisanry, chiefly. The realtime artists aren't afforded the chance to fine-tune each frame or create one-off gimmicks to improve the overall look of a scene, or reaction, not only because of time constraints (both in development and in rendering) but also because of practical constraints. As noted in the interactivity comment, the player is probably not going to do exactly the same thing every time, but the movie will play the same each time.

This means that the video game artist has a very different set of priorities where it concerns model quality and final render quality than the film artist. Both require enormous amounts of skill, but each demand different techniques. I expect these techniques to increasingly converge as consumer level rendering hardware continues to progress and further mathematical genius continues to emerge at your SIGGRAPHs and your GDCs.

Answer 9

A single animated movie could take many years to render on a single beast of a machine. Since it is pre-rendered it does not matter how much costly effects like light bounces, shadows, etc we add to the scene. These movies usually get rendered by render farms where thousands of PC's are linked together working on the same job.

The reason we cannot achieve the same quality for real time is simply because the machine needs to render the current frame in about 16ms to achieve a FPS of 60. A single frame for a animated movie could take hours to render on a single PC.

In a game, besides the graphics, a ton of other stuff is going on. Each frame there are tons of calculations. Simply said, an animated movie does not need to calculate the damage taken by the player from a rocket launcher, you just animate the gibs and the polygons will render.

Luckily there are some genius people walking around on this planet. Apart from better PC's that can render more polygons and larger textures there are some ingenious inventions made like normal maps that can animate light/shadow on a flat polygon very cheaply, making a boring 3D object look extraordinary detailed.

Some of these expensive effects that make movies have that extra flare.

• Light bounces. In real life light bounces until it is completely absorbed by the surfaces it bounced on. For each light source calculations have to be made. Try to hold a colored object next to a piece of white paper, you will see the color bleed on to the paper.
• Games have harsh limitations on the amount of polygons used, prerender movies do not have these limitations, it just takes longer to render. Back to light bouncing, if we have more polygons then there are more light calculations for even better visuals but at the expense of exponential growing cost.
• Texture size, graphic cards can only hold a certain amount of data in them and switching this data is expensive. A common 1024x1024 texture including all the shader maps could easily take up a couple percentages of you 1GB GFX card. When the card needs to switch data this goes at a cost. Obviously we don't care so much for this when prerendering a movie. Also, in a movie we just need high resolution textures where the camera gets near, for a game we want this for each object the player can get close to.
• Pathfinding is very demanding on the CPU. Pathfinding algorithms, for example, make enemies find routes towards you or make your player walk a path when you click somewhere. These pathfinding can be costly, when there are 10000 way-points (which is just each square meter on a 100m x 100m map) then it is possible we need to traverse all these way-points many times on a single frame. There are many ways to make this cheaper depending on needs but the fact is we do not need this for prerendered scenes.

Simpy put, we need to come up with "cheap" tricks to make a game look good. But the human eye is hard to fool.