Graphics Pipeline, how exactly it works and how graphics card interprets it?

Question

I've looked at this Wikipedia's link, http://en.wikipedia.org/wiki/Graphics_pipeline, but the article lacks in-depth explanation.

The only thing, I suppose, I understand and can expand upon, is, Texture part:

• Apply bump map
• Apply specular map
• Apply displacement map
• Apply texture map
• In between apply any other maps(?)

What interests me in the answer, is, what happens with the model on every frame rendered, like:

1. Application launched, requests a 3d model (OpenGL/D3D)
2. OpenGL/D3D calls hardware for help
3. Polygons get loaded into graphics card memory,
4. Something gets done, blah, blah
5. The Texture part comes in (the upper list), loading every texture in memory card memory texture that needs to be applied to the mesh.
6. Maybe something more?
7. Display to monitor or send out to operating systems display interface?

In short: OS -> Application -> OS -> Hardware -> OS -> Display device.

My examples are just what I understand/can make out of all this. But I'm looking for an answer that goes well in depth on graphics pipeline.

Would like to know how AAA tools like CryEngine, UDK does graphics rendering, where we have all the dynamic lightning and stuff like that.

I'd be more than thankful with answer that contains an example of, for instance, what does the character model (CryEngine/UDK) has to go through to get rendered in full fashion, with approximate included memory consumption/hardware instruction called, like:

• Load polygons [OpenGL issues command, hardware instruction, memory used, possibly GPU usage?]
• Next function [OGL, HW, Memory, GPU]

And, what are the best practices for graphics pipeline?

P.S. Please correct me if that's not Graphics Pipeline I'm referring to.

Thanks in advance!

Answer 1

In my opinion you're asking for information on a topic that is so broad and extensive - a topic for which entire books (or a great part of many books) have been written - that it might be extremely difficult to try and summarize everything in a concise fashion. This is just much more likely the subject of an entire article, chapter, or book, than something that could be easily written out as an answer.

I can also tell that your understanding of the topic is very hazy, even on the parts you claim to understand. For instance, most maps that you mention are not just textures that are applied by the graphics cards! In fact they're usually nothing more than data (of any kind) encoded in image format in order to be used in computations further down the pipeline (e.g. in the pixel shader).

This data may range from storing the surface normals at each pixel of the scene, to the coefficient of specularity, motion vectors, pretty much anything.

So my recommendation would be to take the long trip, and turn to the bibliography. In particular I'd recommend Real-Time Rendering 3rd Edition as the flagship of all graphic programming books in order to provide a real in depth coverage of this and many other related topics.

Well, but I guess could try to give a basic overview of the pipeline. But think of it as watching the earth from space. You can get a grasp of the shape of the continents, how there's oceans, etc. But if you need a more detailed view, you'll need to come down and do a lot of exploring on your own.

---

Summary of the Graphics Pipeline

The graphics pipeline is a name given to the entire process that relies on hardware and software to transform the representation of some 3D scene into a 2D image that you can see on the screen.

It can be roughly divided into three big stages:

• The Application Stage
• The Geometry Stage
• The Rasterizer Stage

Taking the following as input/output:

• Input - The Application Stage takes as input a 3D scene, which is in rough terms just a collection of data. This data takes the forms of geometry (which is usually made up of triangles), light sources, textures, camera, etc. Geometry is usually organized in a very specific way, for instance as a list of triangles.

• Output - The output of the Rasterizer Stage is the image you see on screen.

Stage Description

1) The Application Stage is everything that happens in your side of the application, before and up to actually submitting the scene to the graphics hardware. This might include things such as collision detection as animation for instance.

2) The Geometry Stage has many, many subphases, but they're mostly all concerned with the manipulation and transformation of the data you submitted. It's responsible for taking your initial scene representation and apply a series of calculations that will end up describing the final 2D image. For instance it needs to:

• Apply world transformations in order to place your objects properly in the scene
• Apply view transformation in order to render from the point of view of your camera
• Calculate how much light is reaches each point in your scene which is a very intricate process relying on a lot of different variables
• Transform your 3D coordinates onto the 2D screen plane usually by applying a perspective projection
• Trim out (or clip) everything that ended up being outside of your screen
• Adapt the 2D representation of the scene to your screen's actual resolution

In sum, the Geometry Stage takes care of per-vertex operations.

3) Finally, the Rasterizer Stage is a bit like a painter. It takes the (2D) description of your scene that was generated by the Geometry Stage, and starts filling it with color, pixel (or fragment) by pixel. It also takes care of applying any textures, and deals with the sorting of what you see (look up z-buffer).

In sum, the Rasterizer Stage takes care of per-pixel operations.

---

And I guess that's the best I can describe it. I might have ended up being inaccurate on a point or two, but this is my current understanding of the usual 3D graphics pipeline. Feel free to edit in any corrections.