video card architecture

Question

Can anyone point me to some information on what a modern video card architecture looks like? I'm particularly interested in understanding how my rendering engine will be executed by the GPU.

Ideally, I'd get two answers: one related to today's monster video cards and one related to today's mobile GPU's.

-- edit follows

Thanks for the input so far. Considering the proprietary and changing nature of video card hardware, it seems there's not the ideal answer that I was looking for. So I'll clarify and ask a couple very specific questions. (I guess I was hoping to get a resource similar to http://www.realityprime.com/articles/scenegraphs-past-present-and-future which would generally provide me with lots of information on hardware that might be relevant to me.)

First off, I'm quite familiar with the OpenGL pipeline. A google-images search for my question title, "video card architecture," reveals diagrams of various video cards which naturally seem to match the graphics pipeline. While this makes sense, and is helpful, it leaves out some details. I'll try to get at a couple specifics.

1) What happens when a texture change (glBindTexture(), or new sampler specified) is issued? Obviously, the command is queued and later processed by the video card, but is the texture loaded immediately into cache, or not until something attempts to sample that texture? I would expect the latter, partly because the texture may not entirely fit into such caches to begin with. How big are these caches, anyway?

2) Older video cards had hardware dedicated specifically for a given type of shader. Newer video cards reuse the same hardware for all types of shaders. Are these "stream processors" typically shared with other computations throughout the pipeline, such as culling?

3) It seems that progress related to video card hardware has largely been lead by innovation in software. Are there any interesting cases where the opposite is true? I still hope that there might be some resource that gives a detailed history of video card design, similar to the scenegraphs article linked above.

4) Mobile GPU was an ambiguous term. I'm hoping that eBusiness assumed I meant mobile as in laptop when he called me ignorant -- I was rather referring to the GPUs embedded in modern cell phones and tablets.

Answer 1

Not all GPU architecture is that hidden, it really depends on the company.

Imagination for example :)

Many mobile devices use the PowerVR SGX, which you can see has a USSE where multiple shader processing units are added as necessary. These take care of your shaders, the culling, etc is generally done elsewhere.

These GPU's often use Tile Based Rendering as it can allow optimizations especially in shader time.

There is also a tendancy to make blending part of the programmable pipeline, where either you reprogram a specific blending chip, or the blending is included in the shaders, see the VR Core.

You can get lots of info about their specific GPU, its definately a good start.

I also found this article about the XBox360 chip layout.

And for your other questions : 1) I don't think it would be easy to see how the data is cached, I would guess the mentality is more 'load things into cache and keep what we think will be needed later', generally caches are small though, because it's very fast, expensive memory.

2) As stated, you generally have many different things going on outside of the shaders (see this). An easy example is the Rasterization & Interpolation getting your from Vertex positions, to thousand of fragments with an interpolated color, which is done on the GPU.

3) Shaders came from Pixar :)

4) I didn't think it was ambiguous, what is also interesting are the Intel HD chips that are directly integrated into the Core ix's.

Answer 2

I'm not aware of any hardware-level design documentation, but there's plenty of information publicly available on hardware interfaces and behaviors that might help you out.

AMD has programming information for their GPUs available on their developer portal, which in turn explains how the hardware works in a high level way.

The Direct3D documentation from Microsoft also fairly clearly explains at a high level how the 3D pipeline used by basically all GPUs works.

Finally, you can also look at Mesa3D, which is a complete Open Source implementation of software OpenGL, which basically means an "emulator" for a generic modern GPU. (Plus Mesa includes hardware drivers for many modern GPUs, including most AMD and some NVIDIA GPUs, which you could also study if the public documentation isn't enough for you.)

Answer 3

A lot of architecture information hare hidden behind NDA:s so it might be hard to give a good answer accurate to the current state. You might want to read this nvidia article that describes how the GeForce 6 works.

Of course a lot has changed since, vertex/fragment units have been unified and can be load balanced, but the overall architecture (that you need to know about for writing general apps) is pretty much the same. More shading stages has been added, and a few more fixed function steps (tesselator etc.) These works very much the same way as the steps in the nvidia article, except that they pull computation resources from a pool of units.

As said by seanmiddleditch in his answer, AMD has released a lot of specifications of their chips. You can find them here. Especially the Radeon R6xx/R7xx Acceleration PDF should be of interest to you.

Answer 4

OK, this is the full blown programmers manual (and I think a dated one) for CUDA which gives a lot of details on the nvidia architecture. It seems overwhelming but if you just start going through it I think you will find that the diagrams along with the explanations of them can be very helpful.

Answer 5

1) What happens when a texture change (glBindTexture(), or new sampler specified) is issued? Obviously, the command is queued and later processed by the video card, but is the texture loaded immediately into cache, or not until something attempts to sample that texture? I would expect the latter, partly because the texture may not entirely fit into such caches to begin with. How big are these caches, anyway?

I'm not sure of the size of the caches. However, everything pretty much is load-on-demand for the GPU- after all, what happens if I want to use that cache due to a queued command between now and when the that particular texture change is queued?

2) Older video cards had hardware dedicated specifically for a given type of shader. Newer video cards reuse the same hardware for all types of shaders. Are these "stream processors" typically shared with other computations throughout the pipeline, such as culling?

As far as I know, every operation which is not explicitly shadable is implemented in hardware. This is faster, but also, the shaders in modern games can dominate the performance very strongly- you're mostly shader-limited.

3) It seems that progress related to video card hardware has largely been lead by innovation in software. Are there any interesting cases where the opposite is true? I still hope that there might be some resource that gives a detailed history of video card design, similar to the scenegraphs article linked above.

Pretty much no. This is because every technique can be implemented in software before it can be implemented in hardware- and indeed, software implementation first would be necessary to learn about relative performance, what data it depends on, and things like that, and the output of compiling programs can be used to determine what kind of hardware is needed, gauging developer demand, that kind of thing. That's ignoring all the time it would take to implement in hardware, of course.

One case that I can name, effectively, is concurrency in CPU design, where the necessity of moving to multiple cores in hardware has spawned massive concurrency efforts in software. Of course, software threads existed before, but concurrency was nothing like as it is now.

4) Mobile GPU was an ambiguous term. I'm hoping that eBusiness assumed I meant mobile as in laptop when he called me ignorant -- I was rather referring to the GPUs embedded in modern cell phones and tablets.

They're basically just desktop GPUs, but downscaled. This is because desktop GPUs are still very optimized pieces of hardware performing a dedicated function, giving them much implementation experience and optimization, and developer existing knowledge. If you have a mobile GPU that works completely differently, then nobody is gonna figure out how to use it, it's gonna be much slower (relatively to how it would be), etc. Of course, some systems require changing to be mobile-feasible, but in the general case, it's the best idea to downscale what's available for desktop. This is not quite the same for CPUs, as mobile CPUs have also been around for a long time.

Answer 6

1) What happens when a texture change is issued?

Most chips simply change where the texture sampler requests memory from. I know of no chip that preloads textures in caches, simply because textures tend to be way bigger than the caches, and with multiple textures, it makes even less sense. Moreover, most PC chips have multiple texture samplers, with dedicated caches. And you certainly don't want to load all the caches with the same data.

To give you an idea, with a back-of-the envelope math: preloading would mean transferring all the texture memory of all the active textures each frame (something along the line of 300MB for a mid-range title, 1GB+ for high-end titles). Whereas just loading the parts of the texture that are effectively used brings that down to about ~30MB-100MB / frame (very broad strokes, thinking 1-2 megapixel, with an average of 30B-50B/pixel texture data). The cost of preloading is higher than not by up to an order of magnitude, ignoring the issue of whether it can fit.

The main issue with not preloading is hiding the latency of the memory fetches. GPUs are built to hide this latency (by typically running many threads at the same time on a single processor)

2) Are these "stream processors" typically shared with other computations throughout the pipeline, such as culling?

It depends on the hardware. Some chips are known to do clipping on the processors, e.g. (though culling is easy enough to do in pure h/w). Many mobile chips do the blending et al. alongside the fragment shader, as opposed to dedicate hardware for it. There's even an nvidia extension to do that yourself in the shader as opposed to use the fixed function state: nv_shader_framebuffer_fetch.

3) It seems that progress related to video card hardware has largely been lead by innovation in software. Are there any interesting cases where the opposite is true?

I'm not sure what you're after with this question. Hardware vendors do not add features on a whim. They do so because they see value in them being used by software (even if it's not today's software). Can you be more specific maybe ?

4) I was rather referring to the GPUs embedded in modern cell phones and tablets.

GPUs in cellphones today are mostly Imagination or Tegra. Those 2 architectures already have very different structures. The Imagination ones (SGX parts) are fairly well documented on their websites (and use a tiled-base architecture. See 3nixios good answer on this). Tegra less so. All I could find was that Tegra 2 is not unified, pointing to a hardware base that is ~DX9.