CPU - GPU memory data flow [closed]

Question

I'm a newbie graphics programmer and I've been wondering recently - how does model data (meshes and materials) flow from application (CPU memory) to graphics card (GPU memory?)? Say I have a static model (e.g. a building) that I load and setup once and don't change throughout the app lifespan.

• Does its data get sent to GPU memory only once and sit there forever?
• When the model gets actually rendered each frame do GPU processors have to fetch its data each time from GPU memory? What I mean is - if I had 2 models rendered multiple times each - would it matter if I first rendered the first one multiple times and then the second one multiple times or if I rendered the first one just once, the second one just once and kept interleaving it like that? I could call this question "internal GPU data flow" in this sense.
• Obviously graphics cards have limited RAM - when it can't hold all the model data necessary for rendering 1 frame I guess it keeps fetching (some of) it from CPU RAM each frame, is that correct?

I know there's a lot of books and stuff about this on the internet but maybe you have some quick general guidelines as to how to manage this data flow (when to send what and how much, when and how to render)?

Edit: I forgot to make one distinction: there's sending the data to the GPU and there's setting/binding the buffers as current. Does the latter cause any data flow?

Edit2: After reading Raxvan's post I'd like to distinct a few actions:

• buffer creation with initialisation (as he said I can store the data in either CPU ram or GPU one)
• buffer data update (which I believe is straightforward when the data is kept in CPU ram and requires fetching from GPU to CPU ram (and then back) when it's kept in GPU ram)
• binding the buffer as active (is it just a way to tell the API that I want this buffer to be rendered in the next draw call and it doesn't do anything by itself?)
• API draw call (here I'd like to hear from you what actually happens there)

Answer 1

Does its data get sent to GPU memory only once and sit there forever?

Usually yes, but the driver is free to do what is "optimal", the data might be stored at VRAM or RAM or could just be cached here is an atricle that explains what actually happens with the VBO flow.

For example if it was flagged as a dynamic openGL buffer (e.g. VBO), it's more likely to be stored in RAM. GPU use direct memory access (DMA) to access the ram directly without the CPU intervention, this is controlled by the DMA controller in the graphics card and graphics driver and is executed in kernel mode.

When the model gets actually rendered each frame do GPU processors have to fetch its data each time from GPU memory, even if a model is rendering multiple sequential times ?

Just like CPUs, GPUs are permitted to re-order GPU instructions and memory access operations, (read: out of order execution) so most likely the GPU will handle the scenario you mentioned by accessing the memory that is in it's cache (usually accessed recently), but sometimes it can't do this.

Obviously graphics cards have limited RAM - when it can't hold all the model data necessary for rendering 1 frame I guess it keeps fetching (some of) it from CPU RAM each frame, is that correct?

You don't want this to happen. But regardless if that happens the GPU will start moving memory between RAM and VRAM (the command processor on the GPU is responsible for this) which will make rendering much slower which will case the GPU to stall, because it will need to wait for the data to be copied from/to V/RAM.

There's sending the data to the GPU and there's setting/binding the buffers as current. Does the latter cause any data flow?

GPUs contain a command buffer, and all API commands are submited to this buffer, notice that this can happen simultaneously with the data being copied to the GPU. The command ring buffer is a communication queue between the CPU and GPU, any command that needs to be executed needs to be submitted to the queue so it can be execulated by the GPU. Just like any operation binding new buffers needs to be submitted to the gpu so it can access some memory location.

That's one of the reasons glBegin/glEnd was deprecated, submitting new commands needs queue synchronization (using memory fences/barriers).

As for your other points:

Buffer creation with initialization

You can allocate a buffer without initialization and keep it for later use. Or you can allocate it a buffer and copy data at the same time (talking about the API level).

buffer data update

You can use glMapBuffer to update memory on the GPU side. whether the memory will be copied from/to RAM is not actually poart of the standard and will vary greatly depending on the Vendor, GPU type and driver.

API draw call (here I'd like to hear from you what actually happens there).

My second point in the main question covers this.

binding the buffer as active (is it just a way to tell the API that I want this buffer to >be rendered in the next draw call and it doesn't do anything by itself?)

Think of binding as using this pointer in any object oriented language, although not strictly the same, any consequent API calls will be relative to that bind buffer.

Answer 2

In general the boundary and the involvement of the cpu versus gpu are specific on the platform, but most of them follow this model: cpu has some ram, gpu also and you can move memory around (in some cases the ram is shared but for the sake of simplicity let's stick to separate rams).

first point: The data that you initialize you can choose to keep it in the CPU ram or on the GPU ram, and the are advantages for both. When you render something the GPU has to do the heavy lifting so it is obvious that the data that is already on the GPU mem will provide better performance. for the CPU it must first send the data to the GPU (which can choose to keep it for a while) and then perform the rendering.

second point: There are many trick in rendering but the main way thing are done is with polygons. On a frame gpu will render the objects made out of polygons one by one and after finishing that the GPU will send the picture to the display. There is no concept such as objects , there are just polygons and the way you put them together will create an image. the job of the GPU is to project those polygons from 3d to 2d and apply effect(if wanted). The polygons go only wan way CPU->GPU->SCREEN or GPU->SCREEN directly (if the polygons are already in the GPU ram)

third point: When you render animations for example it is better to keep the data close to the cpu because there he does the heavy lifting , it would not be optimal to keep the data in GPU , move it to CPU and back every frame. There are lots of other examples like this to count but in general all the data will stay close to whoever is doing the computations. Usually you would want to move as much data as possible to the GPU ram to gain performance.

The actual sending the data to the gpu is done by the API you use (directx/opengl or other) and the concept of binding and stuff like this are just abstractions so that the API understands what you want to do.

Edit for the edit:

• buffer creation with initialisation: it's like the difference between int a = new int[10] and a[0] = 0,a[1] = 1.... etc when you create a buffer you make room for the data and when you init the data you put the stuff you want in there.

• buffer data update if it's on the cpu ram then you have vertex * vertices that you can play with it, if it's not there , you would have to move it from GPU vertex * vertices = map(buffer_id);(map is a mythological function that should move data from GPU to CPU ram, it also has it's oposite buffer_id = create_buffer(vertices);

• binding the buffer as active it's just a concept that they call binding rendering is a complex process and it's like calling a function with 10000 parameters. Binding is just a term they used for telling which buffer goes where. There is no real magic behind this term, it does not convert or move or reallocate buffers, just tells the driver that on the next draw call use this buffer.

• API draw call After all the binding and setting buffers up this is the place where rubber meets the road. The draw call will take all the data (or the id's that point to the data) you specified, sent it to the GPU (if needed) and tell the GPU to start crunching the numbers. This is not entirely true on all the platforms there are many differences, but for keeping thing simple the draw will tell the GPU to .... draw.

Answer 3

The most correct answer is, it depends on how you program it, but this is a good thing to worry about. While GPU's have become incredibly fast, bandwidth to and from GPU RAM is not, and will be your most frustrating bottleneck.

Does its data get sent to GPU memory only once and sit there forever?

Hopefully, yes. For rendering speed, you want as much data to sit on the GPU as you can, instead of re-sending it every frame. VBO's serve this exact purpose. There are both static and dynamic VBO's, the former being best for static models, and the latter being best for models whose vertices will change every frame (say, a particle system). Even when it comes to dynamic VBO's, though, you don't want to resend all the vertices every frame; just the ones that are changing.

In the case of your building, the vertex data would just sit there, and the only thing that changes are your matrices (model/world, projection, and view).

In the case of a particle system, I made a dynamic VBO big enough to store the maximum number of particles that will ever exist for that system. Each frame I send the data for the particles emitted that frame, along with a couple uniforms, and that's all. When I draw, I can specify a start and end point in that VBO, so I don't have to delete particle data. I can just say don't draw those.

When the model gets actually rendered each frame do GPU processors have to fetch its data each time from GPU memory? What I mean is - if I had 2 models rendered multiple times each - would it matter if I first rendered the first one multiple times and then the second one multiple times or if I rendered the first one just once, the second one just once and kept interleaving it like that?

The act of sending multiple draw calls instead of just one is a much bigger limit. Check out instanced rendering; it might help you out a lot and make the answer to this question useless. I had some driver issues with it that I haven't worked out yet, but if you can get it working, then problem solved.

Obviously graphics cards have limited RAM - when it can't hold all the model data necessary for rendering 1 frame I guess it keeps fetching (some of) it from CPU RAM each frame, is that correct?

You don't want to run out of GPU RAM. If you do, then change things so you don't. In the very hypothetical scenario that you do run out, it'll probably crash somehow, but I've never seen it happen so I honestly don't know.

I forgot to make one distinction: there's sending the data to the GPU and there's setting/binding the buffers as current. Does the latter cause any data flow?

Not any significant data flow, no. There is some cost to it, but that's true for every line of code you write. Finding out just how much it's costing you is, again, what profiling is for.

buffer creation with initialisation

Raxvan's answer sounds good, but it's not quite accurate. In OpenGL, creating the buffer does not reserve any space. If you want to reserve space without passing any data, you can call glBufferData and just pass null. (See the notes section here.)

buffer data update

I'm guessing you mean glBufferData, or other functions like that, right? This is where the real data transfer occurs. (Unless you pass null, like I just said in the last paragraph.)

binding the buffer as active (is it just a way to tell the API that I want this buffer to be rendered in the next draw call and it doesn't do anything by itself?)

Yes, but it can do a little more than that. For example, if you bind a VAO (vertex array object), then bind a VBO, that VBO becomes bound to the VAO. Later, if you bind that VAO again, and call glDrawArrays, it will know what VBO to draw.

Note that while many tutorials will have you create a VAO for every VBO, I've been told this is not their intended use. Supposedly you should create one VAO and use that with every VBO that has the same attributes. I haven't tried this yet, so I can't say for sure whether it's any better or worse.

API draw call

What happens here is pretty straightforward (from our perspective). Say you bind a VAO, then call glDrawArrays. You specify a starting point, and a count, and it runs your vertex shader for every vertex in that range, which in turn passes its outputs down the line. That whole process is another essay of its own, though.