Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

Let's think platform-agnostic: I want to load some graphical resources while the rest of the game is running.

In principle, I can load the actual files on a separate thread, or using async I/O. But with graphical objects, I will have to upload them to the GPU, and that can (usually) only be done on the main thread.

I can change my game loop to look something like this:

while true do
    update()
    for each pending resource do
        load resource to gpu
    end
    draw()
end

while having a separate thread load resources from disk to RAM.

However, if there are many big resources to load, this could cause me to miss a frame deadline and eventually get dropped frames. So I can change the loop to this:

while true do
    update()
    if there are pending resources then
        load one resource to gpu
        remove that resource from the pending list
    end
    draw()
end

Effectively loading only one resource per frame. However, if there are many small resources to load, loading all of them will take many frames, and there will be a lot of wasted time.

Optimally, I would like to time my loading in the following manner:

while true do
    time_start = get_time()
    update()
    while there are pending resources then
        current_time = get_time()
        if (current_time - time_start) + time_to_load(resource) >= 1/60 then
            break
        load one resource to gpu
        remove that resource from the pending list
    end
    draw()
end

This way, I would only load a resource if I can do it within the time I have for that frame. Unfortunately, this requires a way to estimate the amount of time it takes to load a given resource, and as far as I know, there are usually no ways to do this.

What am I missing here? How do many games get to load all their stuff completely asynchronous and without dropped frames or extremely long loading times?

share|improve this question
add comment

1 Answer 1

up vote 6 down vote accepted

Let's start by assuming a perfect world. There are two steps to loading a resource: first you get it off your storage media and into memory in the correct format, and second you transfer it across the memory bus to video memory. Neither of those two steps actually need to use time on your main thread—it only needs to get involved to issue an I/O command. Both your CPU and GPU can go on doing other things while the resource is being copied. The only real resource being consumed is memory bandwidth.

If you're using a platform without much of an abstraction layer between you and the hardware, the API probably exposes these concepts directly. But if you're on a PC there's probably a driver sitting between you and the GPU, and it wants to do things its way. Depending on the API you may be able to create a texture that is backed by memory that you own, but more likely calling the "create texture" API will copy the texture into some memory that the driver owns. In that case creating a texture will have some fixed overhead and some time proportional to the size of the texture. After that the driver might do anything—it might proactively transfer the texture to VRAM, or it might not bother uploading the texture until you attempt to render using it for the first time.

You may or may not be able to do something about that, but you can make a guesstimate of the amount of time it takes to make the "create texture" call. Of course, all the numbers are going to change depending on the hardware and software, so it's probably not worth spending a bunch of time reverse-engineering them. So just try it and see! Pick a metric: either "number of textures per frame" or "total size of textures per frame", pick a quota (say, 4 textures per frame), and start stress-testing it.

In pathological cases, you might even need to keep track of both quotas at the same time (e.g. limit to 4 textures per frame or 2 MB of textures per frame, whichever is lower). But the real trick to most texture streaming is figuring out which textures you want to fit into your limited memory, not how much time it takes to copy them around.

Also, pathological cases for texture creation—like lots of tiny textures being needed all at once—tend to be pathological cases for other areas as well. It's worth getting a simple working implementation before worrying about exactly how many microseconds a texture takes to copy. (Plus, the real performance hit may not be incurred as CPU time on the "create texture" call, but instead as GPU time on the first frame you use the texture.)

share|improve this answer
    
That's a pretty good explanation. Lots of stuff I didn't knew but make a lot of sense. Instead of stress-testing it, I would measure the texture creation overhead in runtime, start gently and throttle up to say, 80% of the available execution time to leave room for outliers. –  Panda Pajama Apr 3 '13 at 5:53
    
@PandaPajama I'm a bit skeptical of that. I'd expect the steady state to be "no textures being copied," and a huge amount of variance. And like I said, I suspect that part of the hit is the first render frame that uses the texture, which is much harder to measure dynamically without affecting performance. –  John Calsbeek Apr 3 '13 at 5:57
    
Also, here's an NVIDIA presentation on asynchronous texture transfers. The key thing that it is driving home, as far as I'm reading, is that using a texture too soon after you upload it is going to stall. developer.download.nvidia.com/GTC/PDF/GTC2012/PresentationPDF/… –  John Calsbeek Apr 3 '13 at 6:11
    
I'm no driver dev jockey, but is that common? It doesn't make too much sense to implement drivers in that way, because texture first-usages are very likely to come in spikes (like at the beginning of each level) instead of spaced along the timeline. –  Panda Pajama Apr 3 '13 at 6:13
    
@PandaPajama It's also common for applications to create more textures than there is available VRAM, and to create textures and then never use them. A common case is "create a bunch of textures and then immediately draw a scene that uses them," in which case being lazy helps the driver, because it can figure out which textures are actually used, and that first frame is going to hitch anyway. But I am also no driver dev, take it with a grain of salt (and test it!). –  John Calsbeek Apr 3 '13 at 6:21
show 3 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.