How to reduce/handle image loading times?

Question

I'm working on a 2D game that has very high resolution images (so that things look good on Retina MacBooks and such), and am seeing a noticeable lag caused by image loading.

I notice the strongest lag when I switch to a screen that loads 72 500x600 PNG images - it takes almost a second on a pretty modern system with an SSD. Loading the files from the disk is fairly quick, usually less than one millisecond. But loading the PNGs takes about 5-7 milliseconds each, using libpng.

Now I have two questions:

What are my options for speeding up the loading?

• Will using sprite sheets help? (My guess is no, since loading times seem to increase with image size).

• Would it make sense to add pre-downscaled versions of the files and use these on lower resolutions?

Should I load all assets upfront?

My current approach is a bit naive: Each entity loads its own texture as soon as it is instantiated. I can change things a bit to make sure all entities are instantiated upfront where possible, but that seems error prone.

Are there better approaches? I think we'll end up with around 100-200 MB of images, so it might be fine to load them all into memory when the game starts. The best approach I can come up with is to have a huge mapping from image files to image IDs and use the IDs to load images in the code - then I'd know all the image files upfront. Or should I shy away from loading everything upfront if I can speed the loading up until it's unnoticeable again?

Answer 1

Create an ImageLibrary class responsible for loading images from disk, and supplying them to the front-end on demand.

Have the ImageLibrary manage background task(s) to load images from disk steadily (individually, so they can be retargeted if necessary as the front end demands specific images that have not been loaded yet).

I suspect the bottleneck is not disk I/O, but rather the current need to perform all the IO serially. Parallelizing this task should help.

This framework could also morph into a disk cache over time, by tracking a most-recently-used time for each image, and retiring the oldest as needed to make room for newly demanded images.

Answer 2

In general terms you'll have the following bottlenecks for texture image loading (since you mention MacBooks I'm going to assume that you're using OpenGL):

• Disk I/O
• Decompression
• Load to GL/hardware

Disk I/O can be solved by using faster hard disks or smaller image file sizes (this includes those that compress well) and from your question you appear to have that well in hand.

Decompression can be solved by using image formats that are fast to decompress, or that don't need decompression at all. Here your use of PNG is working against you as PNG is quite slow to decompress. This will be one of your primary bottlenecks. Note here that not needing to decompress at all and smaller file sizes are not mutually exclusive; we'll discuss that later.

Load to GL/hardware overhead at first glance looks like just a bandwidth limit, but that's not all. Since most GPUs like texture image formats to be laid out in BGRA order, if you're loading any other format your GL driver has to do a lot of extra work to convert it to what it likes. In particular, the frequently seen GL_RGB format is a worst-case format, as it may require the driver to both expand to 4-component and to reswizzle the data. This is further discussed here: http://www.opengl.org/wiki/Common_Mistakes#Texture_upload_and_pixel_reads and here: http://www.opengl.org/wiki/Common_Mistakes#Slow_pixel_transfer_performance and is likely to be another one of your bottlenecks.

Using traditional image formats (TGA, BMP, PNG, etc) you're going to solve one or two of these bottlenecks, but not the other(s). I'd recommend that you solve them all before looking at threading/etc and there is one texture image format that is going to do this for you: DDS with DXT/S3TC compression.

Despite the name ("Direct Draw Surface") this format is in reality just a normal binary image format and can be easily loaded by OpenGL too (using the GL_ARB_texture_compression and GL_EXT_texture_compression_s3tc extensions or GL 1.3 or higher). It provides the following advantages for you:

• File size is one-eighth (for images without alpha) or one-quarter (for images with alpha) the uncompressed size so it can come off disk reasonably fast.
• No decompression step is needed at all; just supply the pointer to your raw loaded data to a glCompressedTexImage call.
• The texture data is already in a GPU-native format so no intermediate driver-side work is needed during the texture upload.
• You'll also get better performance during rendering (in the order of 20% to 30% faster) owing to lower bandwidth requirements there too.

Of course there's no such thing as a free lunch, so it's necessary to warn you that this is a lossy compression format, and so you may see some compression artefacts. On the other hand, if it's good enough then it's good enough, even if lossy. It's also necessary to warn you that S3TC texture compression is burdened by IP and so may not be available in all drivers, but in practice it's ubiquitous outside of open source implementations.