How do we solve big video memory requirements in a 2D game?


We are developing a 2D game (Factorio) in allegro C/C++, and we are facing a problem with increasing video memory requirements as the game content increases.

We currently gather all the info about images that are going to be used first, crop all of these images as much as possible and organize them into big atlases as tightly as possible. These atlases are stored in video memory, the size of which depends on the system limitations; currently it is usually 2 images of up to 8192x8192, so they require 256Mb to 512Mb video memory.

This system works pretty good for us, as with some custom optimisations and splitting the render and update thread we are able to draw tens of thousands of images on the screen in 60 fps; we have many objects on the screen, and allowing a big zoom-out is a critical requirement. As we would like to add more, there is going to be some trouble with the video memory requirements, so this system can't possibly hold.

One of the things we wanted to try is to have one atlas with the most common images, and the second as a cache. The images would be moved there from the memory bitmap, on demand. There are two problems with this approach:

  1. The drawing from memory bitmap to video bitmap is painfully slow, in allegro.
  2. It is not possible to work with video bitmap in other than the main thread, in allegro, so it is practically unusable.

Here are some additional requirements we have:

  • The game must be determistic, so the performance issues/loading times can never alter the game state.
  • The game is real time, and soon to be multiplayer, as well. We need to avoid even the smallest stutter at all costs.
  • Most of the game is one continuous open world.

The test consisted of drawing 10 000 sprites in a batch for sizes from 1x1 to 300x300, several times for every configuration. I did the tests on the Nvidia Geforce GTX 760.

  • Video bitmap to video bitmap drawing took 0.1us per sprite, when the source bitmap wasn't changing between individual bitmaps (the atlas variant); the size didn't matter
  • Video bitmap to video bitmap drawing, while the source bitmap was switched between drawings (non atlas variant), took 0.56us per sprite; the size didn't matter, either.
  • Memory bitmap to video bitmap drawing was really suspicious. Sizes from 1x1 to 200x200 took 0.3us per bitmap, so not so horribly slow. For larger sizes, the time started to raise very dramatically, at 9us for 201x201 to 3116us for 291x291.

Using atlas increases the performance by a factor greater than 5. If I had 10ms for the rendering, with an atlas I'm limited to 100 000 sprites per frame, and without it, a limit of 20 000 sprites. This would be problematic.

I was also trying to find a way to test the bitmap compression and 1bpp bitmap format for shadows, but I was unable to find a way to do this in allegro.

  • 1
    Big fan of your game, I backed the Indiegogo campaign. I binge on it every few months. Nice work so far! I removed the "which technology to use" questions that are off topic for the site. The remaining questions are still pretty broad, if you have anything more specific you should try to narrow the scope. – MichaelHouse Apr 12 '14 at 20:49
  • Thanks for the support. So where is the place to ask which technology to use then? I'm not looking for answer with specific engine recommendation, but I wasn't able to find in-depth comparison of 2d engines and inspecting them manually one by one including performance and usability tests would take ages. – Marwin Apr 13 '14 at 0:21
  • Check out the bottom of this page for some places to ask questions like "which tech to use". You have a totally valid and reasonable question, it's just not the type of question we deal with at this site. Even though you're not looking for a specific engine, that's really the only way to answer the question of "Is there any technology that does X?". Someone could just answer "yes" and not give a recommendation for a specific one, but that wouldn't be very helpful. Good luck with it! – MichaelHouse Apr 13 '14 at 0:28
  • 2
    Are you compressing your textures? – GuyRT Apr 14 '14 at 13:10
  • 3
    @Marwin, Compressed textures may perform much better than uncompressed textures because they reduce the memory bandwidth needed (this is especially true on mobile platforms where bandwidth is much lower). You could save a huge amount of memory just by compressing your textures. Really, the only downside is the artifacts which are inevitably introduced. – GuyRT Apr 24 '14 at 13:54
up vote 17 down vote accepted
+100

We have a similar case with our RTS (KaM Remake). All units and houses are sprites. We have 18 000 sprites for units and houses and terrain, plus another ~6 000 for team colors (applied as masks). Long-stretched we also have some ~30 000 characters used in fonts.

So there are some optimizations against RGBA32 atlases you are using:

  • Split your sprites pool into many smaller atlases first and use them on demand as covered in other answers. That also allows to use different optimizations techniques for each atlas individually. I suspect you will have a bit less of a wasted RAM, cos when packing to such a huge textures there are usually unused areas at the bottom;

  • Give a try to using paletted textures. If you use shaders you can "apply" the palette in the shaders code;

  • You might look into adding an option to use RGB5_A1 instead of RGBA8 (if for example checkerboard shadows are okay for your game). Avoid 8bit Alpha when possible and use RGB5_A1 or equivalent formats with smaller precision (alike RGBA4), they take half the space;

  • Make sure you are tightly packing sprites into atlases (see Bin Packing algorithms), rotate sprites when necessary and see if you can overlap transparent corners for rhombus sprites;

  • You might try hardware compression formats (DXT, S3TC, etc.) - they can dramatically reduce RAM usage, but check for compression artifacts - on some images the difference can be unnoticeable (you can use this selectively as described in first bullet point), but on some - very pronounce. Different compression formats cause different artifacts, so you might pick one that is best for your art style.

  • Look into splitting big sprites (of course not manually, but within your texture atlas packer) into static background sprite and smaller sprites for animated parts.

  • 2
    +1 for using DXT, it is a very good thing to have. Great compression and used directly by the GPU so the overhead is minimal. – Thebluefish Apr 25 '14 at 5:47
  • 1
    I agree with dxt. You could also possibly query for DXT7 support (DX11+ hardware), which is the same size as DXT1 but (apparently) higher quality. However you would either need to have double the textures (one DXT7 and one DXT1), or compress/decompress during loading. – Programmdude May 2 '14 at 14:03

First of all you need to use more, smaller texture atlases. The less textures you have the more difficult and rigid memory management will be. I would suggest a atlas size of 1024, in which case you would have 128 textures instead of 2, or 2048 in which case you would have 32 textures, which you could load and unload as needed.

Most games do this resource management by having level boundaries, while a loading screen is displayed all resources that aren't needed anymore in the next level get unloaded and resources that are needed get loaded.

Another option is On-Demand loading, which become necessary if level boundaries are unwanted or even a single level is too big to fit into memory. In this case the game will be trying to predict what the player will see in the future and load that in the background. (For example: stuff that is currently 2 screens away from the player.) At the same time things that weren't used anymore for a longer time will get unloaded.

There is one issue however, what happens when something unexpected happened that the game wasn't able to foresee?

  • Panicking and displaying a loading screen until all necessary stuff is loaded. This might feel disruptive to the experience.
  • Have low resolution sprites for everything preloaded, continuing the game and replacing them as soon as the high resolution sprites finished loading. This might look cheap to the player.
  • Make it impact the gameplay and delay the event as long as necessary. E.g. don't spawn that enemy until it's graphics are loaded. Don't open that treasure chest before all graphics for that loot are loaded, etc.
  • I added some of the requirements that I omitted. The loading screen, or any kind of loading is not possible. Everything must be done in the background, or inbetween individual ticks (less than 15ms for each) while there is most of the time usually already used by render preparations and game update. Anyway, splitting to smaller parts might add some flexibility in the switching, it would be faster for sure. The question is how much does it hurt performance when rendering, as switching the source bitmap while drawing slows the render down. I would have to make exact measurement to say how much. – Marwin Apr 24 '14 at 13:00
  • @Marwin Performance impact, yes, but since you are dealing with 2D you it should be still be far away from it becoming a issue. If the rendering currently takes 1ms per frame, and through using smaller textures it suddenly takes 2ms then that still is more than fast enough to reach consistent 60 FPS. (16ms) – API-Beast Apr 24 '14 at 13:16
  • @Marwin Multiplayer is tricky buisness, always was, always will be. You will very likely have to make compromises there. You will have stutters, simply because you have to transfer data over the internet, packages will be lost, pings may suddenly spike, etc. Stutter is unavoidable so you what will be more important is making the network model itself resistant to stutter. Knowing when to wait & how to wait for other players. – API-Beast Apr 24 '14 at 13:25
  • Hello, stutter is almost avoidable in multiplayer, we are working on that area right now, and I believe we have a good plan. I could even post and answer my own question describing what we researched in detail later :) It might be surprise, but the render time actually is an issue. We have done lots of optimisations to make rendering faster. The main render is now made in separate thread and other small tweaks. Don't forget, that on max zoom, the player can easily see tens of thousands of sprites at the same time. And we would even like to allow even higher zoom-levels later. – Marwin Apr 24 '14 at 14:34
  • @Marwin Hm, 10k objects should usually be no issue for a PC or modern laptop if you use proper batching, have you profiled your rendering code? – API-Beast Apr 24 '14 at 14:36

Wow, that is a hefty amount of animation sprites, generated from 3D models I presume?

You really shouldn't be making this game in raw 2D. When you have fixed perspective a funny thing happens, you can seamlessly mix pre-rendered sprites and backgrounds with live-rendered 3D models, that has been used heavily by some games. If you want such fine animations that seems like the most natural way to do it. Get A 3D engine, configure it to use isometric perspective, and render the objects for which you continue to use sprites as simple flat surfaces with an image on them. And you can use texture compression with a 3D engine, that alone is a big step forward.

I don't think loading and unloading will do a lot for you as you can have pretty much everything on screen at the same time.

Firstly find the most efficient texture format that you can while still being happy with the visuals of the game whether this is RGBA4444, or DXT compression etc. If you are not happy with the artifacts generated into a DXT alpha compressed image, would it be viable to make the images non-transparent using DXT1 compression for the colour combined with a 4 or 8 bit grayscale masking texture for the alpha? I imagine you would stay on RGBA8888 for the GUI.

I advocate breaking things up into smaller textures using whichever format you decided on. Determine the items that are always on screen and therefore always loaded, this might be the terrain and GUI atlases. I would then break up the remaining items that are commonly rendered together as much as possible. I don't imagine you would lose too much performance even going up to 50-100 draw calls on PC but correct me if I'm wrong.

The next step will be to generate the mipmap versions of these textures as someone pointed out above. I would not store them in a single file but separately. So you would end up with 1024x1024, 512x512, 256x256 etc versions of each file and I would do this until I reach the lowest level of detail I would ever want to be displayed.

Now that you have the separate textures you can build a level of detail (LOD) system that loads textures for the current zoom level, and unloads textures if not used. A texture is not being used if the item being rendered is not on screen or is not required by the current zoom level. Try to load the textures into video RAM in a thread separate to the update/render threads. You can display the lowest LOD texture until the required one is loaded. This might sometimes result in a visible switch between a low detail/high detail texture but I imagine this would only be when you perform extremely fast zooming out and in while moving across the map. You could make the system intelligent by attempting to preload where you think the person will move or zoom to and load as much as possible within current memory constraints. You can cache where the user has been until they move away.

That is the sort of thing I would test to see if it helps. I imagine to get extreme zoom levels you will inevitably need an LOD system.

I believe that the best approach is to split the texture in many files and loading them on demand. Probably your problem is that you're trying to load larger textures that you would need for a complete 3D scene and you are using Allegro for that.

For the big zoom-out you want to be able to apply, you have to use mipmaps. Mipmaps are lower-resolution versions of your textures which are used when the objects are far enough from the camera. This means that you could save your 8192x8192 as a 4096x4096 and then another of 2048x2048 and so on, and you switch to the lower resolutions the smaller you see the sprite on the screen. You can both save them as separate textures or resize them when loading (but generating mipmaps during runtime will increase loading times for your game).

A proper management system would load the required files on demand and release the resources when nobody is using them, plus other things. Resource management is an important topic in game development and you're reducing your management to a simple coordinate mapping to a single texture, which is close to having no management at all.

  • 1
    By splitting to files, you mean files on HDD? I assume, that I could store all the pictures on RAM for starters, and even copying from memory-bitmap to video-bitmap is too slow currently, so loading from HDD would be certainly even slower. Having mimpaps will not help me, as I still will have the biggest resolution in vram. – Marwin Apr 13 '14 at 0:13
  • Yeah, you don't have to load everything, you have to load only what you use. Whenever you want to change a pixel on a texture loaded in VRAM, the system has to move the ENTIRE TEXTURE to RAM, just for you to modify a single pixel, the move it back to VRAM. If you have everything in a single texture, this involves moving 256 MB to RAM then back to VRAM again, which locks the whole computer. Having it separated in different files and textures is the correct way of doing it. – Pablo Ariel Apr 13 '14 at 0:39
  • The modifying of texture that triggers copy to memory and back to ram applies only for persistent bitmaps, the cache would probably not be set to persistent, the only downside would be the need to refresh it when display is lost/found. But in the allegro, even simple copy of 640X480 picture from vram to memory bitmap (save game preview) takes quite a long time. – Marwin Apr 13 '14 at 0:43
  • 1
    I need to have everything in one big texture to optimise the drawing itself, without it, the effect of switching context inbetween individual sprites slows the render way too much at least in allegro. Don't get me wrong, but you are kind of captain obvious here, as you are vaguely suggesting me to do something I ask for in this question. – Marwin Apr 13 '14 at 13:51
  • 1
    Having these mip-mapped textures in different files would force me to reload all the atlas when player zooms in. As the engine has just few units of ms for it at most, I don't see a way how to do it. – Marwin Apr 14 '14 at 5:10

I recommend creating more atlas files that can be compressed with zlib and streamed out of the compression for each atlas, and by having more atlas files and smaller size files you can restrain the amount of active image data in the video memory. Also, implement the triple buffer mechanism so that you are prepping each draw frame sooner and has a chance to complete faster so that the stutters do not appear on the screen.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.