What is a good way to load more than a million images for a collectible card game?

I'm using C# with MonoGame and I'm wondering what would be an efficient way to load more than a million card images for a collectible card game. More cards would be continuously added in future updates and what not. I'm sure there are some genius ways to go about this which are both elegant and efficient. I'm hoping to obtain some insight on how others would solve this problem.

The cards will all be on a client side application the user downloads. The server will make sure the players are not cheating. I'm trying to figure out what are efficient ways to manage a large amount of card images on the client side regarding the loading and unloading of the cards.

I understand the cards should loaded and shown only when necessary. However, there are points in the game where the user can look through their entire collection. I was thinking with a scrollbar or something along those lines. They can filter their cards based on certain properties of the cards. In this scenario, how would I manage the loading/unloading of the cards in real time to make it seamless and efficient.

Thank you for any help!

• While this isn't part of your question, it's worth asking: where are you going to get a million card images? Creating the artwork presumably isn't feasible on your budget (a team of a hundred artists creating 10 cards a day would still take 3 years to build that many images!), there isn't that much creative-commons artwork out there, and if you're looking at combining different base images to generate card images, then you shouldn't be loading a million images, you should be loading the image pieces and combining them client-side. – Steven Stadnicki Jan 23 '13 at 4:25
• If your player spent half a second looking at each of his million card images, it would take nearly six days to look at them all. So if he was looking for a particular one, that's still three days on average. Might put a bit of an awkward pause into that "points where the user can look through their entire collection" portion of the game. – Trevor Powell Jan 23 '13 at 7:47
• Do you really need a million cards? Magic: The Gathering has ~12k cards, and that's a lot. Also, will every player own every card? If not, you can just load their cards (which is likely much less than 1m.) – Luke B. Jan 23 '13 at 13:45
• That's a lot of fi... cards! – Oskar Duveborn Jan 23 '13 at 14:26

A million images are obviously not going to fit in RAM at the same time, so you need some sort of caching and streaming mechanism to constantly load in the images you need.

An easily-implemented but slow solution is to load in the image as soon as it is needed and unload the image as soon as it is unneeded, i.e. when the face of the card becomes visible/invisible. This will involve a lot of file I/O and will be extremely slow.

An alternative and better solution is to have some sort of cache. The basic function of the cache is to retrieve a card image given an ID, be it a string, integer, or something else entirely. Internally, the cache will store a Dictionary<ID, CardImage> to map a loaded image to this ID. When the cache's Retrieve(ID card) function is invoked, it will first check to see if the image has been loaded, i.e. check if the dictionary has the key card. If so, that card image will be returned. If not, the card image will be loaded and then returned.

Of course, the cache doesn't have unlimited memory. You need to have some system in place to discard unneeded card images. The simplest way to do this is to sort the images by when they were last used. The most "stale" card image is then discarded to make room for a new one. You could also implement a more complex system that determines which image to discard based on the number of cards in play, how often the card is used, etc.

A side benefit of having a cache is that you can easily "prefetch" card images before they are needed so that the game doesn't appear to pause and load the image when the card comes into view. For example, if your game has decks, you can prefetch the images of the top few cards. Make sure that you have your images loading on a background thread so that the game doesn't stall if many images need to be loaded at once. Simply play an animation to delay the card being shown if its image is still pending.

The great thing about a cache like this is that it can be used for any kind of resource, like shaders, models, sounds, and even game object prototypes or blueprints. Just make sure you have a robust prefetching system for more real-time and demanding games, especially open-world games that shouldn't have loading screens :)

EDIT: Here's a simple, single-threaded implementation that only implements a time-based discard system. Card images can be prefetched simply by calling CardImageCache.Retrieve("card name") without worrying about the return value. Card images come in the form card.png, where card is the value passed into Retrieve().

public static class CardImageCache
{
public static CardImage Retrieve(string card)
{
if(_cards.ContainsKey(card))
{
// The image is already in memory
return _cards[card];
}
else
{
if(_cards.Length() >= _size)
{
_cards.Remove(card);
}

CardImage image = CardImage.FromFile(card + ".png");
_times.Enqueue(card);
return image;
}
}

private static Dictionary<string, CardImage> _cards = new Dictionary<string, CardImage>();
private static Queue<string> _times = new Queue<string>();

// 100 card images can be stored at a time
private static int _size = 100;
}


There may be some errors, my C# is a little rusty...

EDIT 2: In response to your edit:

Basically, just prefetch the images within a certain radius of the visible region of the collection. If you configure the system such that a CardImage that hasn't finished loading yet (in a background thread) can be "drawn" without any errors, the user won't care that the image is suddenly popping in as they scroll over the card. They're probably used to using the internet.

• These are some great ideas! Thank you for your thoughts! – rotaercz Jan 23 '13 at 3:10
• What would be a good way to determine which cards are currently in the viewable region of the collection? – rotaercz Jan 23 '13 at 3:15
• That totally depends on how you are planning to let the player view their whole collection. If, for example, the collection was shown as a huge grid of cards that the player could pan around and maybe zoom in and out to an extent, you would make sure that the images of cards that intersect or are contained by the the viewing rectangle are loaded. You can also extend this rectangle to prefetch card images before they are needed. – jmegaffin Jan 23 '13 at 3:20
• These ideas are golden. Thank you for all the great ideas! :) – rotaercz Jan 23 '13 at 3:23
• No problem. Feel free to ask me more about this stuff; I've spent a lot of time on figuring out resource management. – jmegaffin Jan 23 '13 at 3:29

Don't load them all into memory at once, that's just a terrible idea. Load them in from the hard drive when they come into play (on another thread), unload them when they're removed from play.

If you've got a deck in-game, pre-load the next few cards to prevent the game from freezing up when you draw a card.

I suggest using some sort of asset_manager (steal it or code it) and flyweight pattern for image itself, since it might be the case there two or more cards use the same image. Also you can load all cards you need during level load or match load, well if you know what card will be used.

// you can also use caching behind asset_manager
image = asset_manager.get_image("evil monster card");


Another solution could be lazy initialization, load image then its actually used, not then game starts. Main draw back with this is speed decrease.

I would consider looking into a procedural generation solution for these. this would include the 'stats',and could possibly include the image.