I'm working on a game right now and running into some noticeable performance issues and limitations.
Basically, I have this:
- A few
QGLWidgets, all sharing contexts with eachother (probably irrelevant that I'm using Qt [also: Windows, fwiw]).
- About 216000 total frames of 128x128 videos (with alpha, internal format GL_RGBA, data format GL_BGRA/GL_UNSIGNED_INT_8_8_8_8_REV, video codec is Quicktime RLE, decoded with libav), spread into approximately 120 videos of 1800 frame average length. This translates to roughly 216000*128*128*4 = 14.2 GB of raw data, not including overhead.
The game behaves as follows:
- There are many "modes". At any given time only one mode is active. Each mode uses a very small known subset of the available videos, say 5-10 videos (= about 9000 - 18000 frames) tops.
- There's a bunch of 2D quads flying around the screen, each displaying whatever videos depending on the current mode. Objects don't all display the same frame, the video time offsets are random. The videos loop.
- Every few minutes, the mode changes. Objects cross-fade to their new videos, so there is some overlap regarding when various sets of videos are needed.
- It is critical that this thing keep running smoothly, at 60 FPS, with no pauses for loading, etc.
My problem is this:
My initial approach was to simply preload all videos off the disk and into textures, in video memory (via
glTexImage2D and friends). I didn't mind the slow startup. This worked great early on, when I didn't have a lot of videos. The game could run smoothly because all the textures were already loaded.
Now I have too many videos. I'm becoming extremely limited by available video memory, and the preload-everything strategy is no longer working. I haven't become limited by system memory yet but that's also inevitable. I suspected this was going to be an issue and kept my fingers crossed, but alas. I hit the limit way sooner than I expected.
I tried switching to a strategy where, every time a mode transition started I pre-loaded the videos for that mode (and every time a transition ended I unloaded the videos for the previous mode). This works as far as memory goes but it pauses everything for a very long time while loading (as in 3-8 seconds depending on the number of videos and the graphics hardware).
I've done the best searching that I know how to do, all of the texture memory management posts, blogs, tutorials etc. just talk about the basic technique of creating and destroying textures, rather than strategies for managing large numbers of them in an application. I went through every question in the auto-search on here that had a relevant-looking title. I did find out that OpenGL is definitely not thread-safe, which kind of kills my asynchronous background loading idea, unless there's some other way to implement that.
So my question is: What's a good strategy to use for swapping out tons of textures (200000+, 128x128 typical size) when at any given time only a small portion of those are needed, while maintaining a smooth frame rate? How is this typically done?