3
\$\begingroup\$

I am not a game developer, but I have been doing iOS software engineering for many years. I have a particular interest in graphics and animation, but the finer details are still a little foreign to me.

Here is the scenario I am working with:

I have an extremely memory-contrained iOS environment (i.e., no more than 16-20MB of active memory usage) where I am trying to display many high-resolution sprite frames in rapid succession to show an animation. Each frame might be up to 1210x1210 pixels, and I can have up to 200 frames in a single animation.

Obviously, these constraints mean I have to completely optimize my memory usage. There's no way I can even consider having more than one frame of an animation inflated and being used by my application's memory at any given point. Yet I need images to be loaded super fast.

I had considered trying to pack a bunch of frames together into a PVRTC2 or PVRTC4 image file. The quality and disk-space usage of PVRTC files is superior to other formats, considering they're only using 2 or 4bpp.

I attempted to use SpriteKit to read the texture atlas I created and display the image frames. This "worked" in that it did what I wanted, but the memory usage was insane, around 600-700MB. Apparently, SpriteKit is always going to make copies of texture data for its own use, which is unfortunate.

My question is if it's possible to completely bypass copying image data by my application before sending it to the GPU for rendering. My understanding was that the whole point of PVR image formats was to be an exact representation on disk of the uncompressed image data that the GPU can use directly, without the need for copying the image into its own format. (As far as I know, PVR is the format the GPU needs to render the texture.)

Indirectly, I was able to mostly achieve what I wanted using a framework I helped develop for iOS called Fast Image Cache. Basically, it creates image tables on disk that are fully uncompressed and are page-aligned in advance such that Core Animation can use the image data directly without doing a single memcpy. Fast Image Cache uses mapped memory to further avoid any copies whatsoever. In essence, data goes directly from disk to being rendered on display without any copies being made. (At least, no copies that are made within the context of your application, which could be held against your memory usage limits.)

The problem with FIC is that it is a very naive image table. It doesn't do any sort of texture packing. For my testing, I loaded each frame of the animation as a separate image inside FIC. There was a ton of disk IO overhead going on. In addition, FIC only supports 32bpp (with or without alpha) and 16bpp bitmap formats, which means the image tables are huge.

Is there a way for me to achieve something similar to FIC using OpenGL and PVRTC images? If I need to build out my own simple texture atlas support to map regions of PVRTC images to individual animation frames, I'll do that, though it'd be nice if something else besides SpriteKit could do this for me.

The crucial point is being able to quickly load fairly large image data to be rendered without impacting my app's memory footprint.

Improve this question
\$\endgroup\$
5
  • \$\begingroup\$ Your memory footprint doesn't seem surprising to me (600-700MB). Remember that you have two framebuffers allocated all the time, with the size of the screen, for double-buffered rendering. With those crazy-high DPI retina displays, the framebuffers alone will consume more memory that you can spare in your 20MB target. \$\endgroup\$
    – glampert
    Apr 2, 2015 at 4:36
  • \$\begingroup\$ Now one tangent that you might look into is Virtual Texturing. Not sure if it would be suitable to you, but you might investigate. I have tested it on the iOS with promising results. Here is my (unfinished) implementation. \$\endgroup\$
    – glampert
    Apr 2, 2015 at 4:39
  • \$\begingroup\$ Thanks. I'll check that out. I should reiterate that I'm unconcerned if system-level memory consumption varies using whatever techniques let me do it. I just need to a) avoid my app doing any expensive copies and b) only keep one or two frames in memory/bound to GL at once. So streaming them from disk would be ideal so they're swapped in and out of memory only as needed to display a particular frame (or series of sequential frames, if I pack a few into a single texture). \$\endgroup\$
    – user5364
    Apr 2, 2015 at 6:00
  • \$\begingroup\$ [Disclaimer]I don't do IOS programming but I know a reasonable amount about PowerVR GPU's and also texture compression[/Disclaimer] Apologies for being lazy and not reading everything, but a) you aren't modifying a given texture from frame to frame are you? That might easily result in extra memory consumption as a TBDR GPU could be running asynchronously and the driver might want to avoid dependencies. b) If it's of help it seems to me the FIC system could be extended to use either 4bpp or 2bpp PVRTC instead of uncompressed data. \$\endgroup\$
    – Simon F
    Apr 2, 2015 at 15:00
  • \$\begingroup\$ Nope, I'm not modifying the textures in any way. I just need to display them in rapid succession. The problem is that FIC works directly with Core Animation and Core Graphics. These systems don't have any notion of PVRTC. I'd have to create an OpenGL ES context and work directly with the GPU, which I may try to do soon. (I don't know much of anything about GL programming yet...) \$\endgroup\$
    – user5364
    Apr 2, 2015 at 16:57

1 Answer 1

Reset to default
1
\$\begingroup\$

So the short answer to my own question is, "Yes."

The long answer is...

Using Metal—and, I assume, OpenGL ES—you can create no-copy data buffers from memory-mapped files on disk. From there, you can create textures from those buffers. There are constraints, however, in that the texture data must be an uncompressed pixel format, can only have one mipmap level, etc. This makes sense, since if Metal is going to trust you to provide the texture data entirely without it touching the bytes, it must be in a "color-renderable" format right away.

The problem with this is that uncompressed textures aren't appropriate for me, as my disk usage would be enormous. However, I learned that PVRTC textures are decompressed on-the-fly as needed by Metal. So while I can't avoid any memory usage with PVRTC, after the initial texture load operation, the memory footprint is rather low.

Improve this answer
\$\endgroup\$

You must log in to answer this question.