I am using
.pngs for my textures and am using a virtual file system in a
.zip file for my game project. This means my textures are compressed and decompressed twice. What are the solutions to this double compression problem? One solution I've heard about is to use
.tgas for textures, but it seems ages ago, since I've heard that. Another solution is to implement decompression on the
GPU and, since that is fast, forget about the overhead.
I am using
The zip-format supports several different compression algorithms. You can use a different algorithm for each file in the archive. When you want to store already compressed files which do not benefit from additional compression (like PNG) in a zip-archive, you can encode these files with the "stored" algorithm which doesn't compress at all. The "Add to archive" dialog of 7-zip lets you chose this under "Compression strength".
But when you do not only have images but also other, more compressible resources in your archives, it might be quite tedious to choose the algorithm for every single file. In that case you might rather opt for an uncompressed image format in a compressing archive.
The TGA format knows a lot of different modes, of which some are compressed and some are not. When you do not want to use compression, make sure you pick the right one in the export options of the graphic editor you are using. Another non-compressing image format is BMP (Windows Bitmap).
Here is a test I made. I added the same image (an asset from my current project) in different formats multiple times to a zip-archive, some with "deflate"-algorithm on normal strength and one with "store". Sorry for the German GUI. 2nd column is uncompressed size, 3rd column is compression algorithm and 4th column is compressed size.
As you can see, deflate-encoding the PNG only saved a meager 0.3%, while the deflate-encoded BMP is reduced to one-tenth of the original file which is even smaller than the PNG version. This quite surprised me. I would have expected the PNG to be smaller because the compression method of PNG should be optimized for image-data while ZIP is not. A likely explanation is that my image editor (GIMP) added quite a lot of meta-information to PNG files which it doesn't do for BMP.
Uncompressed TGA behaved similar to BMP regarding filesize before and after zipping while the compression of the compressed TGA file was further improved by ZIP, although not as much as the uncompressed versions.
It might be worth to experiment with other algorithms than deflate and with other compression-strength setting. Which combination will have the best results will likely depend on the style of your textures. But you might also consider to benchmark the asset-loading of your game and have the decompression-performance influence your decision which setting you use.
Bottom-line: When you want to avoid double-compression while still having a low filesize, either use
Store zip algorithm or
BMP with a compressing zip algorithm.
Don't worry about it.
When the "deflate" algorithm used in .zip files encounters a block of data that's already well-compressed, like the pixel data of a .png image, it finds that it can't compress it effectively, and it will store it as literal uncompressed data. This takes very little overhead on the decompression end to copy out.
Looks like some very good answers have been given already, but I thought I'd give one more:
What are the solutions to this double compression problem?
Solution: Do Nothing.
Rationale: There has been no problem stated - yes you are compressing the information twice, but why are you worried about this? Is the data size too big? Is decompression too slow? Is this more or less important than the dozens of features you could be adding, refining,testing and/or debugging at this moment in time?
If you use specialized formats such as PNG and OGG you won't need the compression of ZIP.
PNG, OGG and other already compressed formats won't get much smaller by compressing them again as ZIP. 100MB of PNGs compressed are still ~100MB.
Scripts, Configuration files and other text based formats benefit greatly from compression, however, typically they are tiny in comparison, they don't store that much data. If your game is 100MB, then the text-files maybe make 1MB of the whole game, even if you can make them 100KB through compression then you only won 900KB, less than 1%, hardly worth the effort.
You might even want to use the file system directly instead of using a virtual, zip based file system. It would make patching very easy: you can just swap out any files you modified.