# State of the art in image compression?

I'm looking for good algorithms for compressing textures offline (ie decompressing them at install or load time, I know about using DXT/3DC to save runtime video memory, but these create awful visual artifacts while often being larger in disk than a lossless format like JPEG-LS, even after using LZMA or something for entropy). Since the data will only need to be read by my program, I don't need a portable format like JPEG, which means I'm open to more experimental algorithms. To be clear, I'm concerned with reducing the download size of my game; I will be using them uncompressed on the video card.

I'm looking for formats that support different features... For example, 10 bits-per-channel color (HDR) for higher color fidelity under dynamic lighting conditions, normal map formats(16 bits in X and Y), alpha channels, etc, etc...

NOTE: My particular use case is for 2.5D backgrounds. In their uncompressed format, they are 10 bytes per pixel stored in 3 textures:

4 bytes - color  - D3DFMT_A2R10G10B10
4 bytes - normal - D3DFMT_G16R16F
2 bytes - depth  - D3DFMT_R16F


I store these in 1024x1024 tiles, and since every area has a different background, these can get really large really fast. However, I think knowing more about image codecs in general will help with finding an efficient method to compress these.

• Is this related to gamedev? – Daniel says Reinstate Monica Oct 11 '11 at 23:11
• @Dan - yes; this is for a game... are you saying this is the wrong place for the question? Images in games have different requirements than other images (ie JPEG would be unsuitable) and a good normal map compressor would take into account the particular format of the data. – Robert Fraser Oct 11 '11 at 23:37
• You are dealing with sprites. I'm not really sure what it is you're trying to do with sprites that would need, "10 bits-per-channel color (HDR) for higher color fidelity under dynamic lighting conditions, normal map formats(16 bits in X and Y), alpha channels". It'd be a lot easier to just make and animate an actual model; it'd probably act as a more effective form of image compression than anything you might find or create. – Nicol Bolas Oct 12 '11 at 1:21
• @Nicol Bolas - I updated the question with my specific use case. – Robert Fraser Oct 12 '11 at 2:30
• If I understand correctly, your whole question could be rephrased to: "What's the best losslessly compressed image format". – Jari Komppa Oct 12 '11 at 9:26

I'm looking for good algorithms for compressing textures offline

What is your purpose in compressing these images? There are generally two reasons to compress a texture:

1. You want to make the texture smaller so that it takes up less memory on the GPU.

2. You want to make the texture smaller so that it takes less time to load/requires less harddrive space, resulting in a smaller download.

The most important thing you can understand is this: you cannot satisfy both of these at once. You must pick one: smaller GPU memory, or smaller disk space/load time.

So let's look at both cases.

## Case 1

Your possibilities here are the various "Block Compressed" types, using the D3D 10 language for them.

I'll assume you know what BC1-3 are, since they are formerly known as DXT1, DXT3, and DXT5, respectively. BC4 and BC5 are for 1 and 2-channel formats respectively. These two can be either unsigned or signed normalized.

BC5, 2-channel compressed, can be used for storing tangent-space normal maps, with the third coordinate (Z) reconstructed in your shader. It does a reasonably good job.

BC6H and BC7 are quite new. BC6H is a compressed, floating-point format. BC7 is a more accurate way to compressed RGBA color data. You can only get BC6H and BC7 support from DX11-class hardware.

The OpenGL internal formats for these are as follows:

1. BC1: GL_COMPRESSED_RGB_S3TC_DXT1_EXT and GL_COMPRESSED_RGBA_S3TC_DXT1_EXT
2. BC2: GL_COMPRESSED_RGBA_S3TC_DXT3_EXT
3. BC3: GL_COMPRESSED_RGBA_S3TC_DXT5_EXT
4. BC4: GL_COMPRESSED_RED_RGTC1 and GL_COMPRESSED_SIGNED_RED_RGTC1
5. BC5: GL_COMPRESSED_RG_RGTC2 and GL_COMPRESSED_SIGNED_RG_RGTC2
6. BC6H: GL_COMPRESSED_RGB_BPTC_SIGNED_FLOAT_ARB and GL_COMPRESSED_RGB_BPTC_UNSIGNED_FLOAT_ARB
7. BC7: GL_COMPRESSED_RGBA_BPTC_UNORM_ARB

The BC1-3 and BC7 formats can also be in the sRGB colorspace. OpenGL provides similar sRGB variants and GL_COMPRESSED_SRGB_ALPHA_BPTC_UNORM_ARB.

Obviously, none of these are lossless.

As I understand it, BC6H and BC7 are not widely supported in tools, though I seem to recall that NVIDIA's texture tools do support them.

## Case 2

If you're talking about normal 8-bit-per-channel images, you already know your options. But for more exotic texture types, like floating-point, 10-bit-per-channel, and normal maps, you're out of luck.

The simple fact is that most general image compression software is intended for regular old images. Most people who deal with floating-point textures don't really care enough about loading speed or disk space to bother with compressing them. They're big, but they've accepted that. And it's really hard to losslessly compress floating-point values.

So really, just stick with known paths for regular images, and use zlib or 7z or some other general compression method for the others. That's generally about the best you can do.

Do note that any lossless image compression technique can be made to work on any pixel data, so long as that data can fit. So you can always put a normal map in a PNG, even if it is a 2-component normal map (the third component is 0). It may not compress well, but it will probably beat out pure-zip.

4 bytes - color - D3DFMT_A2R10G10B10

If you're storing a color, a normal, and a depth, then I can only assume that you are doing some form of deferred rendering-type thing here. That is, the color is the diffuse reflectance, not the light radiating from the background.

Given that, 10-bit colors are pretty much overkill. You don't need that level of color accuracy for a diffuse reflectance value; you can get the same level of accuracy from an sRGB color value. HDR is all about the lighting; you can do HDR just fine with lower color-depth.

Once you're dealing with regular sRGB colors, you can start using standard image compression techniques.

4 bytes - normal - D3DFMT_G16R16F

There's no reason you can't take that down to G8B8 using signed-normalised values. Games do this all the time, and normals are not sensitive enough to need 16 bits worth of accuracy. And they're normals, so even using floating-point is way overkill.

Whether you can live with the artifacts produced by feeding it through BC5 compression is up to you. But you can at least go down from 32-bpp to 16-bpp with no detectable loss of fidelity.

Ultimately, everything is a tradeoff. You need to decide how much image quality you are willing to give up to get your game's data size under control. And since it's a tradeoff, you need to run actual tests to see where you are willing to draw that line.

• Thanks for the detailed answer - yes, I'm mostly interested in case 2 and it saddens me that there's not more research in this area :-(. – Robert Fraser Oct 12 '11 at 2:29
• Wow; thanks for the upate! I will see how it looks with 8-bit normals... As for the colors, the idea is to bring out detail under different lighting conditions. If a very bright light shines on a dark area, you don't just want gray. The difference was noticeable when I implemented it, but perhaps not enough to justify the increased texture size. – Robert Fraser Oct 12 '11 at 19:59
• @RobertFraser: One more thing: the depth should also not be floating-point. Since it's on the range [0, 1], you're wasting bits by using a 16-bit float over a 16-bit unsigned normalized integer value. – Nicol Bolas Oct 12 '11 at 20:03

There's WebP which Google have been suggesting as an improvement over JPEG.

For 2D images where compression artefacts are unacceptable, I'd go with some form of lossless compression like PNG.

Also note that DXT compressed images will usually shrink significantly when zlib or other lossless compression is applied to them.

For more exotic formats like HDR and 16-bit normals you could try OpenEXR. I've not used it myself though.

• Thanks. Yes, I'm aware of PNG, but if I wanted true lossless for a 24bpp color image, I'd use something like JPEG-2000 or JPEG-LS. I'm looking for more exotic things here -- most codecs for web are 10+ years old and are unsuitable for many data formats used in textures. – Robert Fraser Oct 12 '11 at 0:45