Compression and encryption are different things AFAIK, so why even use compression if you have to still decompress anyway, is this just to reduce distribution size or is there some other reason?


closed as unclear what you're asking by Philipp, bummzack, Anko, Kromster, congusbongus Sep 22 '14 at 2:06

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    \$\begingroup\$ Compress what and where? \$\endgroup\$ – Philipp Sep 17 '14 at 14:49

Mainly compression is used to reduce the size of the distributive and game resource package. You should distinguish compression of the distributive and the resource package of the installed game. The distributive usually is compressed by high compression ratio algorithms, while the package of installed game can be compressed by algorithms with fast decompression speed to be able to decompress it on fly. For the distribution there is only one benefit - reduction of its size. However, for the package of installed game the benefit is not only a reduction of its size, but also possible a light speed up of file read operations. The resource package of the installed game might not be compressed, it is not always necessary.

The speed up of read operations could be achieved in case of slow disk read operations (mobile devices for example) and using algorithms with very fast decompression speed, such as LZ4

On mobile devises, I have noticed the following thing. The ratio of read time of compressed data to decompression time with LZ4 can be up to ten times. So depending on compression ratio, you can chose what types of files should be compressed and what not.

You should be aware of the following disadvantages of on fly decompression:

  • It is hard to implement streaming of compressed data and there could be serious loss of performance.
  • Memory consumption would be increased due to the need of allocating buffers for decompression.
  • \$\begingroup\$ Rreading from a rotating disk is super slow. Even the xbox one gets a budget of 10MB/s (recommended by microsoft). This is low. If you have a need to stream data at more than that you need to compress. \$\endgroup\$ – v.oddou Sep 18 '14 at 1:52

Mainly compression is in fact used to reduce distribution size.

If you think about it with digital music, you could download a CD as .wav and it's 650 MB, or as 256 KBPS AAC files and it's 1/5 the size with the same sound quality*.

It just makes sense to compress anything you can. You're taking the compression time on your side once to save time downloading or reduce storage media size for everyone who uses the product.

As a side note, yes encryption is different from compression. Encryption makes it hard to read the data if you're not the intended recipient. When making an iOS game, we send a compressed archive to Apple, then they encrypt it, which makes the package bigger.

* I have done blinded testing on this. There's no discernable difference between uncompressed audio and 256 KBPS AAC audio.

  • \$\begingroup\$ when you tested AAC did you test with (almost) profesional grade headset and sound card ? may be quite off topic though. \$\endgroup\$ – v.oddou Sep 17 '14 at 5:00
  • \$\begingroup\$ Since this is an argument I don't want to have again, I'll just say this. I've done blinded tests with both an expensive stereo, and expensive headphones. The baseline test showed the subject could hear the difference if they knew which was which. As soon as the blinders go on, there is zero ability to discern the difference. So if you haven't done a blinded test, it's all in your head. :) \$\endgroup\$ – Almo Sep 17 '14 at 12:31
  • \$\begingroup\$ That's debatable, have you ever seen blind food taste tests, it's amazing how people can't identify chicken, beef or most foods for that matter, after their palette has been trained it's a different story. \$\endgroup\$ – user3333072 Sep 17 '14 at 13:03
  • \$\begingroup\$ One of the subjects was a musician. There are other factors with food; like texture and scent. Until you've done a blind test with this stuff, anything else is speculation on your part. Here's an example of just such a test, but this one is about amplifiers: tom-morrow-land.com/tests/ampchall \$\endgroup\$ – Almo Sep 17 '14 at 13:43
  • \$\begingroup\$ To prevent getting lost in semantics, how about you say "with no quality differences I could discern"? Or better, aking a comparison to FLAC, or other lossless algorith which is closer to the idea of compression in game assets \$\endgroup\$ – Panda Pajama Sep 18 '14 at 1:44

A big reason to compress game assets is to decrease loading times.

Example scenarios:

  1. Read uncompressed asset of 20MB size, or
  2. Read compressed asset of 10MB size and decompress.

In scenario 2 less data needs to be read from the hard disk which will save some time. If the CPU is faster at decompressing the read 10MB to the original 20MB than the hard disk in scenario 1 is capable of reading the remaining 10MB, compression will improve the loading time. Note that hard disks are generally much slower than CPUs. Many algorithms (you mentioned zlib's) even allow decompression on the CPU while the hard disk is reading in parallel.

You should pick an algorithm that

  1. compresses well (to reduce the time spent reading from the hard disk) and
  2. decompresses quickly (to reduce the time decompressing using the CPU).

Algorithms that would increase loading times are used when compressed size on disk is more important than decompression time, for example during installation of the game assets.

  • \$\begingroup\$ exactly. that's the main reason for a complicated geometry compression algorithm developped for alembic cache files in crytek's Ryse. \$\endgroup\$ – v.oddou Sep 17 '14 at 5:01

The answer is time

Use Compression?
    if uncompressed (down)load time > compressed (down)load time+decompress time then
        Win! use compression
        Lose! don't bother

Not the answer you're looking for? Browse other questions tagged or ask your own question.