# What do you use to bundle / encrypt data?

More and more games are going the data driven route which means that there needs to be a layer of security around easy manipulation. I've seen it where games completely bundle up their assets (audio, art, data) and I'm wondering how they are managing that? Are there applications / libraries that will bundle and assist you with managing the assets within? If not is there any good resources that you would point to for packing / unpacking / encryption?

This specific question revolves around C++, but I would be open to hear how this is managed in C#/XNA as well.

Just to be clear -- I'm not out to engineer a solution to prevent hacking. At the fundamental level we're all manipulating 0's and 1's. But, we do want to keep the 99% of people that play the game from simply modifying XML files that are used to build the game world. I've seen plenty of games bundle all of their resources together. I'm simply curious about the methods they're using.

• Good question, while i don't belive there is a way to 100% protect it, i do belive there are neat way of compacting and obfuscating it and i would like to know more about it... – Prix Aug 3 '10 at 18:13
• It's worth questioning whether your art/audio assets should even be considered "sensitive". Many games purposefully expose their assets to players, in order to support a healthy mod community, which in turn can add more value to the game (at virtually no cost to the developer, even). – Ian Schreiber Aug 3 '10 at 19:05
• I'm mainly concerned with sensitive data that drives a data-driven game (xml files, for example). I've just seen games bundle everything together into one or two files. – David McGraw Aug 3 '10 at 19:09
• What's wrong with players toying around with that data? – jsimmons Aug 4 '10 at 1:54
• to beat my own drum.. CFL: iki.fi/sol/cfl – Jari Komppa Feb 2 '11 at 6:41

A good strategy is to make your own archive format from scratch, and to make it an indexed one with a table pointing to data chunks instead of storing whole sequential files in your archive, then encrypt your index table and compress every chunk separately using LZMA/Bzip2.

Another approach is using bits of procedural data generators here and there (If you want to go that far you could even get the 'parameters' for said generators from an on-line server as an extra measure), And by offloading some of the work to the GPU you can also prevent full memory dumps and make up for some of the lost performance.

Yet I find such techniques to generally hinder performance and waste development time, all the while offering nothing to the end-user but potential problems and unneeded bugs.

Nevertheless, nothing is 100% secure.

• +1 for this. Compression satisfies both the need to obfuscate, and will improve loading times and app footprint. Archive/packing of data will require an abstraction layer anyway, so combining it with something that can load/decompress in place shouldn't be too much engineering. If you do go with compression, I'd make sure to keep the compression/decompression logic separate from the loading/archive stuff. That way you can swap in a better algorithm at a later date, the infrastructure needed doesn't change between algorithms. – MrCranky Aug 4 '10 at 12:25
• @MrCranky: If you look at the way EA does it with their .package (DBPF) archive format (used in SPORE and The Sims games), you'll actually see this in action. Their engine has an extra field in the index table for "compression type", an internal code for what algorithm to use. They use different compression algorithms based on the chunk type (text chunks use a special compression for example) The versioning system they use also allows for integration of mods and backwards compatibility. More on the DBPF format here: simswiki.info/DatabasePackedFile – voodooattack Aug 4 '10 at 12:45
• Interesting stuff: I've only ever done it at compile time with a #define, but runtime opens up some nice possibilities (such as being able to change your compression type without having to rebuild every file for your game). – MrCranky Aug 4 '10 at 13:00

Don't ship sensitive data.

There is no way to ship encrypted data so that only the game can access it. You can try really hard but you quickly get into the DRM issues.

If you want to keep your game assets away from prying eyes.

1. zip it up
2. xor the file
3. change the extension to something boring (ie not my_level.zip.xor, use my_level.map) .

That will stop 95% of mischeivieous users. The other 5% are going to get it no matter what you do.
You don't need to actually zip and xor. You could instead lzma and use AES with a known key. The point is to not spend a lot of time on it.

• That is what i usually see around from most games ... cookie for you. – Prix Aug 3 '10 at 18:19
• Really just zip+rename is probably enough. – coderanger Aug 3 '10 at 18:23
• i dont think so, i like the That will stop 95% of mischeivieous users. out of breaking it in, while i do belive there are still people that will actually get it i would appreciate if not all and each are able to. These days most 10 y old kids know how to open files in notepad and rename it ... – Prix Aug 3 '10 at 18:41
• Know how, yes. Will ever care, no. – coderanger Aug 3 '10 at 19:11
• zip+rename isn't enough. Linux rarely uses file extensions to determine the file type. Instead it looks at the first few bytes and uses that to determine its type. A renamed zip still shows up as a zip in Linux. Windows XP does not do this, it still uses extensions. I don't know how Windows7 does it, but I expect that in the future Windows will use a similar technique. – deft_code Aug 3 '10 at 19:35

Spending any large amount of time on this is probably foolish. There are tools to grab texture and model data directly from the GPU, and sounds can easily be pointed at a virtual loopback device. Anything that needs to be truly secure needs to live on a server and work only via challenge/response. Doing client data validation is another story, something more like Valve's -verify_all. That is still hard, since you can't trust the local executable to enforce it (or at least you can't trust it more than you trust your end-point anti-cheat system, which is also a silly idea).

My job is in antitamper software. I know somebody who is an old school game cracker. He cracks games like breathing. You can't stop people like him. If your game is popular enough, multiple cracked versions of your game will hit the torrent sites only a few days after release no matter what you do. My advice is to just do the bare minimum to keep out the average joe who thinks he might want to give this reverse engineering thing a stab.

• That is the goal. I have absolutely no desire to fight hackers. – David McGraw Aug 3 '10 at 20:57

Good question but this is a far from easy to question to answer.
From my experience with data security you have to think about the following:

1) What are you securing, from whom and what is the impact if that data is insecure - security is far more about risk assessment than systematic technical solutions
2) I assume your platform is desktop (e.g. Windows/OSX) in that case you have a different problem space than proprietary hardware such as PS3/Xenon which have their own attempts at kernel level security ( whether this is a valid solution is open for debate )
3) Encryption != security. Encryption only guarantees that your data can't be read, it does not guarantee that your data can't be written or replayed/proxied.

If your requirements are to make your data tamper proof then you need to think about hashing your data ( preferably with SHA-256 or SHA-512) To re-iterate though, think about why it's important and what you gain from adding security to your system.
It's also worth knowing that adding encryption tech to your product may have legal implications due to export restrictions ( encryption tech is classed as restricted weapons grade technology believe it or not ) so using libraries that have overcome these legal issues is advisable.
Finally - never, ever roll your own encryption tech - use well known, documented, researched algorithms such as AES, Blowfish, RSA etc

ps Public/Private Key is not suitable or practical for encrypting large amounts of data hence DRM/PGP etc use a hybrid RSA + symmetric system i.e secure the symmetric key with RSA then do standard symmetric encryption using that key

It's a very large complex subject and I've definitely not done it justice

Hope that helps

If you want an example of a format that was designed and built (probably overengineered) for being small (compressable), ridiculously secure and flexible, you should look to Blizzard's MPQ format.

http://wiki.devklog.net/index.php?title=The_MoPaQ_Archive_Format

The general archive layout is as follows:

Archive Header File Data File Data - Special Files Hash Table Block Table Extended Block Table Strong Digital signature

This format has EVERYTHING. Blocktable, extended blocktable, data split into sectors (for easy streaming), support for 7 (!) compression algorithms (if you count sound compression) that can be combined arbitrarily, weak digital signature, strong digital signature and so on.

If you were to buy the API from Blizzard that deals with file architecture, you'd get everything you'd ever need and so much more.

• Thanks for the link, I'd never thought to look more into the details of MPQ (and generally assumed they'd be unavailable) – Bill Jan 31 '11 at 21:04
• If it is so secure, why has it been reverse engineered to such a degree and even has 3rd party libraries to interact with it? ;-) – coderanger Feb 1 '11 at 0:37
• Most (read: all) tools for interacting with *.mpq files depend on Stormlib, which is written and maintained by one guy. – user5052 Feb 1 '11 at 7:15
• As far as I know, Stormlib requires you to have the original game that the archive stems from, as the keys for decryption are stored in the executable. The biggest reason why it is so secure, though, is not because it is impossible to break into. But for on-the-fly data sent through the net, it is second to none. The data is so heavily encrypted that it would be futile for a third party to decrypt on the fly. – user5052 Feb 1 '11 at 7:24
• And in fact with 'World of Warcraft', Blizzard has pioneered streaming technology, allowing you to play the game with only a few percent of the entire data downloaded onto your harddrive. The *.mpq format excels at this kind of job. Not to mention that when the Starcraft 2 installer was released by Blizzard some days before the official release of the game, its archives were so heavily encrypted that they weren't openable until way after the game was released. :P – user5052 Feb 1 '11 at 7:27

As this has recently been resurrected, your question (and therefore all answers so far) only seems to deal with the file loading, which is only as secure as the executable that wants to load it. On the PC there's nothing that's really secure (you just delay people), you can check against stuff on servers, but even that check can be removed so modded exe's can be run no matter what. The consoles are still "pretty" secures so in most cases there I generate a hash of important files, then sign the hash with a private key, embed the public key in the code and verify the secure hash against the file.

I thought I'd throw another attack into the conversation, something I only came across last year as I don't tend to ship much on the PC. Cheat Engine is a very tidy little program that almost automates making your own hacks (which means anyone can use it). It's pretty advanced in what it can do, a lot of hacks/cheats that I've seen people sell are made by this or something very much like it from what I've seen (all the mass market ones). With some work you can protect against stuff like this again, I guess it all depends on how popular the game is and if someone cares enough to bother.

Opps, forgot to mention a good lib for encryption (in C++) is LibTomCrypt.