How can I create a pack file / archive for game data files, and then load them? (zlib?)

Question

I'm starting to think about how I'm going to go about loading models, images, sounds, etc for my game. I have two competing lines of thought to deal with:

1. If possible, it would be nice to make use of existing libraries to load things like PNGs, etc.

2. I would like to pack all of my game data into a single archive, and it would be great to use an existing library here also.

Is it possible to do this? I've heard that I could use something like zlib for (2), but I've never used it before so I'm not sure how it would work. Is it possible to yank a file out of a zlib archive in memory, and then pass it to some compatible library for the actual loading? How would these two systems interface with each other? Would using zlib slow down the loading process?

I've not had much experience with I/O in my career, so I'm not really sure where to start.

Answer 1

I did pretty much exactly what you describe. Here's a run-down of how my system works:

I maintain a list of all asset files, along with some meta data. This takes the form of a table of contents (TOC) file at the root of a directory tree that will become the archive. Any file in the TOC is included in the archive, along with any metadata about the file.

The TOC is included in the actual archive file at a static offset so it can be very quickly read. It contains offsets for each file, so I know where to seek to grab the compressed data for a given file. Once decompressed, my ResourceManager takes the data from the archive and passes it to the appropriate format handler (ogg, png, etc.) which then produces raw data (PCM data, bitmap, etc.) The raw data is then given to an appropriate loader which will make sure the data is ready for use (on the video card, etc.)

I'd say what you want to do could be a good idea, but it could very well be a lot of unnecessary work. I would suggest you consider whether alternatives would suffice before embarking down this road (especially if you've not done a lot of IO before.) Get ready to spend a good bit of time in a debugger and looking through hex dumps if you go down this route. If you do, I suggest lots of test cases.

Answer 2

For starters you could go with just a folder (it's easy to add and modify 'stuff' in a folder) and later on when your game is up and running nicely, just add a level of zip/unzip (or whatever method you find that suits you) to the loading function and zip the folder up.

For the "Loading .PNG:s and such": This has absolutely nothing to do with the second question and will most probably heavily depend on the platform you are using (XNA, SDL, OpenGL, DirectX, etc.)

Answer 3

After looking into this for a while, it seemed that zlib wasn't going to satisfy my needs. It seems to be focused on compressing and decompressing individual files. I need something that is able to handle archives.

One potential solution I've found would be to use libarchive, which is an archiving library that will handle extracting files from an archive such as a ZIP file. It even allows you to assign the extracted file to a standard FILE pointer, which would make interfacing with any other libraries (PNG, my model loader) potentially more straightforward.