68
\$\begingroup\$

I've just finished a basic RPG game written in C++ SFML, I've put a lot of effort into the game and I'd want to distribute it, however I've ran into a small issue.

Problem is, I have well over 200 images and map files (they're .txt files which hold map codes) all in the same folder as the executable, when I look in the folder, it makes me want to cry a little bit seeing so many resources, I've never seen a game which shows you all the resources directly, instead I believe they pack the resources in a certain file.

Well, that's what I'm trying to achieve: I'm hoping to pack all the images in 1 file (Maybe the .txt files as well) then being able to read from that file or easily add to it.

\$\endgroup\$
3

9 Answers 9

64
\$\begingroup\$

Game programmers have relied on one of two main methods of data storage:

  • store each data file as a separate file
  • store each data file in a custom archive format

The drawback to the first solution is the wasted disk space problem, as well as the problem of slower installations.

The second solution provides it's own pitfalls, first is that you must write all your own image/sound/etc. loading routines which use a custom API for accessing the archived data. A further drawback is that you have to write your own archive utility to build the archives in the first place.

Unless you will always load all files from the archive, TAR/GZ might not be a very good idea, because you cannot extract specific files as you need them. This is the reason many games use ZIP archives, which do allow you to extract individual files as required (a good example is Quake 3,​​​​ whose PK3 files are nothing but ZIP files with a different extension).

EDIT : "Hide" the game folder structure and "Keep" only the executables

Another solution is often used to "hide" the game files in folder structure. Keep only your executables and maybe a readme file in the main directory and move the game files into a sub folder named "data" or other related.

EDIT : Gamedev Tuts Plus have a nice resource

EDIT : libarchive

One potential solution libarchive, which is an archiving library that will handle extracting files from an archive such as a ZIP file. It even allows you to assign the extracted file to a standard FILE pointer, which would make interfacing with any other libraries potentially more straightforward.

EDIT : ZIP Format files with alternative extension

Here, I like @Joachim Sauer's Comment

Using ZIP-format files with alternative extensions has a great tradition outside of games as well: Java does it, the OpenDocument Format (a.k.a. the OpenOffice/LibreOffice format) does it, even Office Open XML (a.k.a. the "new" Microsoft Office format) does it.

\$\endgroup\$
25
  • 2
    \$\begingroup\$ Yeah, Thief/SystemShock2 used .zip files renamed something else too. In Java you can use something like TrueZip to access files in zips as if they were files in folders, I imagine there are similar libraries for C++. \$\endgroup\$
    – Nick
    Sep 25, 2012 at 10:46
  • 18
    \$\begingroup\$ Using ZIP-format files with alternative extensions has a great tradition outside of games as well: Java does it, the OpenDocument Format (a.k.a. the OpenOffice/LibreOffice format) does it, even Office Open XML (a.k.a. the "new" Microsoft Office format) does it, ... \$\endgroup\$ Sep 25, 2012 at 12:46
  • 5
    \$\begingroup\$ @Svish. Depending on the platform, the amount of disk space taken up by a lot of small files can be considerably greater than the amount taken up by one large file, even if that large file is not compressed at all (.tar, for example). It's to do with the way data is physically stored on the disk. \$\endgroup\$
    – TRiG
    Sep 25, 2012 at 16:33
  • 1
    \$\begingroup\$ Ah, that's true. Shouldn't be much though, unless there are a lot, but in that case it can indeed build up. I can imagine installation and uninstallation also takes a hit with lots of small files. \$\endgroup\$
    – Svish
    Sep 25, 2012 at 17:01
  • 2
    \$\begingroup\$ Slightly related to Kylotan's comment, here is a post by Jonathan Blow on The Witness' dev-blog about using archives and how they interact with Steam's patching mechanism. "The Witness packs most of its data into a single 2-gigabyte file stored in the .zip format. Before uploading the initial version to Steam, I took a safeguard to minimize patch sizes, or so I thought [...] (Imagine that files are stored in random order: probably every patch involves every player re-downloading the entire game!)" \$\endgroup\$
    – Eric
    Sep 25, 2012 at 18:25
39
\$\begingroup\$

Another solution often used to "hide" the game files is folder structure. Keep only your executables and maybe a readme in the main directory and move the game files into a sub folder "data". I don't think that it is very uncommon to do so. Many games I know store their content in such a way.

\$\endgroup\$
6
  • 8
    \$\begingroup\$ +1 If disk space or protection isn't an issue, that's the easiest fix. \$\endgroup\$ Sep 25, 2012 at 8:14
  • 1
    \$\begingroup\$ +1 because the other guy said +1. Can't be wrong, eh? (Joking aside, great answer.) \$\endgroup\$
    – jcora
    Sep 25, 2012 at 14:20
  • \$\begingroup\$ This, in my opinion, is the ideal choice. If the filesystem can't handle that efficiently the filesystem is broken and you should be leaning on the OS maker to improve it. \$\endgroup\$ Sep 25, 2012 at 18:05
  • 3
    \$\begingroup\$ @Omnifarious That's not always true, e.g. when reading from an optical disk, even with a perfect filesystem, it's common to pack files in loading order to prevent excessive seeking (the speed up can actually be quite surprising). But this is a totally different story for hard disks, and I tend to think the OP is in this case. \$\endgroup\$ Sep 25, 2012 at 19:17
  • 3
    \$\begingroup\$ @Mr.Beast You missed my point. I'm talking about optical disks. I've worked for a studio where it was a single programmer's full time job to make sure files would be packed as efficiently as possible on a DVD, to get the best loading time possible. I'm sure he'd be glad to learn he could just have relied on the DVD file system ;) \$\endgroup\$ Sep 26, 2012 at 10:14
22
\$\begingroup\$

I really like PhysFS for this. It allows you to access either folders or zip archives with the same code. It works well for all stages of a Games lifetime.

  1. During development: access the resources directly from a folder hierarchy. This way compressed archives are not in the way and you can rapidly iterate.
  2. Initial deployment: zip up your resources for easy deployment / quick installs. No code needs to change. Just point PhysFS at the zip instead of the folder.
  3. Updates / Modding: PhysFS can load multiple zip archives where the resources from one override the other. Updates can be as simple as a new zip with only the changed files. Similarly for modding.

update:

I used the tutorial included with the source and the doxygen documentation to learn PhysFS. The API really is fairly simple, you'll spend more time learning how to load images into SFML via memory buffers instead of by file path.

\$\endgroup\$
2
  • 4
    \$\begingroup\$ I'm using PhysFS for my current project and I love it. I don't always need to regenerate my content archive file; I can just drop an updated version in the search path and BAM! It works. \$\endgroup\$ Sep 25, 2012 at 17:37
  • \$\begingroup\$ After some answers I decided I'll go with physFS. Would you care to recommend a nice tutorial? :) \$\endgroup\$
    – Bugster
    Sep 25, 2012 at 17:51
12
\$\begingroup\$

I'd suggest using a ZIP file. It's an ubiquitous format and you'll find ready-made libraries that allow you to load files from within a ZIP file. A quick Google search even revealed a zip loader for sfml.

\$\endgroup\$
3
\$\begingroup\$

Well, you could cheat like they mentioned:) You can make a sprite-sheet from the imagery, there is no need to zip it nowadays unless it saves a ton of space. After you create a sprite-sheet or several sprite-sheets from the images, you could simply rename their extensions. Most Gamers will not bother to check and why not leave that option to Artists who may want to mod your game in the future? That way they can too create a sprite-sheet, rename it's extension and start rolling.

With the text files, I would humbly suggest you convert that data into binary form. I am pretty sure you will find a simple way to do it.. For instance, if you only use A-Z, a-z and the 0-9, you can use 6 bits to represent each character, that will somewhat protect your copyrighted material, it will also prevent others from editing maps.. You can always add a map converter if you like. Zipping is completely reasonable for text though.

\$\endgroup\$
4
  • \$\begingroup\$ Converting to binary is good advice but not for protection - it's just going to be easier to parse and faster to load. Would suggest it as a pre-processing step, but that's going to off-topic for the question. \$\endgroup\$ Sep 25, 2012 at 14:04
  • \$\begingroup\$ Well, I am guessing, if he wants to protect it more effectively, he could try to encrypt it in RSA or a similar algorithm(there is no need). I doubt anyone will make unauthorized use of the map files from an indie game if it's inside a binary file. It just doesn't seem worth investing the time in figuring our the parsing. Text files are just completely vulnerable and as you suggested something ineffective for storing data. \$\endgroup\$
    – AturSams
    Sep 25, 2012 at 14:13
  • 3
    \$\begingroup\$ On HDDs, loading a single zip-file is faster than a lot of small files because there are less expensive random seeks to the physical disk. A binary for a small game can be mapped into memory and work "magically", almost no parsing required. \$\endgroup\$
    – Liosan
    Sep 25, 2012 at 14:16
  • \$\begingroup\$ @Liosan You could have misinterpreted what I meant or I misunderstood you.. I said binary is better for storing data and that other game designers are unlikely to want to learn how to parse your binary map code, unless it is documented and encouraged for modding purposes. Therefore it offers some rudimentary protection over text files. \$\endgroup\$
    – AturSams
    Sep 25, 2012 at 14:23
2
\$\begingroup\$

It's not only about the number of resources or disk space.
With a lot of different texture files, as you are using SFML, which renders with OpenGL, each time you are going to render a texture, OpenGL needs to bind the next texture to the video card (check glBindTexture() ), and it is a really expensive task.
With that in mind you can realize why games usually put a number of sprites in the same texture, the so called spritesheets, according to your menus/levels/maps of the game.
The result is a huge gain in performance, since you are rendering a lot of sprites with the same bind texture call.

\$\endgroup\$
1
  • \$\begingroup\$ After getting several references to making a spritesheet I've considered some refactoring to lower the image count. Thanks \$\endgroup\$
    – Bugster
    Sep 25, 2012 at 19:42
2
\$\begingroup\$

I would personally head down the custom binary file route for this. This is something I do myself all of the time now, it isn't as difficult as it might sound, and it also provides a level of obfuscation for you game resource files. I wrote a small introductory Gamedev article about this not so long ago if you are interested:

http://gamedev.tutsplus.com/tutorials/implementation/create-custom-binary-file-formats-for-your-games-data/

Sprite-sheets are a different subject entirely but they are the norm for most games :)

\$\endgroup\$
1
\$\begingroup\$

Another method you can use:

The free version of XnView provides a feature called "Strip of Images" (SHIFT+A), which allows you to create sprite-collections.

This minimizes file access and is particularly important for web applications/games to minimize the number of HTTP-roundtrips.

Access to individual images is then granted through coordinate maps (for instance css-definitions).

\$\endgroup\$
0
\$\begingroup\$

First you must write all your own image/sound/etc. loading routines which use a custom API for accessing the archived data. A further drawback is that you have to write your own archive utility to build the archives in the first place. Unless you will always load all files from the archive, TAR/GZ might not be a very good idea, because you cannot extract specific files as you need them. This is the reason many games use ZIP archives, which do allow you to extract individual files as required (a good example is Quake 3,​​​​ whose PK3 files are nothing but ZIP files with a different extension). http://zpp-library.sourceforge.net/

\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .