Game programmers have relied on one of two main methods of data storage:
- store each data file as a separate file
- store each data file in a custom archive format
The drawback to the first solution is the wasted disk space problem, as well as the problem of slower installations.
The second solution provides it's own pitfalls, first is that you must write all your own image/sound/etc. loading routines which use a custom API for accessing the archived data. A further drawback is that you have to write your own archive utility to build the archives in the first place.
Unless you will always load all files from the archive, TAR/GZ might not be a very good idea, because you cannot extract specific files as you need them. This is the reason many games use ZIP archives, which do allow you to extract individual files as required (a good example is Quake 3, whose PK3 files are nothing but ZIP files with a different extension).
EDIT : "Hide" the game folder structure and "Keep" only the executables
Another solution is often used to "hide" the game files in folder
structure. Keep only your executables and maybe a readme file in the main
directory and move the game files into a sub folder named "data" or other related.
EDIT : Gamedev Tuts Plus have a nice resource
EDIT : libarchive
One potential solution libarchive, which is an archiving library that will handle extracting files from an archive such as a ZIP file. It even allows you to assign the extracted file to a standard FILE pointer, which would make interfacing with any other libraries potentially more straightforward.
EDIT : ZIP Format files with alternative extension
Here, I like @Joachim Sauer's Comment
Using ZIP-format files with alternative extensions has a great
tradition outside of games as well: Java does it, the OpenDocument
Format (a.k.a. the OpenOffice/LibreOffice format) does it, even Office
Open XML (a.k.a. the "new" Microsoft Office format) does it.