Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

Why do console game developers use large files to store game data instead of small files like on PC? Less streams in memory? Need to access file tree many times? Other reason?

share|improve this question
    
Perhaps an analog in the web world: CSS Sprites –  Nick T Nov 26 '11 at 23:53
    
CSS sprites is a nasty downside of still using that old rusty HTTP 1.1 which apparently makes servers unable to send many resources with a single connection. I wish for a new standard. –  Kos Nov 27 '11 at 9:57
1  
@Kos Your wish has been granted, although unfortunately not yet implemented. –  phihag Nov 27 '11 at 15:26
    
Thanks for this! I want to bake them a cake so that they'll work faster. –  Kos Nov 27 '11 at 15:43

2 Answers 2

up vote 32 down vote accepted

Having a single big file has a few benefits for performance, regardless of platform. Many PC games use big files too.

  • You can manually cache the offset of each file within the big file, to avoid the need to read file system directory entries (which can involve multiple seeks per file opened if you miss the cache).

  • You get full control of the order of the data within the file, so you can arrange it to minimize the need for slow seeks.

  • Data compression is generally more effective on a single big file, than on individual files.

Note that avoiding seeks is very effective at speeding up load times. A typical hard drive might have a 10ms seek time, and a 100MB/s transfer rate. Let's say you need to load 100 shader files with a total size of 1MB. If you have to do a seek for each one it would take one second to load them all. With a single seek you're looking at 20ms to load all of them.

Of course not all consoles have hard drives, and optical drives have even worse seek performance, so those shaders might take 5 or more seconds to load off a DVD when seeking. Also consoles have much less memory for use as cache than a PC does, so they tend to load data more often.

share|improve this answer
8  
Very nice answer. To the point, with examples of seeking one file vs multiple files. –  William 'MindWorX' Mariager Nov 26 '11 at 18:47
1  
Additionally, for competitive online games, it's much, much faster (and uses a lot less disk space) to verify that the player hasn't tampered with a single huge file, than it is to individually validate thousands of small files. –  Trevor Powell Nov 27 '11 at 2:09
    
Google keyword: virtual file system. Also the well-stated need for virutal file systems seems to indicate that non-virtual file systems could use some more polishing :(. –  Kos Nov 27 '11 at 9:58
    
However this is bad for those of us using SSDs - we can't move the rarely-loaded stuff onto HDDs, because all the game data is inside a big file. Say for WoW I never ever go to Outland; I'd just move that file off the SSD. Not so doable for say... CoD 4. –  Mircea Chirea Nov 27 '11 at 11:02

Apart from very good answer given by Adam I would also like to point out some other important factors:

  1. The fewer files you have the easier is the deployment of your application. This is especially true if you want to deploy your application on mobile platforms, like iOS, Android or Windows Phone. Even if in case of Windows Phone you still can have as many files as you want, since XAP-files, used for deployment, are just a zip archive, it is

  2. Having all your assets in one single files ensures that they are all downloaded/read "in piece". Many gamers will excuse you a noticeable delay for downloading assets or unpacking them before the game starts, but they will be non-indulgent as long they face even subtle pauses during the game. Of course you can pre-cache your resources in RAM, but loading them "in piece" is still easier for this purpose than loading them on per-file basis.

share|improve this answer
    
I have done Android and BlackBerry development and never ran into any problem with having too many small files. I was unaware of the benefit of larger files until now however... –  ADB Nov 28 '11 at 23:15

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.