Does using the XNA Content Pipline eliminate the overhead caused by file IO when working with a large number of files?

Question

I'm working on a game in XNA with my goal being that it is fairly data driven so that I can easily tweak the system without having to update code. The research I've done shows that using XML files combined with the content pipeline should provide me with what I need to accomplish this.

That being said, being a developer that really has only been exposed to object oriented programming I have this "one class per file" mindset that I really like because it makes finding and updating what I want to change very easy. This leads me down a path of wanting to set up a system where instead of a large, heavily nested XML file that defines everything (like the web.config files I am used to), I want to go with a folder structure with files that define individual objects - or at the very least something in between. At this point in time I'm not entirely sure how many files I could end up having but I believe it could escalate to a point where it could cause a performance problem if done wrong and if I discover a performance problem because of it, redesign and consolidation of it all could be a big enough chore that I definitely want to try my best to make the right decision up front.

Usually I would see the above mentioned approach being a big performance hog because of the large amount of file IO that would have to be done when loading up the game. Based on my experience with images in the content pipeline, I am assuming that XML files are also combined into a single XNB file when compiled. From this my questions become:

• If it is true that all XML files become a single XNB file should I still expect a notable performance loss when loading the data from these "files" or has that overhead disappeared since its all within a single XNB file?

If yes I am sure there tools out there that can make managing / updating a single and very large XML file more manageable which I will begin researching and am open to suggestions on.

Answer 1

Resources like shaders, textures, fonts, etc. Each live individual XNB files after being processed by the content pipeline.

XML files may be used as an index of resources and configuration data, but I'm having a hard time imagining a situation where it would be beneficial to include the XNB contents within the XML.

Given that, XML files are typically pretty small. I doubt you'll see any performance degradation from that.

If your concern is about loading large numbers of different files from disk(platform?) then remember that the won't be loaded until you call Content.Load. If your app is structured correctly, I doubt you'll be loading thousands of items at the same time. They should be loaded just before their first use when possible.

Answer 2

XML files do not get merged into a single XNB, unless you have a custom content processor that you've specifically configured to do so (you'd know if you had, it's a non trivial process).

I recommend going with whatever file structure helps you develop most quickly, and in the end if you have a tremendous number of files you may want to consolidate them into a single data file that is read by a custom content manager. This generally isn't necessary on Windows, since the file system there is pretty good at dealing with quite a few files, but the XBox does noticeably suffer when there are lots of files in a folder. Here's a thread on that topic: Pack of resources in one big file with XNA

A proper setup will take care of all this automatically during a standard build.