What are the techniques used to save space with very large levels?

Question

Some days ago I was thinking "how would I store the real world into a continuous game level ?".

I guess it must have been some challenge for RockStar engineers (for GTA 4 or Red Dead Redemption) or when developing RAGE (the ID game, not the rockstar engine), and I wonder how they solved it.

Of course current RAM and hard disk capacities don't allow it if you don't use high definition content, so you simplify by abusing the reuse of objects:

• Use doodads for trees, barriers, bushes, benches, trashbins, etc
• Use palettes instead of textures for the ground and walls and buildings
• Use heightmaps for altitude
• Use procedural method for modeling roads, houses, buildings, sidewalk, pavement
• Also use procedural methods to model the insides of buildings: walls, ceiling, stairs, doodads.

These are the only things which make sense to me, but I want to know if they are other methods I'm missing, like for example reducing vertex precision while storing models, but requiring to make an exporter.

Answer 1

I use a template model. Every object in my game has a template ID, which is an integer. You can use the template ID to look up from a database or spreadsheet (or whatever you'd like), all the information about that kind of object. For example

Template ID     Name           Render Mesh          Physics Mesh
1               Pine Tree      Pine_Tree.nif        Pine_Tree_Phys.hkx
2               Wooden Crate   Wood_Crate.nif       Wood_Crate_Phys.hkx

Then all I store in my level files for each unique object is the template ID, position, rotation, scale, and optionally any custom data by using strings.

So if you have 2 trees and 1 crate in your level, the level file would look something like:

Template ID     Position           Rotation         Scale        Custom Data
1               (0, 1.5, 3.7)      (1, 0, 0, 0)     1.5          ""
2               (17.1, 3.2, 19.5)  (1, 0, 0, 0)     1.0          ""
1               (9.1, 52.4, 35.5)  (1, 0, 0, 0)     0.9          ""

Your level files should end up being relatively small by doing this. You could probably manage to pack the data even tighter if you placed all matching template IDs sequentially in your file so you didn't have to continue specifying template IDs for every matching entry. You could also compress the level files, uncompressing them only to load a level into memory (or parts of a level).

You can also break up your level files by physical locations within a level. For example if a level is very large then you could break up the level file into multiple smaller files, making each file easier to load into memory.

Heightmaps can be done fairly cheap. Generally only grayscale is required to get enough elevation detail for a map, so using a grayscale PNG file should keep the files fairly small, even for large maps.

Another technique is to scale your terrain horizontally. As you convert the heightmap image to actual vertices, multiply each X and Z coordinate by a scalar value (e.g. 4). Using a scalar of 4 would get you a terrain that is 4 times the size of your heightmap image. So if your heightmap image is 1024x1024 then your terrain will span 4096x4096 in the world. This allows you to have larger worlds, or the same size world with a smaller heightmap image. You lose detail on your heightmap as you scale, so I recommend finding the highest scale possible that is still an acceptable quality for your game.

If you're using texture splatting through an RGBA texture, to achieve multi-textured terrain, you can save on memory by breaking up the splatting texture into many smaller textures and using a RAW format instead. Let's say your terrain is 256x256, and your terrain can have up to 4 different textures that can overlap. By subdividing the splatting texture you may find that some of the 128x128 quadrants of the terrain only have 1, 2, or 3 textures in use in that section, so you can now store less information in those quadrants. If one quadrant only had 2 textures, for example, let's say using the Red and Green parts of the RGBA texture, then storing information for the Blue and Alpha for that quadrant would be a waste. So you break up the 256x256 texture for your terrain into four 128x128 raw files, and the quadrant using only Red and Green can now store only the raw data for those two colors, using nothing for the other two colors. If you use this method you also only have to load up the raw files for the portions of terrain that are visible, rather than the larger texture for the entire terrain all at once. Games like World of Warcraft use technology similar to this.

Answer 2

What most games do is break the map up into pieces that can be loaded dynamically.

Answer 3

Well there's many ways, simplifying the geometry.but also, that game is just huge. Also however, pieces are probably reused.