How does Terraria generate such large worlds initially?

Question

I understand how large worlds can be rendered and drawn efficiently by splitting things into chunks, but as I understand it Terraria generates the entire world when you first create it.

The largest world size available in Terraria is 8400 x 2400 which would be a total of 20,160,000 tiles.

Is the entire world generated in one go? Like for example having a large array with 20,160,000 tiles and then saving it to a file when it's finished? Or would the world generation itself do it chunk by chunk?

Just looking for help to find out what the best method would be to approach this as I'm not really sure.

Answer 1

When the world generates, you'll see various texts appearing while waiting. These texts aren't random, the world is actually processing what the text says.

I've seen a video that showcases how the world generation looks like behind the scenes. So that gives you a rough idea how it's creation works: (https://www.youtube.com/watch?v=bF-_AeRRbmU)

It starts out with the surface, and the variation of the dirt and stone, then it start making caves, increasing in size each time. After that you'll also see sections been converted to biomes (jungle and ice specifically), and then more detailed structures, like granite, cabins and the dungeon happens afterwards.

The world is technically generated in one go, but it keeps generating new parts over itself, and finetuning the details afterwards.

As for rendering, while the world is loaded all at once, it will only draw (or render) the blocks when you're close to it. so when you're moving fast, you sometimes see the tiles ahead of you needing some time to appear.

Answer 2

Couple of additions to Steven's informative answer, at a technical / processing level... speaking here in a general sense of the techniques used to generate large worlds (not Terraria specifically).

Generating the initial space

Firstly, there are fast block copies (memcpy / memset in C) used to allocate the initial memory (2D array) representing the world. The entire initial world can consist of zeroes, or structures all of whose members are set to zero. This could represent either sky or ground. Let's assume the whole maps start as sky, so zeroes represent empty space. This is a very fast operation. If the entire 20 million tile map uses 4 bytes per tile, that is 80MB - large, yes - but not ridiculously so by any modern measure. Expect milliseconds or less to allocate such a large space to main memory (RAM).

Terrain can be rapidly written (& read) in a couple of ways.

• memset, or series of memsets on a 2D array, most of which resides in main memory (RAM) + regular array access to read; you could even chunk this into smaller arrays (but many of them) to operate on smaller areas. Still, this is not particularly efficient; see below.
• instead, the whole world may be represented as RLE columns, running from top to bottom. This makes access much faster, since an entire column of 2400 high can be represented as nothing but a min value (2 bytes), max value (2 bytes), and the tile type (typically 1 byte) = 5 bytes total. Less data = less time to access it. If you have earth (basic stone) and sky in the same column, that's 2 such entries = 10 bytes. For the whole map then, before you have started making elaborate details, that's 10 bytes * 8400 = 84000 bytes, or about 84kB. That's miniscule, and fits into CPU L1 cache on pretty much all systems, making access super fast. Now let's imagine you have at least 50 different terrain variations (types) per column, then that is 50 * 84kB = +- 4MB. Still very small.

Data access speeds improve enormously when a dataset becomes small enough to fit into CPU L1 & L2 cache, as in RLE, rather than our RAM-sized (80MB) 2D array representation.

Generating detail

We could read from our RLE-encoded world map and temporarily copy areas into a small 2D array (for example 100x100) to create a pyramid, a small dungeon, a village etc. Once work is done, we copy these back into the RLE structure using an optimal function we've built for that purpose. What this means is that 99%+ of the entire world space is ignored as we elaborate in just that small area. As the world is gradually generated, we probably touch less than 20% of the entire world space.

The general principle in such games / worlds is...

"Don't explicitly specify (in memory) what can be safely assumed."

This is the principle which RLE (and other spatial subdivision / compression techniques) rely on.

Answer 3

@Steven and @Engineer both provided fantastic information, but I'd like to add one additional note regarding world generation commonly used for games like Terraria and Minecraft (and many others) that increase the performance of these things - noise.

In many cases, a world seed is provided that is used for noise functions. Those noise functions can provide information in layers for things like biomes, cave systems, locations of objectives or special objects, and really pretty much anything. The beauty of that is that the entire world, even if "generated" in one go, doesn't need to be held entirely in memory during or after the generation process because the game can calculate what is at a specific coordinate (whether that's a single tile, or a chunk) simply by running the algorithm.

In the case of worlds like Terraria, it might make alterations to the base noise function to make certain things "work" or "feel right". Those alterations can be saved per-chunk on another layer of data that overrides the default noise generated terrain - similar to how a game would save any changes made by the player, directly to each chunk of data.

If designed well, the initial generation, including alterations, could be done per-chunk across multiple threads, with each thread processing one chunk at a time. I suspect Oxygen Not Included does something similar with its world generation, though I cannot say for sure.

If you're using Unity (or don't mind using Unity as a learning tool), there's an absolutely fantastic course on Udemy that guides you through creating a Minecraft clone and is pretty fantastic about showing how to work with chunks and using noise to manufacture a world. While it's geared toward a Minecraft-like world that is more or less infinite (and therefore generates chunks only as needed), the same principles can be used for a finite world space. Really, any courses from Penny de Byl are worth the time.