0
\$\begingroup\$

I'm trying to recreate the grid system from RimWorld, and I'm facing an issue when having to save the map, which can be up to 500x500 tiles in size.

My map has multiple layers, the base layer is procedurally generated from a seed, so I can just save that seed and regenerate that terrain without having to save all of the tile data.

However, on that base layer, you can place floors and other things which by themselves don't hold any extra information.

The way Rimworld does this (according to their save file and based on my understanding), is that it somehow converts all of that grid data to one giant string, which at runtime is then "de-obfuscated" and loaded back as actual floor tiles.

Here's an example:

example of grid data conversion

However, every other object which needs to hold any significant data is saved like this:

other save example

What would be a good way to save such simple data, similar to how Rimworld did it? 500x500 maps are huge on their own since every single grass already has to keep data about itself, and doing the same way of saving for things which only need e.g. location saved would create insanely large save files.

\$\endgroup\$

1 Answer 1

Reset to default
1
\$\begingroup\$

This looks like they've...

  • taken their array of tile information...

  • converted it to an array of bytes...

  • run the Deflate compression algorithm on those bytes to compress the byte stream...

    (You can do this with the DeflateStream class )

  • and converted that byte stream to a Bas64-encoded string.

    (You can do this with Convert.ToBase64String() )

Then to load the map, you can run this process in reverse, to convert the Base64 string to an array or stream of bytes, decompress the byte array with the Deflate algorithm, and use those uncompressed bytes to deserialize into your map array.

\$\endgroup\$
4
  • \$\begingroup\$ Is there any reason why I wouldn't want to do this with ALL data? Why have they kept the data for the grass uncompressed and readable? What's the advantage of doing this only for certain parts of the save data? \$\endgroup\$
    – caleidon
    Commented Jun 4, 2021 at 18:17
  • 1
    \$\begingroup\$ 1. The grid data might compress much better than the others, due to large swaths of unchanged/empty/repeated tiles. 2. The other objects might change more frequently during development / post-release patching, making them difficult to nail down to a consistent binary representation. The more verbose serialization gives more flexibility to change/add/remove/re-order fields while maintaining compatibility with past saves. \$\endgroup\$
    – DMGregory ♦
    Commented Jun 4, 2021 at 18:20
  • \$\begingroup\$ Just a side question, is there any point in converting the bytes to a Base64-encoded string? Doesn't that just add speed and size overhead? \$\endgroup\$
    – caleidon
    Commented Jun 9, 2021 at 11:41
  • \$\begingroup\$ As explained on the Wikipedia page, Base-64 is intended to carry binary data over "channels that only reliably support text content". In particular, since a Base64 stream is guaranteed to not include the characters " or <>, it will not cause parsing errors from an unlucky byte pattern in the stream matching a terminator sequence used in JSON or XML, which seems to be how your examples work. If you store the raw bytes in a binary file, using a different method to delineate where the stream begins/ends, then that might not be a concern for you. \$\endgroup\$
    – DMGregory ♦
    Commented Jun 9, 2021 at 11:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .