# Custom extensible file format for 2d tiled maps

I have implemented much of my game logic right now, but still create my maps with nasty for-loops on-the-fly to be able to work with something. Now I wanted to move on and to do some research on how to (un)serialize this data. (I do not search for a map editor - I am speaking of the map file itself)

For now I am looking for suggestions and resources, how to implement a custom file format for my maps which should provide the following functionality (based on MoSCoW method):

• Must have
• Extensibility and backward compatibility
• Handling of different layers
• Metadata on whether a tile is solid or can be passed through
• Special serialization of entities/triggers with associated properties/metadata
• Could have
• Some kind of inclusion of the tileset to prevent having scattered files/tilesets

I am developing with C++ (using SDL) and targetting only Windows. Any useful help, tips, suggestions, ... would be appreciated!

## Result of the discussion below

I have been designing my map file format the last few hours and came up with a skeleton (contains only layers for now - I'll leave the rest to everyone designing his/her own format) which I wanted to share with everyone interested - if you have the same intentions you can get some kind of inspiration. The full-size screenshot can be downloaded at Imgur.

• XML is a terrific format to start with, its extensible, backwards compatible, you can add any information and metadata you wish. There is wide support for reading, authoring and editing XML. Support for serializing to and from XML exist for just about every language. The downside is that it is fairly space inefficient, this can be improved by using a binary version or use zip compression on the file itself. None the less, its a good starting point for making your own file formats – Daniel Carlsson Sep 11 '12 at 20:47
• @DanielCarlsson: As mentioned in the accepted answer below I will stick to XML at the beginning to have something handy to debug and work with. Later on I will move on to a custom binary format. Nevertheless upvoted because XML is awesome in combination with the RapidXML libs in C++. – Christian Ivicevic Sep 12 '12 at 15:02

Personally, I'm more of a fan of binary formats with sections (like Windows PE, just much simpler). They are also easier to parse (but that's just my opinion.... I did work with XML enough to give me headaches, checking whether getElementByName has returned a single value or a list of values... ugh). So, if I were you, I'd make it something like this:

".MMF\0" // magic value at the start, null-terminated string. stands for My Map Format :)
char header_length // useful when parsing. char is a byte, of course, an unsigned one
char version // version of the map file. (you don't really need ints here, because you probably won't be needing more than 255 versions for example, but you can also use them)
char* map_name // null terminated string describing the name of the level/map
char* author_name // if you are going to have a map editor for the general public, it would be nice to credit the person who made the map
int width // it's probably wise to plan ahead and expect an int here when you're parsing the file
int height
".layer\0" // we begin another subsection
char type // type of the layer. for example, you can put 1 there if you want this to be a layer describing different tiles/block in a Terraria like game
".data\0" // yet another subsection. this will hold the data for the tiles
// in a hypothetical terraria 2d game, you would lay down tiles from
// the top-right corner (0,0) and then begin writing row after row
// write(1,0); write(2,0); write(3,0); ... then write(0,1); write(1,1);
// write(2,1); write(3,1); and so on..
char t1 // tile at (0,0). for example, value 0 is empty, or passable tile
char t2 // tile at (1,0). this might be a dirt block - value 1
char t3 // tile at (2,0). a rock, perhaps? value 3
(...)
char tn // tile at (width-1, height-1) or the bottom-left tile
".layer\0" // another layer.
char type // let this on be of value 2, and let it describe portals.
// putting portals in a game makes it instantly 20% cooler
".data\0"
char t1  // 0, no portal here at tile (0,0)
char t2  // still nothing
char t3  // nope, try again
(...)
char t47 // at some location, you made a red portal. let's put 1 here so we can read it in our engine
(...)
char t86 // looke here, another 1! you can exit here from location corresponding to t47
(...)
char t99 // value 2. hm, a green portal?
(...)
char tn  // bottom-left tile, at (width-1, height-1)
".layer\0" // another layer
char type // value 3, player&enemies spawn points
char something // you don't have to have header len fixed. you can add stuff later
// and because you were smart enough to put header length
// older versions can know where the stuff of interest lays
// i.e. version one of the parser can read only the type of layer
// in version two, you add more meta-data  and the old parser
// just skips it, and goes straight to the .data section
".data\0"
char t1  // zero
char t2  // zero
char t3  // zero
(...)
char t42 // a 1 - maybe the player spawn point. 5 tiles to the right
// there's a red portal
(...)
char t77 // a 2: some enemy spawn point
(...)
char tn  // last tile


,

• Looks cool.
• Makes you think you know something about programming, doing stuff the old fashion way.
• You can manually write your levels in a hex editor:
• Generally faster than INIs and XMLs, both from writing and reading perspective
• It's a long stream of byte data, really. No need to spend time on making it look pretty, indentation wise (like what you'd want to do with XML).
• It's easy to add stuff in the headers. If a piece of data comes at the bottom of the header, old versions of parsers can be instructed to avoid it and jump to the part of the file they understand.

• You have to take good care of data placement.
• Data fields must be ordered.
• You must know their type in parser - like I said, it's just a long stream of bytes.
• Moving data by one location (for example, you forget to write the type of the layer; the parser expects a byte there and it finds the value of '.' - that's not good) messes up the entire data array from that point onward.
• Harder to jump right into - there is no API, no function like getLayerWidth() - you have to implement all of that yourself.
• There's potentially a lot of wasted space. Take the third layer for example. It will certainly be packed with a lot of zeros. This can be circumvented though if you use some sort of compression. But again, that's messing with low-level stuff yet again...

But the best thing in this approach in my opinion is - you get to do it all by yourself. Lots of trial-and-errors, but at the end, you end up learning a lot.

• You are just saving ... let's say IDs for the tiles, but what about their metadata? How should I save whether tiles are passable or not? What about triggers and maybe even scripts/code/function calls associated with them? – Christian Ivicevic Sep 12 '12 at 8:13
• @ChristianIvicevic: There's a number of way to do do that: – Vladimir Mitrovic Sep 12 '12 at 13:07
• @ChristianIvicevic: 1. Pack metadata in a single byte. You only have eight possible tiles? Great, save the rest five bits for something else. In your case, you asked about passable tiles. You can have the first bit (0th bit) store that information. A little bit of bit manipulation :) will do the trick (codepad.org/Q6zfTV44). 2. Use layers for that. Have a layer with a unique type and make it filled with zeroes and ones, ones for passable and zeroes for impassable tiles. 3. Use more than one byte per tile. One byte for value, the other for metadata. – Vladimir Mitrovic Sep 12 '12 at 13:48
• @ChristianIvicevic: As for scripts, I presume you have implemented a script parser that works. You can add a section to your map file and put them there: pastebin.com/yUKncz19 OR You can put them in a separate file, and save a filename in ".scripts" section. You then prepend another ".layer" section describing which tiles fire which script: pastebin.com/BgPCR2xQ – Vladimir Mitrovic Sep 12 '12 at 14:12
• Quite a bunch of examples - TOP! Now I will stick to XML to create some basic levels to debug/work with and eventually I will move on to such a binary format you described for the raw mapfile holding the data and pack this file with both the tilesets (png etc.) and the scriptfile(s) into a zip to have everything structured in better way. It will be up to writing code, that actually reads such binary data - but that is another topic in my story... thanks! – Christian Ivicevic Sep 12 '12 at 15:00

You could use the TMX map format used by the Tiled editor (as well as several other map editors).

Even if you don't use Tiled yourself, the TMX format supports all the functionality you mentioned and it has several existing loaders/parsers for a variety of languages. It's also very easy to understand the format and extend it for your own game.

• I will have my current XML concept based a bit on the TMX map format and then read all files with RapidXML - later on I will move on to some custom binary file format. – Christian Ivicevic Sep 12 '12 at 15:04