LibGdx efficient data saving/loading?

Question

Currently, my LibGDX game consists of a 512 x 512 map of Tiles and entities such as players and monsters. I am wondering how to efficiently save and load the data of my levels. At the moment I am using JSON serialization for each class I want to save. I implement the Json.Serializable interface for all of these classes and write only the variables that are necessary.

So my map consists of 512 x 512 tiles, that's 260,000 tiles. Each tile on the map consists of a Tile object, which points to some final Tile object like a GRASS_TILE or a STONE_TILE. When I serialize each level tile, the final Tile that it points to is re-serialized over and over again, so if I have 100 Tiles all pointing to GRASS_TILE, the data of GRASS_TILE is written 100 times over. When I go to load/deserialize my objects, 100 GrassTile objects are created, but they are each their own object. They no longer point to the final tile object. I feel like this reading/writing files very slow.

If I were to abandon JSON serialization, to my knowledge my next best option would be saving the level data to a sql database. Unless there is a way to speed up serializing/deserializing 260,000 tiles I may have to do this. Is this a good idea? Could I really write that many tiles to the database efficiently?

To sum all this up, I am trying to save my levels using JSON serialization, but it is VERY slow. What other options do I have for saving the data of so many tiles.

I also must note that the JSON serialization is not slow on a PC, it is only VERY slow on a mobile device. Since file writing/reading is so slow on mobile devices, what can I do?

Answer 1

So in general, JSON works really well for storing parameters and settings, but for storing big blocks of data like tile maps, you'll probably want to use your own format. JSON can be really repetitive, storing 256K copies of the string "GRASS_TILE", and that could be part of what's causing the slowdown.

Using a SQL database probably isn't what you want to do either. Like JSON, it can be great for setting and parameters, if there are a ton of them like sim games, but in general, it's best non-game applications, when you need just one row of data out of many rows.

Instead of JSON or SQL, I think a good alternative would be to store just one character per tile in a separate text file. So, for example, you could decide that 1 = "GRASS_TILE", 2 = "STONE_TILE", etc. Then a small field with a rock in it could look like

1111111111
1111221111
1111221111
1111111111

512x512 would be bigger of course, but you get the idea. The big saving here is that you're not storing the whole string "GRASS_TILE" for every grass tile, so reading it into memory from disk will take less time. Then when you're loading it, you can use something like this:

FileHandle file = Gdx.files.internal("level_01.map");
String tiles = file.readString();
for(int y = 0; y < 512; y++) {
    for(int x = 0; x < 512; x++) {
        char tileType = tiles.charAt((y * 512) + x);
        if(tileType == '1') {
            addTile(x, y, "GRASS_TILE");
        }
        // etc
    }
}    

This is just one option, but the advantage of this one is that you can read and edit your tile map since it's plain text. Another option would be to separate the values by commas, and that would let you call String.split(",") which will give you an array of tile values. The most flexible option, however, is probably to store the file as binary data, and then call file.readBytes(). Each tile/byte can be any value between 0 and 255, so you can have a lot of tile types, but the disadvantage is that the map itself would be harder to create and edit without external tools, which you may have to write yourself.

And finally, you mentioned that your loader was creating many copies of the same objects. If you don't need separate copies, then you could initially load that data in an ObjectMap (different kind of map!), and then look it up when you need it. You could use something like ObjectMap<String, TileData>, and then look up with your data with dataMap.get("GRASS_TILE") as needed.

EDIT:

In response to your questions in comments, I was thinking of using the ObjectMap as a place to store your unchanging data. So say you have an entity type called "bad_guy_01". Your JSON can look like:

"entities": {
  "bad_guy_01": {
    "name": "Bad Name",
    "hitPoints": 100,
    "activeTexture": "images/bad_guy_01_a.png",
    "inactiveTexture": "images/bad_guy_01_b.png",
  },
  etc...
}

Then your JSON loader can load "entities" into a ObjectMap<String, EntityData>. Now all this gets loaded at the beginning of the game, and that's all unchanging data.

Then later, when your code decides you need another entity of type "bad_guy_01", you'll want to create a new entity with just the changeable data, like currentHitPoints, position, isActive, etc. I usually just store all the active entities in an Array<GameEntity>, or sometimes use the Stage if you're using Scene2d.

The idea is that you leave it to you logic code to match the two. So for example, assume entity.type is "bad_guy_01", then whatever code is rendering the sprites can do something like

if(entity.isActive) {
    draw(entities.get(entity.type).activeTexture, entity.position);
} else {
    draw(entities.get(entity.type).inactiveTexture, entity.position);
}

Likewise, whatever is checking if the entity is alive can check the entity's currentHitPoints vs the entity data's hitPoints. You get the idea.

You can do the same thing for the maps. So you'll need a Tile[] currentTileMap[512][512], where each Tile is something like

class Tile {
    String type;
    Entity currentEntity;
}

and the type is the lookup for the unchanging tile data, like the 1 = GRASS_TILE, etc.

For serializing and writing out the map, if you need to write out the basic map with the tile type IDs, the 1s and 2s, then that you may need to do manually or create a map editor. If you need to write out the current state of the map, like for a save file, then you can write out the Tile[512][512] currentTileMap. If you can do that through the JSON serializer, then that size should be reasonable, esp if the output can be minified.