How to efficiently store and display a tile map on the web?

Question

About

These are actually two questions in one. First of all I am looking for a way to efficiently store large amounts of tile data. The other aspect deals with querying the data set and displaying tiles. Let me give you some background first.

We are making a browser based multiplayer tycoon game using the CraftyJS library to render it to Canvas. In the background of the GUI we're running Yii Framework on PHP and it all connects to a Python random map generator and game engine.

This is how the first rough map render looks: https://i.sstatic.net/hqdLn.jpg

Storing the map data

The game world is randomly generated each time the game starts. The size is 100x100 hexagonal tiles for each player. That means that for a three player game, there are 90.000 tiles created. Currently I just create a JavaScript array from which I render the map.

This works fine for rendering, but for any kind of interaction with the map we need to store which player owns the tile, what kind of structure is build on top of it, what's it's current price and so on. At first, at least for the prototype, we wanted to use MySQL, but after some testing, it's not exactly as fast as I'd like. Maybe an object store like MongoDB would be better suited for storing tile data instead of an SQL table. Or maybe something else?

Displaying the map

Another problem I see is moving around the map. Currently I'm creating Crafty entities for each tile even if it's not in the viewport. This is slow, because even though Crafty renders only the ones in the viewport, it stores and possibly iterates through all the tiles on each render event. What I currently have is a drawn generated map which is very slow to load and stutters when you move around, now I'd like to make it playable.

My first idea was to load the displayed subset of tiles that are in the viewport. But when a player would move the viewport to a blank area, I'd need to query the server and wait for the response back, only then can the map be rendered. This would be fine in a native application, but it's laggy in a web game.

The way to get smooth performance from the map could be preloading a bigger subset of tiles into a javascript array and use it as a cache. The player would have a few screens "cached" and when he moves the viewport, I'd load more tiles to the JS "cache".

Am I headed in the right direction? I would love to get some more information from someone who has done something similar. I am new to game development, but have been going through a lot of sources over the last couple of weeks.

Answer 1

Gameplay
First of all I'd like to ask you, do you actually need 10000 tiles per player? While I don't know what kind of game you are making it is generally true that large maps make long games. The largest map in Civilization 5 is 10240 tiles, and that only sorta works because you don't need to play on more than a fraction of it.

Database
You should not try to run a game like this off a database, you need to keep the data in application memory. You could use a database to back up the game. For a complete backup save a serialization of the game data, and then you can increment it by saving the orders given and then rerun those if the backup needs to be used.

JavaScript storage
As for the client, I'd say you better keep the entire map loaded, at least if you stick to the gazillion tiles having it all in a nice object tree might be a bit too much, so you should probably keep the data in a "semi-binary" encoded format. Strings work excellent for this kind of stuff, alternately, you can safely store unsigned integers up to 53 bits in a 64 bit float, stuff a lot of those in an array and I think you'll see a pretty modest memory footprint.

JavaScript visualization
While I won't say that you necessarily shouldn't use canvas, you actually don't need it for stuff like this. Set it all up as a bunch of img elements, and change the src property to display different parts of the map.

Pro tip
By the way, sometimes it is easier to share the seed used for generating it than sharing the whole map.

Answer 2

MySQL is not slow. Most likely you are performing naïve queries or have sub-optimal indexes. NoSQL approaches like MongoDB may be faster, maybe not. Your access pattern and choice of query is what really matters here. It's possible to give advice on how to improve performance, but not without seeing what you are already doing.

The way to get smooth performance from the map could be preloading a bigger subset of tiles into a javascript array and use it as a cache.

Yes, of course. There's not much to add here - the client asks for tiles from the server, so you just have to ensure that the ones you ask for cover an area larger than the screen.

Addendum:

we're running Yii Framework on PHP and it all connects to a Python random map generator and game engine.

If you're competent with Python then I'd advise that you ditch the PHP middleman. If you run your game as a Python process then you can keep your tile data in memory much more easily and reduce the MySQL accesses significantly. It helps that Python is a much saner language than PHP, too.