# noSQL - Is it a valid option for web based game? [closed]

Out of an opportunity and boredom, a friend and I decided to make a web based game. This is the first 'game' I will be making, since usually I program web apps in django.

I've opted to use the same framework for the game, but I am not totally sure about the data storage, since I cannot foresee future requirements and DB related issues. Lately, the noSQL movement is gaining force and I would like to explore that area.

Therefore I have the following questions:

• Would a noSQL datastorage (document-based, such as MongoDB) would be suitable for a web-based game?
• What are the issues that might arise using a conventional RDBMS, such as MySQL?
• How to insurance data consistency amongst users with MongoDB, during gameplay or at timed events, such as cron jobs?

Game overview: Typical adventure game, with the player fighting monsters and gaining items and such.

• Some data is static across all players: items, weapons, potion, maps, etc.
• Some data tied to the player: inventory, metrics (stats), progress, etc.
• A player might need to know the stats and state of another player
• Scheduled tasks will be run at certain time interval
• – Ricket Nov 8 '10 at 4:53
• An issue with some noSQL databases is that they cannot do queries unpredicted at "design" time in a fast way on huge datasets like event logs: gamedev.stackexchange.com/q/4465/450 Please keep in mind that noSQL databases require the same techniques against query injection normal than normal databases do. – Hendrik Brummermann Nov 8 '10 at 7:18
• @nhnb: One nice way of restating your point is "use a relational database for relation data, and use an object/document database for object/document data". Unfortunately I don't think most people have experience with or spend a lot of time thinking about modelling, which leads to bad designs no matter which they choice. – user744 Nov 8 '10 at 19:17
• In response to your question "Is NoSQL a valid option for web based games?" I believe the answer is yes. NoSQL databases have been used for web-based games before (such as FarmVille) and offer a lot of flexibility. The main point of contention for this question seems to be "which is better (NoSQL or SQL)?". Each system has pros and cons, but both are perfectly legitimate options. If you're looking for a detailed list of the advantages of one system over the other, you may want to pay the Programmers StackExchange a vist. :) – Ari Patrick Nov 9 '10 at 0:41

## Would a noSQL database be suitable for a web-based game?

Absolutely! Generally speaking, non-relational databases (such as MongoDB) are much better for games, as they are more flexible in how they model data, while being more performant than relational databases (such as SQL) - making them a "win-win" choice.

## What are the issues that might arise using a conventional RDBMS?

There are two major drawbacks to using relational databases:

• Lack of flexibility in the way you model data
• Performance

Lack of flexibility in the way you model data is a major drawback, because using a relational database forces you to model data to fit the needs of the database, rather than the database accommodating the needs of the model. For example, consider you're creating the model for how you'll store items in a game using a relational database and you decide on the following model:

weapon_type_id | weapon_type
1 | sharp

weapon_id | weapon| weapon_type_id | damage
1 | Short Sword | 1 | 5

potion_type_id | potion_type
1 | HP
2 | Mana

potion_id | potion| potion_type_id | modifier
1 | Healing Elixer | 1| 5
2 | Mana Drainer | 2| -5


Now consider how you'd need to modify the model to meet do the following:

• Add a new item type
• Create a weapon with multiple weapon types
• Create a potion item that can also be used as a weapon

Certainly all these actions are doable, but you won't be the happiest person while you're doing them, and you'll probably wonder, "is there a better solution out there for what I want to do?," when you could design a much more flexible model in a non-relational database.

Note: If you are unfamiliar with how document-based databases store information, please check out this link before continuing. The model presented below designed to use similar logic to what is presented in the aforementioned article.

db.itemTypes

name: Weapon
types: Sharp

name: Potion
types: HP, Mana

db.items

name: Short Sword
weapon
type: Sharp (db.itemTypes reference)
damage: 5

name: Healing Elixer
potion
type:  HP (db.itemTypes reference)
modifier: 5

name: Mana Drainer
potion
type:  Mana (db.itemTypes reference)
modifier: -5

name:  Healing Potion Sword
potion
type: HP (db.itemTypes reference)
modifier: 10
weapon
type: Sharp (db.itemTypes reference)
damage: 15


There are a lot of good articles on the performance of NoSQL vs SQL databases, but here is one which provides application examples so you can perform your own tests: MongoDB vs. SQL Server 2008 Performance Showdown

## How do I insure data consistency amongst users with MongoDB?

MongoDB supports read/write atomic operations on single data entities (documents), so query results from MongoDB should always be accurate with what is stored within the database.

Edit: Provided NoSQL example of item database

• In the example that you wrote using sql, how would you, at a high level, model it using nosql? – Pran Nov 8 '10 at 17:17
• -1, your answer shows only static data. The whole debate over SQL vs. NoSQL for game databases involves highly dynamic data. The best thing to do would be to show a NoSQL transaction that trades something between two players - a very common occurrence, and one most games get wrong regardless of which type they choose. – user744 Nov 8 '10 at 19:13
• @Ari: There's a lot more static data, but no one cares about the throughput for writes to it, because no one writes to it - no writes, no transactions, no ACID, no real database question, even if you happen to store it in one. That question is then easy to answer - just keep it in memory. Certainly "How can we load static data faster?" is an interesting question, but I don't think it's at all related to questions of SQL vs. NoSQL. -- As for multi-document operations, does this mean your database would have one document with e.g. all players in it? – user744 Nov 8 '10 at 21:08
• @Ari: Please don't confuse NoSQL, which is an idea, and MongoDB, which is a particular database. At my last job we shipped two games on a NoSQL database and got excellent concurrency with low-latency. But our NoSQL database also supported multi-document transactions with field-level locking. NoSQL is not a single "thing" like SQL, it just refers to any database not using SQL (and usually not using relational schemes). – user744 Nov 9 '10 at 9:47
• Just my $.02, but "not performant" is not a drawback of RDBMS. It's a drawback of storing non-relational data in an RDBMS, not tuning your RDBMS, not understanding what your SQL is doing, not indexing, etc. etc. etc. If you have relational data, you'll find RDBMSes are incredibly optimized. – Robbie Jul 15 '11 at 20:57 Couple of words do defend SQL databases. 1 - If you have SQL database you can work with your data not only by primary key. Most of queries in MMO goes by PK but when you need to find all users with level > 30 what will you do in NoSQL world? 2 - If you have SQL language you may create "hotfixes" to repair broken data. For example: "update player_items set flag=0 where item_type_id = 100". How can you fix this in NoSQL? I think you will have to make a script which will parse all your data... 3 - SQL databases are not so slow as you can think. My collegues create the web based MMO game, wich make 5000 transaction per second in MySQL. Is it not enough for you? If not, you may use sharding to scale your database. 4 - SQL has a foreign key constraints and such good stuff, which will help you to find bugs in your code. NoSQL is not a silver bullet. It solves only couple of problems and brings many new problems. So my advice - think twice before choose NoSQL. • These are all great reasons to avoid NoSQL until you actually have a need for it. Especially #3. Unless you have zillions of users already (and refuse to shard anything), you'll probably be much more productive with MySQL. – ojrac Nov 8 '10 at 16:57 • 1. You'd use: "for x in db.user.find({"level": {"$gt" : 30}})" 2. Technically, what you wrote is a script as well, so I'm not quite sure what your point is. 3. It is VERY possible to make an MMO using SQL, and in fact, many MMOs have been developed using the technology. NoSQL technology is fairly new and has already been adopted by services like Amazon, Google, and Facebook for it's performance and flexibility. 4. In NoSQL you would need to write your own constraints, but that's simply the cost of added flexibility. – Ari Patrick Nov 8 '10 at 17:10
• I agree with your statement of thinking twice. That's partially what I am doing here with this question :) – Pran Nov 8 '10 at 17:10
• SQL is not a sliver bullet. NoSQL is not a silver bullet. Think twice before using any technology. – stonemetal Nov 8 '10 at 17:54
• Regarding 3, the problem is not so much that SQL is slow, but that SQL is laggy. Generally speaking, SQL databases get excellent throughput at the expense of latency. For a web-based game, that's probably what you want! For a real-time networked game, it's probably not, and most "WoW-like" MMOs that use SQL use a caching layer in front of it that removes most of the benefits of SQL, which is why NoSQL is a big step up for them. – user744 Nov 8 '10 at 19:11

The answer is yes, it's a valid option but no, it probably isn't a great idea.

There are two major problems with SQL that NoSQL tries to address, when it comes to games.

The first is latency, or lag. In one sense, SQL databases are blazingly fast, processing thousands of transactions or more in tenths of a second. In another sense, they're incredibly slow, because processing just a few transactions can take that same amount of time. For most database uses, this doesn't matter, but games have a real-time component so it's not just about how many transactions you can do per second, but about the average time it takes to respond to a database transaction.

This latency is a major problem for real-time games. Many developers that need large databases (usually for MMOs) build out a caching layer that sits between the database and the game servers in order to avoid these stalls, and then flush the cache periodically. The problem? You just threw out the main reasons to use a database - atomicity, consistency, isolation, and durability.

But that problem doesn't apply to web games. You've already got a high-latency connection between the player and the game, so you might as well get the throughput SQL provides, as well as the benefits of mature, well-known software.

The second problem, however, does apply to you, and that's the object-relational impedance mismatch. Games deal with objects, databases deal with rows. If you're lucky an object can be made into a row just by listing its fields, but most of the time objects are hierarchical, and you've got to flatten them out in some way to get them into rows.

Document-based databases, like CouchDB and MongoDB, solve this problem by promoting the document to the top-level object, rather than row ("document" in this case is a synonym for "object"). But by doing this, they lose many of the benefits of SQL databases. The throughput is much lower, and the locking algorithms become much more complicated.

The good news is, there are already a lot of object-relational mapping (ORM) tools available for whatever language you're using. They let you translate between objects in memory and rows in the database fairly transparently. These tools are generally not appropriate for large-scale real-time games, because they can present the worst performance aspects of SQL and NoSQL databases. But it's fine for a web game - at worst, when you run into performance problems, you can go back to doing transactions the old-fashioned way. The latency problem doesn't hurt you, and the increased throughput helps you.

Finally, to belabor a point I've made elsewhere: This isn't about static game data. I'm not talking about your item descriptions or your weapon damage or sprites or anything here. I mean the real, mutable game data, like what's in a player's inventory or what their stats are. All you need for the static data is a data store. Maybe it turns out that the easiest thing to do is use the same database as the data store for that because you have a great ORM; maybe you'll just need the filesystem. It's two separate problems, and one answer needn't (and probably won't) fit both cases.

• +1 Thanks for the comparison of both options. Furthermore, you make a good point by saying that static data should not be in the database. – Pran Nov 9 '10 at 0:57
• +1 However I think you neglect to mention one of the main pros (well, pro or con depending on perspective) of No SQL, which is that it's schema-less, which allows for storing of highly dynamic data. I'm actually going to be using a mix of both for my current project with PostgreSQL for stuff that fits nicely into the relational model, and Mongo for semi-static stuff that doesn't. (for instance a log that can used to generate a replay, where relationally I'd have to have either a column that's stores a PK from one of many other tables, or a table with generic column names e.g. param1 param2) – Davy8 Mar 1 '11 at 13:53
• @Davy88: noSQL is not necessarily schema-less, and having a "highly dynamic" database is not really a plus. If all you have is data soup, you can't do indexing, you can't do field-level locking, and even simple queries become fraught with questions about what happens if a key doesn't exist. – user744 Mar 1 '11 at 14:31
• @user744, what do those things you mention accomplish? – expiredninja Jun 14 '14 at 1:37
• Would a noSQL datastorage (document-based, such as MongoDB) would be suitable for a web-based game?

I'd advise to give a look at couchdb

Its interface is javascript-based, so it will blend pretty well with HTML5/javascript based games.

It is also able to work 'off-line' (making a local copy of the db) and then synchronize itself with the server later on (you might use this for instanced dungeons, for example)

• What are the issues that might arise using a conventional RDBMS, such as MySQL?

The main issue would be that no-sql databases handle pretty differently from relational databases. Making the mental switch can be difficult.

For example, take a look at how "queries" are done in couchdb. Everything is done in javascript.

• How to insurance data consistency amongst users with MongoDB, during gameplay or at timed events, such as cron jobs?

The process of 'synchronizing' databases is called replication in couchdb's lingo. You will also see the term consistency a lot. There are several built-in services that deal with replication & consistency. See for example the incremental replication process.

• Yes, I've heard about couchdb too, in fact, even on the mongodb's website, they commonly do comparison to it. I think that I need to research on couchdb vs mongodb. – Pran Nov 8 '10 at 17:15

Depending on what you have in your game you might end up using both, and connecting them through the models in your game. For example player could and should be in RDB (login info) but player inventory/variable storage could go to noSQL , so your player module could login() using RDB and fetchInventory() using noSQL by the primary key.

Maps for example better to be saved in NoSQL (coordinates => array of different objects for example) i do not imagine saving what an area contains in RDB (except a serialized string which you need to fully unserialize to read some part of ? more CPU and RAM wasted !) well you could still save areas in RDB for example area "city X" contains the following cordinates/block ([3,4],[8,7] ...)

you could and probably should use both, and take a look into volatile storage like memcache which you might need or some temporary data (would be faster than using hard-disk for sure ! and saves you a lot of CPU but sure not RAM)

so in the end >

it depends on what you want to have in your game ! cheerz

• which features do you think would drag you more into NoSQL or the persistence layer as i like to think of it? – expiredninja Jun 14 '14 at 1:24
• maybe a coordinate system if ur game is a gps based, something that might need advanced search. basically if youre good with MySQL then stick to it, is it possible to use it as a nosql as well – Ronan Dejhero Jun 14 '14 at 19:32