How to detect and prevent abuse (botting) of online game API?

Question

I have been ocassionally working on a game idea in my free time. The gameplay and content renders it to be implemented as a online multiplayer game built with well established web technologies. You should know that it falls into the strategy and simulation genre. That means: No running around with characters or similar but only atomic actions (regarding client-server communication) like "build thing A on location X".

At one point I realized there is a huge problem: having a browser based front end relying on a REST API back end makes it a more than excellent target for bots. While automation is desireable for business it is poison to a game which is about fun. I experienced it first hand in a browser game in the past were the most successful players were bots which subdued everybody.

From my current point of view I do not see any possibility to protect against bots when building a multiplayer online game based on a REST API. Exception: Making it open source so everybody can host his own instance for private groups or even just oneself (to not get annoyed by jerks with bots).

Is there any way to differenciate between an honest player who just set an alarm for the next possible action and a bot automatically taking every chance on appearance of it? Besides such show stoppers like captchas. If not, I would think about another technology stack which makes it at least much harder to mess with client-server communication (proprietary encrypted binary protocol in a native code client).

Edit: Thank you, your answers are inspiring but also made me realize that specific countermeasures cannot be made by going further into details of the game. However, that would be too much for a question on Stack Exchange. So I just want to point out the most important points:

• Players are exploring, developing and managing whenever they want to. Their businesses still run fine when they are offline (it is a peaceful game, no weapons involved). Only expansion and progress need the players action. Those actions are limited by:
• time is one of the key resources (like in EVE Online skill training). All processes in game require it. There is no point in being online 24/7. The average player should be successful already with spending not more than an hour every day in one or two sessions (roughly, the concept is still in development).

Answer 1

If people want to bot, I don't think you can really stop them.

You can of course implement many measures that make botting more or less of a pain. But you can only do so much before your codebase turns into a gigantic mess that's hell to maintain, error prone, and annoys legitimate users. Meanwhile the botters will always find a way to defeat your countermeasures:

• There's more of them than you
• They have more free time on their hands (you have to split time between actual development and bot proofing, they can hack their bot code all day long)
• As you create outlandish, challenging safeguards the bot authors will be further encouraged because it's fun to break your bot protection
• If there is a black market of people commercially profiting from bots, the harder you make it to write a bot the more valuable working bots become, so you incentivize botting

You will basically be trapped in an arms race with the botters, and based on logic along the lines of the above points, as well as my experience with such games, you will not be able to keep up.

Some game developers employ extremely aggressive anti-cheating measures: For instance, Steam will scan memory and the file system to look for hacks, and offenders can be punished by losing accounts worth hundreds of dollars. Yet there are still bots and other hacks for Steam games, and some of them even work half the time. Unlike them, you have an API that's wide open to the user, and no control over the users computer. It's an uphill battle from the start.

The problem you are trying to solve is essentially a Turing test: Except it is a very easy Turing test, because you cannot cheat by requiring hard AI problems like language. No matter how many heuristics you create, it would be trivial for a botter to add a little randomness to the bot's action to have it mimic almost exactly a human. It wouldn't even be very hard to have the bot watch you play for a bit, and learn how to time actions exactly like you. Then when you ban the bot, the botter posts a huge rant on the forum about how he's just a dedicated player (and maybe he is actually a false positive) and your core audience of hardcore players will rise up in arms against you.

Make the game too fun to bot

The main reason someone uses a bot is because there is a part of the game they want to skip. If they enjoyed the game and found it fun, they wouldn't have the bot play it, they'd play it themselves.

But if the game is so boring, why play it at all? Presumably, some parts of the game are boring and mandatory to get to parts that are fun. For instance, in MMOs everyone loves going up a level, but nobody wants to kill 42,324 dire undead poison rats to get the XP. So they let the bot grind and drop in to play the fun part.

This is not a criticism of you or your game, but clearly at least some players find some parts of your game tedious. You should see if you can reduce these tedious parts, and look into adding more difficult, meaningful decisions: Bots aren't good at deep strategy or lateral thinking compared to human intelligence, and besides humans enjoy making difficult game decisions.

From your description, I get the impression that this is a browser game similar to Travian, where there is a build queue with a single best build order and certain "maintenance" (such as keeping farming raids going) tasks that must be done. You say there is no conflict, but in any MMO drama and petty politics is inevitable (IMO it's the main attraction) so I'm sure your players find ways to butt heads. With these sorts of games, a lot of the tedium comes from these "maintenance" tasks - what players really want to do is make alliances and play the diplomacy game with rival clans, the maintenance then becomes a sort of tax where you have to wake up to an alarm at odd hours to be allowed to get into that fun diplomatic part. So cut out the tedium: Automate boring things yourself (but maybe imperfectly to keep it interesting), so that players can focus on the parts they like.

This approach may not always work, unfortunately. Not all players have the same tolerance to tedium or the same concept of fun. You could have a prominent mechanic that 99% of your players enjoy, but the 1% find boring. What if the 1% then start writing bots, ruining the fun for the 99%? But ultimately, it is a matter of degree. You can never remove botting completely, but you can minimize the damage.

Undercut botters

A lot of the really negative effects of bots come from bot authors commercializing their work. If this is the case for you, you could simply compete with the bots. Many real-time based online games already have premium features that allow time-skipping and automation (such as extended build queues). These amount to a developer-sanctioned official bot. If you have these, and price them appropriately, players will buy your premium instead of buying bots. The good news is, you are in charge of the API, so you always have a strong advantage at developing quality automation for your own game, so this time it's a losing battle for botters.

This will not eliminate amateur botters, or people who feel that your premium is not good value for the money, so again the effectiveness of this approach depends on the situation.

Manually look for them

As I said above, what you are doing is essentially a Turing test. Since interaction with humans is famously considered a difficult Turing test challenge, you can try to leverage that.

Manually inspect top-level players and see if you find anything suspicious. You might even be able to get away with occasionally probe them in ways restricted only to your imagination, to see if you can trick the bot into doing something it wouldn't do.

While it is hard to write an algorithm that will detect bots reliably, it isn't so hard for a human to learn how to spot them. I think a lot of browser games use this strategy, and it can be pretty effective. The disadvantage is that either you need to do a lot of boring work all the time, or you need to pay game masters to do regular bot patrol.

Answer 2

The best and only effective defense against bots is to design your game in a way that players don't feel the need to automatize in the first place. When your players automatize simple tasks which do not actually require skill, it is a sign that your user interface is lacking and they are substituting an UI feature they are missing.

• Does your game include repetitive tasks where the player performs the same action over and over again to grind?

  Make these aspects of your game less repetitive and more interesting to play and re-play.

• Does your game require players to perform actions at times inconvenient for them, like in the middle of the night or when they are at work?

  Allow them to queue orders (like "build X1, then X2, then X3, starting each as soon as the resources are available") or schedule actions at specific times in advance (build X at 3:22 AM).

• Does your game require players to act on events within a very short timespan?

  Allow them to configure the game in a way that these actions are triggered automatically

  • when building X gets destroyed, rebuild it immediately.
  • When someone offers resource X for less than Y money, immediately buy up to Z units.
  • Auto-sell all resource X when stock exceeds Y units, but not for less than Z money per unit.

Monitor your community. When you notice that they automate anything else, steal the idea and add the option to automatize it to the core game.

Answer 3

Don't make your game so vulnerable to johnny-on-the-spot effort

First, make sure that players who only play your game for twenty minutes or an hour a day in a single sitting aren't at a huge disadvantage to players who leave it open at work and play 16 hours a day.

This may require a change in your game mechanics - for instance a move allotment that fills up when you aren't playing and allows players to execute many quick moves when they can play, rather than a mechanism that requires constant tending.

This will disincentivize bot makers because they can keep up with the game, and they have no pressing need to make an automaton that performs actions for them.

Require server info for each action, and make the server responses slower

When a player plays the game, they request a page which present them with the UI they interact with to make their move. You can insert elements, such as invisible form elements, that verify they actually requested the page first. A random number which your server stores along with the user's ID and what page they last requested. When an action comes in, the server verifies that this user has returned the same random number, and is performing an action on the page they were last seen on. This not only means the bot has to make two API calls for every action, but that regular users can't open multiple pages and execute sequential fast actions (if this is a problem with your design).

The two API calls gives you a way to slow bots down later on. Eventually you'll see suspicious patterns, and you'll be able to pick up on them. When your server detects a suspicious pattern, it can delay the API response in the first call by a second - which will annoy users, but will really slow down bots. Further, if the second request comes back "too quickly" (whatever that means for your game) then you can reject it, or reload the page with some in-game error or reason for requesting a re-submission. "You can't build that quickly. Please take your time." for instance.

Listen to your users and you'll find what they are ok with, and what annoys them.

Make the moves and actions require thought

IF the game is a simple button clicker, then you can't do much about bots. Consider re-designing your game so that the game itself performs a captcha-like test.

Not knowing anything about the game, I can't suggest much. If it involves choosing where to place a building, design the interface so the user can choose anywhere, even though there are obvious placements that wouldn't work. Or perhaps the user has to match the orientation up correctly for the placement to be successful. If it involves choosing an action, include a longer list of actions, some of which would make no sense in a given situation.

Include random very simple bot checks

Add an occasional dialog, "Are you sure?" for instance. Change the question and wording frequently and use it as a spot check for bots.

Change the API

Change the API frequently, forcing them to actually parse the webpage for the correct variables and how they are returned to the webserver. If you develop your interface using a template, replace the template with code that uses a salted hash or random variable generator for each variable name. Then start using javascript to encode the responses, and change that encoding frequently. You can probably write software that does all this for you, and either have everything random all the time, or just do a static template that you change occasionally.

Answer 4

You can't stop them. But you can make their lifes miserable, as they have to spend lots of time writing their bots, and updating them. You have to use whatever you have to verify if user is valid.

• Check for request headers, and reject requests with invalid values. Either set custom headeror check for existing like user-agent. Sure it's easy to overcome, but it's even easier to check. If botwriter is unexperienced it might take him a while to think about it! Your task is to force as many of them to give up right?
• You probably have some sort of hash signature or something in your requests, which is generated on client side with some function. Well, then keep changing it! Make it as separate non-cacheable script, and change it at random intervals. Make few of thems, or edit on fly, by adding random salt to hashing process. While it's again, easy to overcome, this will force them to keep eye on your function. They might also need some sort of constant deployment pipeline to keep users updated. Else, their bots will start sending incorrectly hashed requests. You can even ban users who are doing it too often. Valid users will move around your client anyway, so they will refetch hashing function every now and then. Just remember to update server side too (script based backend will really help here).
• Keep track of resource usage efficiency. If they are spend instantly, after something become affordable, or if new raids are always send within 1-2 seconds from last one, 24/7, then you are dealing with some hardcore player or a bot. You request captcha from such person, and force logout him. If he keeps making requests after being logged out (poorly written bots!) then it's a bot. If he attempts to login multiple time without success, then he's probably a bot. If it takes him few hours to relog, then he might be a bot (bot might request human assistance at this point, and owner might be sleeping).
• IP. This one might eliminate bot-selling. If someone writes a bot with $$$ in mind, they might not want to share source. They will sell bot time instead. If they are greedy, they might use single machine, with single/few ips. That means alot of users will start sharing IP. This is risky, because the same case is for people who are sharing a network. You will have to check if any of previous checks is successfull on certain IP, and decide manually if it's legit user group, or an army of bots.

Answer 5

In general, distinguishing between bots and humans fully automatically is hard, some form of human-assisted decision process works best.

What I would do: define some heuristics that hint the user is probably a bot - doing a lot of actions, doing stuff 24/7, ... Then if these heuristics get over a certain threshold, do an invasive check.

You can manually scan the players activity and see if it seems OK. Or you do a captcha the next time the user is online. Or (even stronger) just send him a chat message where you explain that you suspect he is a bot and want him to respond with something human-like to prove he is not a bot (how was your day? What is your favorite movie? ... - you can have a bunch of those). If he does not respond at all, he is probably a bot. If he responds, you manually check the answers for human-likeness and ban robots. If your heuristics are reasonable, only a small minority of users will be questioned and the amount of responses to scan will be small. And if you keep the set of questions private and changing, there is no way the robots can be prepared, unless someone invents SkyNet.

Answer 6

Embrace the botter. You've built a restful API, perfect for a coder to experiment with automation of your game. Design your gameplay so that the bot doesn't gain an advantage over a human player due to being automated - eliminate the advantages of speed of execution etc that a machine has; design your game so the bot provides the same revenue as a human player, if possible - or at least provides a richer environment to attract more human players.

The question is, why are people creating bots for your game, and will they pay money for the privilege?

Answer 7

I don't neccessarily disagree with the soft-science answers but there are technical things you can do to detect botters and some things that just make life harder for them.

• Grade accounts by how much you suspect they're using a bot. This will feed into several other techniques and protect legitimate users from your wrath.

• Rotating the session cookie key. This ensures the bot preserves its cookies but also makes it much harder for a botter to share cookies between browser and bot (which is really handy while developing and testing).

• Rate-limit certain actions. Work out the quickest you could actually do something, and monitor how quickly these bots are doing it. Might also be worth keeping a running log and working out standard deviations. Even if somebody is only doing something every 10 seconds, if they're doing it exactly every 10 seconds they're probably a bot. Modify their rating and log them out.

• Make login more difficult on accounts you suspect could be a bot. CAPTCHAs and questions and other unavoidable things make things much harder for a would-be botter to automatically script things.

• Monitor mouse/page movement. This is fairly simple with javascript these days but bots won't organically have any mouse movement. If you aren't getting any between "clicks", they might be mobile or they might be a bot. Investigate. It's a lot of data to log though... So you might want to reserve this for accounts that have already triggered previous steps.

• Build tools to link accounts by behaviour, IP, action sequences, action cadence. If you're going to ban people, make sure you have the tools to manually verify their wrongdoing.

• Rotate out URL schema, CSRF variables, etc and do it often. It's a big step and probably requires more up-front work than it's ever going to be worth but at the very least, any accounts that suddenly just stop working (or keep hammering the old URLs and sending the old variable names - log it!) need their bot index level increasing.

Answer 8

Have you considered making bots an integral part of the game? It's hard for bots to ruin the game for everyone else if everyone is encouraged to create them. Add support for scripting and all of a sudden the dynamics of the game changes from manual resource management strategies to bot design strategies.

Answer 9

Create a separate bot only server. Create a leaderboard and celebrate the winners. Look at the data this generates. Ban users from normal severs whose behaviour profile looks like that of a bot.

Answer 10

Whatever you do, remember to NOT make it more annoying for the real player! A lot of the responses I've seen (slower page results, not allowing multiple pages open to facilitate faster input, etc) would also prevent legitimate players from doing things fast, which will just needlessly frustrate them.

imho the easiest approach may be to apply social engineering to the problem: 1) add an explicit NO-BOT clause to your TOS, and add that detecting bot activity will remove the player from the high-score list (or have the label "cheater" attached, so people aren't as frustrated with them; if you go that route, allow people to hide cheater records).

2) check for bot-like behavior (e.g. regularly timed pings to the server, or close-to-perfect timing for many hours in a row), and if so then put the account into "bot mode" - display occasional captchas and whatnot. if these are ignored but the player keeps actively playing the game, mark the player as "bot" and exclude him from the rankings. if they're addressed, mark him as "power player" and take him off the bot list.

this way the vast majority of players are never affected, the hardcore gamers are affected for a short duration, and the bots don't "ruin the game" for anyone else.