I've been interested in artificial intelligence in computer games for a long time. The question that I have been asking for a long time and to which I have not yet found an answer is the following: How to write a bot that can move through an FPS game without the game offering an API for developers? Is this possible by having only the information that the player sees on his screen? What options are there to implement something like this? Must the pixel information on the screen be processed by neural networks to operate object recognition (e.g., walls, doors, adversaries)? How can a bot, e.g. understand if he is running against a wall and is hanging on the wall? The bot would have to get a feedback about the fact that he just does not move forward in the world, because he hangs on the wall. So far I have not found an answer to these questions. For hints I would be very grateful, like also scientific articles.

As a supplement: The starting point for my question should be that you have no access to the source code of the game. Also, the game has no API through which the position data, environment data, etc. can be read.

  • \$\begingroup\$ Too incomplete for an answer, but: If the bot works on the system the game runs on, it should have access to whatever data is being transmitted to the GPU. That means it can know the worldspace vertex positions of all rendered triangles, allowing it to map the entire game world. \$\endgroup\$ – Peethor Jul 1 '19 at 19:33
  • \$\begingroup\$ Are you trying to build such a bot? For which game, and to what end? Asking about the end goal you're trying to achieve can help focus answers on the aspects that are most relevant to what you need. \$\endgroup\$ – DMGregory Jul 1 '19 at 19:36
  • \$\begingroup\$ @Peethor Thanks for your comment. Can you give a little more detail to your statement? I think that this data is generally not easy to obtain, since the game developers are trying hard to encrypt the whole !? \$\endgroup\$ – Lukas Nothhelfer Jul 1 '19 at 19:37
  • \$\begingroup\$ @DMGregoryIt is not a specific game at the moment. I've recently seen first-person shooters like Apex Legends and CS: GO. But also third-person shooters, such as Fortnite Battle Royale by Epic Games. I'm just interested in whether there are methodologies that can be used to write bots for such games that can behave intelligently in the world. \$\endgroup\$ – Lukas Nothhelfer Jul 1 '19 at 19:48
  • \$\begingroup\$ Cipher data that goes to the GPU? not imposible... yet, that is dumb. It implies that your shader code has to decipher. That will kill performance like hell. There could be ways to inspect the process memory to find objects, if whatever anti-cheat solution they have lets you. Besides... can you record the screen? Currently there are aimbots that use computer vision on screen captures, and computer vision solutions are more accesible than ever (OpenCV, OpenPose, TensorFlow, etc...). Go learn computer vision. \$\endgroup\$ – Theraot Jul 1 '19 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.