How do you parameterize turn based AI?

Question

I would like to create/find an AI algorithm/process that can be parameterized such that a change in the parameters causes a change in the AI behavior. I am specifically looking to apply this to a turn based game environment where the player is able to perform several moves (or none at all) in a single turn. The player has full knowledge of the board and should be able to take the opponent's current positioning into account when deciding what moves to make in the current turn.

My question is how can I create this type of AI? An example of behavior that I would like it to show in various situations would be:

Turn X: Attack location A with 7 units and attack location C with 12 units
Turn Y: Do nothing
Turn Z: Invest in resource collection

Then a small change in a parameter may cause the AI to only send 6 units to location A instead of 7 in the same situation as Turn X.

My only thought of how to implement something like this would be to have the parameters act as probabilities of doing action T with magnitude R. However, I could imagine how this wouldn't facilitate long term thinking, and I also have no idea how that approach would take the opponent's positioning into account. I also suspect that this approach would cause the AI to overspend within a single turn, as in, it tries to attack 3 locations with a total of 30 units, but it only has 15.

I've spent the last few hours looking through Google Scholar and have found a few papers that say that they have used a parameterized AI approach, but they don't discuss any actual implementation details. If anyone could tell me what this concept is normally called (so I can Google and research it further) or has any reference links to code that uses an implementation like this then I would really appreciate you adding it as an answer.

Also, just so people don't waste their time answering something that isn't needed, I'm only interested in the design/structure of the described process, I am not looking for ways to get good values for the parameters.

Thanks everyone!

Answer 1

I communicated with Christoph Salge who was one of the authors of "Using Genetically Optimized Artificial Intelligence to improve Gameplaying Fun for Strategical Games" and he had the following helpful advice:

If it helps, I can tell you in a nutshell what we have told the students who have been developing the AIs back in our project.

The basic idea is to first develop a classical AI that plays the game.

This can be a rule based system, some swarm intelligence approach, etc. You saw the list in our paper, but basically anything that can play the game to some extend works.

Then you go through your code an identify anything that is a parameter. Basically anything that is a specific value that you guessed or just fixed, like thresholds, probabilities, etc. For example, lets say that your AI send agents to search for food when their health is low; how low, ... well you just guess that below 30 % would be a good value. So, 30% would be such a parameter.

I don't know what programming language you are working with, but a lot of them offer the option to define certain values as constants. C++ for example has the #define CONSTANT value precompiler command, which basically just replaces every mentioning of the word CONSTANT with value. So you could then replace 30% in the code with Constant_Search_For_Food, and define Constant_Search_For_Food as 30%.

After you identified which values you can tweak in your AI code, you can then put them into a container class which stores the genome. You then replace all the CONSTANTS with functions. So Constant_Search_For_Food will then be replaced by genome.whatIsTheValueOf(Constant_Search_For_Food), which looks up the value. This class can then also be used to map parameters with different value ranges to the range between 0 and 1. So, externally, Constant_Search_For_Food goes from 0 to 100 percent, but internally in the genome class, it is then represented as a value between 0 to 1.

After that is done, you can then apply a genetic algorithm to optimize the genome. So, your genome is an array of values between 0 to 1. You take a genome, and evaluate its fitness by having the AI play the game with this genome, and tallying the result. The fitness is the success in the game, and you can use this fitness to determine who should procreate. There are also a lot of good toolkits for that out there, so you do not necessarily need to write your own.

This should then produce AIs that are better at the game, or just better against other AIs, or as we found out, good at crashing the game, if that is rewarded.

As an afterthought: If you are going for machine learning approach, such as using a neural network, the idea is slightly different. In this case, your whole AI basically is parameter values, so you can skip the design an AI part, and immediately start with the artificial evolution, like we did in "Relevant information as a formalized approach to evaluate game mechanics". But then the technique is not explained in great detail there either.

When I get around to applying this technique to the 'Counselor AI' paradigm that Christoph mentions in section 7.2 of his paper I'll add that info to this answer.