libgdx - how to make enemies not overlap

Question

I'm currently making a roguelike shooter in libgdx to get a better understanding of the library but I recently just stumped onto a problem where enemies would overlap each other while tracking the player's movements. The code I use to calculate the angle for the enemy to take is simple trigonometry. After moving the player around the screen a for a little bit, the enemies' paths intersect to the point where it just looks like a single mesh of textures following the player. Here's the code:

// track player
radians = MathUtils.atan2((player.getY() + player.getHeight() / 2) - (y + enemy.getRegionHeight() / 2),
        (player.getX() + player.getWidth() / 2) - (x + enemy.getRegionWidth() / 2));
dx = MathUtils.cos(radians) * speed;
dy = MathUtils.sin(radians) * speed;

An solution that allows the enemies to recalculate their angle would be much appreciated.

Answer 1

Two fairly straightforward approaches come to mind:

1. A simple anti-overlap mechanic.
2. A separation force which grows stronger the closer your enemies are. (This is in the "swarming behavior" or "steering behavior" space, if you want to learn more.)

Anti-overlap

The gist of this is that you test for your enemies actually overlapping, after computing movement, and you de-conflict them in some way. Two approaches you might consider:

1. Push them apart enough to prevent overlap (can cause rippling effects),
2. Give one higher priority (closest to player?) and have the other move as far as possible without colliding.

Separation Force

Your "track player" implementation above is often called a 'seek' behavior, essentially a force or vector that moves an agent (your enemies) towards a target (your player).

To avoid collision, you need to add some other force/vector.

The classic 'Boids' system may offer some intuition, here. Of the three rules in that link, the one you'd be implementing is called 'separation'. Generally, it means that when calculating the path an enemy takes, you check the locations of other nearby enemies, and then you calculate a force (a separate direction vector, in your case) that makes them move away from those enemies. That vector can usually be simply summed with the 'seek' vector.

I've had the most success when the force is inversely proportional to the distance they are apart - you only want enough separation force to keep them from actually running into each other.

This can be somewhat tricky to tune correctly, but it can also look a bit more believable. The main risk here is that you get two enemies trying to fit into a narrow space, preventing either from getting to the target.