Java: AI to make a mob face the player working half-way correctly

Question

While creating some basic AI for my "ghost" mob that is just up for testing at the moment, I thought of allowing it to face the player, depending on the player location. I get it to work correctly, but it doesn't act exactly as I want it to. Here is the current code I use for when getting the ghost to face the location of the player:

if (player.getX() < this.getX())
    dir = 3;
else if (player.getX() > this.getX())
    dir = 1;
else if (player.getY() < this.getY())
    dir = 0;
else if (player.getY() > this.getY())
    dir = 2;

It will switch viewing position as I want along the x-axis, but along the y-axis I have to be exactly lined up for it to face me as so (the sprite below the first one is the player, I am just using the same sprite for the ghost at the moment):

As long as I am down there I would like it to face that location, but if I adjust one pixel this happens:

For the mapping, I want it to act similar to this, positions will change within a certain range is what I really want though, so if I were to move to the left side of the ghost, it would switch to that view, but if I were to stay at the bottom, I want it to stay facing that way:

My problem is that I want the ghost to face the player depending on the location, and I just can only get it to face the way I want it to on the x-axis, and not how I want on the y-axis, I have to be exactly lined up by the pixel for the y-axis for the ghost to face the player vertically. How would I accomplish this?

Answer 1

You need to figure out if the distance on the x-axis is larger or smaller than the distance on the y-axis, and then make the sprite look along the axis which is longer.

double dist_x = player.getX() - this.getX();
double dist_y = player.getY() - this.getY();

if (Math.abs(dist_x) > Math.abs(dist_y)) {
    if (dist_x > 0) {
        dir = 1;
    } else {
        dir = 3;
    }
} else {
    if (dist_y > 0) {
        dir = 0;
    } else {
        dir = 4;
    }
}