Rectangular selection on isometric map

Question

Before we get started I checked the following questions:
Isometric Tile Selection with rectangular bounding boxes
Isometric rendering and picking?
Calculating the number of tiles shown on an isometric map <- this is exactly how I get my top-left and bottom-right tile, but I can't calculate what's between them I can pick tiles perfectly, I'm struggling with the automatic selection given two tiles (top-left and bottom-right in screen coordinates).

I've attached a drawing to help you visualize:

As you can see it's a Diamond setup, not staggered isometric. I'm struggling with finding the orange tiles, I have the green tiles' coordinates.

Btw, it's not true isometric, it's 2:1 "isometric".

Answer 1

Let's define the row and column a particular diamond sits in:

// Increases top to bottom down the screen.
int Row(Vector2Int tile) { return tile.x + tile.y; )

// Increases left to right across the screen.
int Column(Vector2Int tile) { return tile.x - tile.y; }

Note that even rows contain diamonds on even columns, and odd rows contain diamonds on odd columns.

From this it follows that we can convert from rows and columns back to x & y:

Vector2Int TileSiteAt(int row, int column) {
    Assert(((row ^ column) & 1) == 0, "Both row & column must have the same parity");

    // row + column = (x + y) + (x - y) = 2 * x
    // (row + column) / 2 = x
    int x = (row + column) >> 1;

    // row - column = (x + y) - (x - y) = 2 * y
    // (row - column) / 2 = y
    int y = (row - column) >> 1;

    return new Vector2Int(x, y);
}

With this in hand, we can iterate over our rectangle from the least row to the greatest row, touching every second column in each row between our least column and our greatest column:

IEnumerator<Vector2Int> TileSitesInRectangle(Vector2Int start, Vector2Int end) {

    int firstRow = Min(Row(start), Row(end));
    int lastRow = Max(Row(start), Row(end));

    int firstColumn = Min(Column(start), Column(end));
    int lastColumn = Max(Column(start), Column(end));

    for(int row = firstRow; row <= lastRow; row++) {
        // If the row is even and our first column is odd, pick the next even column.
        // Or if the row is odd and our first column is even, pick the next odd column.
        int shift = ((row ^ firstColumn) & 1);
        for(int column = firstColumn + shift; column <= lastColumn; column += 2) {
            yield return TileSiteAt(row, column);
        }
    }
}  

---

Lua 5.3 code

function Row(tile) return tile.x + tile.y end

function Column(tile) return tile.x - tile.y end

function TileSiteAt(row, column)
    local x = (row + column) >> 1
    local y = (row - column) >> 1
    return {x=x, y=y}
end

function TileSitesInRectangle(tile_start, tile_end)
    local firstRow = math.min(Row(tile_start), Row(tile_end))
    local lastRow = math.max(Row(tile_start), Row(tile_end))

    local firstColumn = math.min(Column(tile_start), Column(tile_end))
    local lastColumn = math.max(Column(tile_start), Column(tile_end))
    
    local result = {}
    for row = firstRow, lastRow do
        local shift = ((row ~ firstColumn) & 1)
        for column = firstColumn + shift, lastColumn, 2 do
            result[#result + 1] = TileSiteAt(row, column)
        end
    end
    return result
end

local res = TileSitesInRectangle({x=2, y=5}, {x=8, y=5})