I am looking for an algorithm that can find separate rooms inside a set of walls, by determining which openings should be considered doorways between adjacent rooms. All is defined on a grid. My main problem is that I cannot be sure that both sides of the same passage will be in one straight line.

In the picture there is an example with five rooms separated with red lines and black walls, although it might be acceptable to merge the middle room with one of the others. I have the black walls as input, and I want to generate the red lines marking the doorways as the output.

worst case scenario

Basically I am trying to avoid having to mark them manually to minimize any chance of making a mistake. I made some attempts with focusing on the corners, but got stuck with the passageways not in a straight line problem.

Any ideas?

The walls are stored in a two dimensional array of (half-)bytes storing a bitmask where

  • 1 means that the wall in this cell has a connection to the cell above
  • 2 -> connection to the cell to the right
  • 4 -> connection to the cell below
  • 8 -> connection to the cell to the left
  • 0 -> cell is empty room

If the wall in a cell has multiple connections the sum of the connections is stored. For example a '3' would represent a "L" shaped wall segment, 'E'(14 in hex) would represent a "T" shape and 'F'(15) would represent a "+" shape.

The example would be stored like this:

  • 1
    \$\begingroup\$ how wide can a door be for you to be considered a door and not some "in room obstacle"? one tile? or can a door be also two tiles wide? Should doors only be horizontal, vertical or diagonal (45°), or are there other possibilities to consider (e.g) diagonal at a different angle? \$\endgroup\$
    – datacube
    Jan 5 at 12:10
  • 1
    \$\begingroup\$ @datacube Unfortunately I can not exclude doors at any angles nor how wide they can be. “indoor obstacles” I haven’t considered, it is a good point. \$\endgroup\$
    – Jamek
    Jan 5 at 15:10
  • \$\begingroup\$ If they can be any width and any angle, then every empty tile could be a door. You have to define some rules on where a door should or should not be. What for example prevents there being a door right through the middle of a room? Or an additional door connecting the horizontal wall sections in the middle? \$\endgroup\$
    – datacube
    Jan 5 at 15:23
  • \$\begingroup\$ @datacube That’s part of the problem. We can assume that a door can only be between two segments of a wall that are themselves connected to exactly one other segment and no obstacles between the two. \$\endgroup\$
    – Jamek
    Jan 5 at 15:43
  • \$\begingroup\$ If you can identify the ends of walls, perhaps connect one end to another and if no other wall ends are detected within the space then you've found a doorway. Then again, the doorway in the lower right room would end up splitting that room. I feel like there needs to be additional parameters for defining a door space. \$\endgroup\$
    – Phaelax z
    Jan 5 at 19:02

2 Answers 2


This is a variation of an algorithm I used myself in a grid based randomly generated level (a simply 0= empty 1= wall grid) to place doors.

Finding possible doorway tiles

There are some simple rules depending on the surrounding tiles to determine whether or not there can be a door.

The tile itself must be empty and have at least two neighboring wall tiles that are not connected with each other (only within the neighbors).

For this you can examine the neighboring tiles counter-clockwise. Whenever you examine a "wall" tile after examining an "empty" tile you increase a counter by 1. If that counter is exactly 2 after one complete circle (examining the first tile a second time as last tile). you have a possible doorway.

However this will produce "to many" doorways, often adjescant to each other. In your example the door at the lower edge has two possible doorway tiles. The "real" doorway you marked as red in the image and the empty tile directly above that.

Finding the right doorway tiles

Now that we have all possible doorway tiles (here X but maybe marked with -1 in the array)


To solve this problem you could create a "ranking" on how the best/worst doorway situations look like. For this ranking we can continue using negative numbers in the array, e.g. -1 being "the best" doorway and "-F" being the worst

  1. The "best" doorway tile is a tile that has exactly two adjescant wall tiles that are exactly opposite of each other and only empty tiles as neighbors otherwise.
  2. Next come those with more than two adjescant wall tiles but who still have two exactly opposing wall tiles
  3. Next those that have exactly two adjescant wall tiles which are not exactly opposite of each other
  4. And lastly all remaining doorways.

With the ranking in place we can always delete the worse of two adjescant doorway tiles or a random one in case of equal ranking.

This should leave you with nicely placed doorways.


It took a while, but I think I got some thing.

It might not work with small rooms, but with at least 3 fields in diameter should work.

  1. Assign to each field an integer value equal to its distance to the nearest wall. This will create disconnected patches of higher numbers. Each patch will be a core of a room.
  2. Expand each patch by adding to it every field adjacent to it that does not neighbors a field with an assigned number higher than its own, unless the field with a higher number already is a part o the patch.
  3. Repeat until no new fields are added to any of the patches.
  4. A few fields may still be outside of every patch. Add them to the nearest patch (several if more than one patch are at the same distance).

Curious thing happens with a long corridors between large chambers, if the is no narrowing at the end. In that case the corridor is considered a part of both chambers. Which turns out suits my purpose.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .