Use a 2D array that links a set of coordinates to a map object. When you get down to it, the only operation you'll be performing here is "get me the map at coordinates (X,Y)" and a 2D array is in a very literal sense made for that.
Each map should also know its own coordinates so that when the player exits right, you can say "get me the map at coordinates (X+1, Y)".
To make it easy to move maps around, map objects should not know their own coordinates until the first time they're requested from that 2D world array. At that point, the 2D world array tells it its coordinates.
A space concern
Since you seem to be making a winding cave network, you may become concerned once you run into a situation like this:

Lots of wasted space in the array, right? Only a fraction of coordinates are used. At this point, you're only wasting, maybe, a few kilobytes of data on unused coordinates. At this scale you shouldn't even be remotely concerned. If, however, the space wastage somehow becomes an issue, you can potentially trim a tremendous amount of fat by splitting your world into regions:

All the black space: Some saved kilobytes.
Each region is a small world in itself. To pass between regions, a region transition would say "go to region X, coordinates (X,Y)." Since your regions have no particular arrangement in space, put them in a 1D array.