(This question is a bit similar to this question, but that question specifically asks how to implement a height ceiling/collision mechanically, and this is more of a design question rather than a programming one. It's also similar to this great question, but my situation is more specific to the "challenging them to get past the wall" scenario, and that question is now five years old, so hopefully this question is different enough to be allowed.)
I've been working with my dev friend on a small 2D tiled game prototype that has X, Y, Z coordinate axes, allowing players to travel in the four cardinal directions as well as "fly" upwards and downwards along the Z-axis. (Think something akin to Dwarf Fortress, with a 2D map that incorporates terrain height and the ability for winged creatures to fly on the Z-level.) The theme of the game is "breaking boundaries," which naturally opens up the door for design questions about how to creatively implement boundaries and allow the player to bend or push past them to create fun.
Obviously, when you have a system where the player can travel freely on an XYZ axis around a world of limited size, there needs to be an eventual limit on two things:
- how high the player can fly up or down in the Z direction, and
- how far they can move in the X and Y directions.
This is both to preserve computing resources and to prevent the player from quite literally getting themselves lost in the sky or the void. As the Dwarf Fortress wiki sagely notes:
The default settings produce levels with around 50 z-levels of land... Reducing the number of z-levels, especially cavern levels, can reduce processor demand and boost framerate.
However, in my situation I want the player to be fully able to try and travel past the game's "boundary limits," and to reward and challenge them for doing so. This matches the general theme of what I eventually want the game to be - a game of "breaking boundaries." I want to come up with a creative way to both justify the world's boundaries mechanically, and make hitting that limit exciting and a challenge for the player to solve, even if there really is a "cap" that they can't get past in the end.
How other games solve this problem
Most games solve the problem of "how to stop the player from wandering too far" with invisible walls - that is, placing invisible walls at the world boundaries and placing an invisible ceiling over the world at the maximum height cap, and mechanically forbidding players from going beyond those world boundaries, without much of a mechanical game explanation for it.
The "invisible wall" approach is undoubtedly the easy way to go - but hitting an invisible wall is a surefire way to annoy players and break immersion, and judging from mods existing specifically to remove them, I don't think this is the most enjoyable option. As I said, I want to make hitting the boundary fun and challenging for the player, instead of a "bummer" experience.
Subnautica, on the other hand, does the idea of world boundaries in a really creative way, and that's the kind of idea I want to emulate. Instead of outright stopping you from going beyond the world boundary of Crater's Edge, you instead get a creepy "Warning: Entering ecological dead zone" message on your PDA, followed by a close encounter with these genuinely horrifying monsters that haunt my nightmares:
You can technically defeat these monsters, but it's a crazy, terrifying challenge - and that's not the point of this encounter. The point was to make hitting the boundary exciting and memorable, and it obviously worked, since I've seen tons of screenshots and videos of cool encounters with the Ghost Leviathans on the world borders. That's exactly the kind of thing I want to emulate in my own game - a sense of excitement and danger from hitting boundaries, rather than boredom or bumming the player out.
What are some other interesting, unique mechanical ways to "make boundaries fun" and use them to challenge the player? I'm specifically looking for ideas that apply to my situation of a 2D world with XYZ axes, but any general design ideas you have for tackling the "invisible wall" problem from this angle are welcome.