We're talking about an expansive game that is not divided in levels or checkpoints. A 3D adventure, for example.
Simple answer: the game is divided into chunks, while you move around chunks are constantly being loaded behind your back before you can see them and thrown away when you leave the area. To every simple answer is a complex solution that weighs innumerable trade-offs in implementation... but you get the idea.
For games where you think you can see tens or hundreds of miles away, everything out there is fake. The real far stuff is just a bitmap, the medium range is a very light mesh. The real stuff gets loaded in and placed on top of the fake stuff before you get to it.
When chunks get unloaded they save the state of any objects or puzzles in progress so that when you come back they reset themselves to where you last saw them.
All this bypasses your main question so here's the real answer: there is always a loading time when you first start up the game but either you don't recognize it because it "always takes time to start up a game" and you expect that or it's hidden behind loading cinematics or happens while you play around in the GUI.
Previous answer pretty much nails it all. Also worth mentioning is Dungeon Siege I.
Here's a paper from one of the developer which actually goes over some of the architecture needed to make it work and common pitfalls:
I'd say it's a must read if you actually want to implement something like that.