The main bottleneck is storing and generating chunks on the server.
Around 10 Chunks in each direction are loaded for every player, that means that if a lot of players are in one spot, it's only going to load the chunks once, but if all players spread out, it's going to load these 10 chunks in each direction for every player. It gets even worse if the players are generating new chunks.
We've never had problems with our bandwidth, but I believe that Minecraft is really lavish with it's network resources, so that could be a potential bottleneck as well.
I've been running my own Server for quite a while now (since the first SMP Alpha) and we've upgraded our Server several times too. For us, the main bottleneck was our HDD, the server had huge lags when people generated new terrain. (running around 32 Players at peak times)
To answer your questions:
- Mostly on the server. The client will leave already loaded chunks loaded as long as enough memory is available. The server however will unload every chunk that is not within a 10 chunk radius of the player.
- All players on one spot won't need as much performance as when everyone spreads out or generates chunks. Different biomes seem to have different effects too, I've noticed that our server really struggled with generating Snow and Nether Biomes.
Also, you're able to modify the viewdistance in the server.properties. It's set to 10 chunks per default IIRC.