Boss battles, both metaphorical and literal, serve a number of important purposes. Not every boss has to do all of these, and even two bosses in the same game may have different purposes. There may be accidents where what is supposed to be a climactic boss battle is quickly forgotten as too easy, with no real story ties. Some may not fit totally into any category, but instead may be a foggy mix.
Story Climax
He's killed your family, along with countless others. He's taken power of the country for his own selfish purposes, and plans on summoning an unstoppable demon. Worse, he's been mocking you the entire time you've been chasing him. But now he's standing directly across from you, sword in hand, smug grin on his face. Now you can end it...
Or, maybe you don't want to fight them. In fact, there may not be anything fundamentally wrong about their position. But for one reason or another, you clash, and regardless of who wins, no one is happy about it.
In fact, you may not even be fighting another individual...
If the battle exists only for the character, and not the player, it is not counted as a boss battle, as the player is not being tested even slightly.
Examples: Kefka from Final Fantasy VI, Gunther Hermann from Deus Ex, the Princess level from Braid.
Aside: Not Story-Related
Not all boss battles need to be story-related. Some games have no story, so such a battle is obviously not story-related. But even when a story exists, some bosses are just irrelevant to it, and simply happen.
They can be a welcome surprise, or blatantly telegraphed from a mile away. In any case, they can perform any function any other boss battle can.
Examples: Master Hand from Super Smash Bros., secret super-bosses in the Final Fantasy series, challenge dungeons in the Zelda series.
Gameplay Climax/Test
All those skills you've gathered and refined over your journey? Let's see just how well you know how to use them. You are presented with a challenge, more difficult than before (probably), and/or uses all the elements you have learned throughout the journey together in a way you may or may not have even seen before.
Perhaps, it's just a bigger and tougher enemy with more health. Not exactly that satisfying for some gamers, but others will love it as a resource management challenge.
Perhaps it's a situation you have come across before, but in a new and dangerous setting, such as over lava, in a strange gravity field, or under a time-limit.
Examples: GLaDOS from Portal, the last level in Mirror's Edge, almost every second encounter in Bayonetta, final levels in the Advance Wars series.
Puzzle Battle/Out-Of-Nowhere Gameplay
Ok, so, we know it's been a sidescrolling beat-em-up this whole time, but we've decided to give you a gun, and turn it into a third-person space shooter, just for the last boss. Or, maybe we've simply introduced a new game mechanic, and want to put you under pressure to learn it quickly. In it's simplest form, everything is as normal, gameplay-wise, but the enemy has some trick to them that you have to discover and exploit. In any case, you probably didn't have time to prepare.
Why is this useful, and not a dick move? Honestly, it sometimes is a bit of a dick move. But any other kind of boss battle can be that, too. It can be used to great effect if done well. If you want to put the player under a lot of pressure, make them have to learn something quickly to survive.
Examples: 02 from Kirby 64, Ozzie from Chrono Trigger, the Spider from Limbo.
Aside: Metaphor
Not all boss battles have to be a battle, or even against an intelligent individual. In fact, even a single decision, made difficult enough, can be considered a boss battle, if you want to stretch the definition to breaking-point. As long as it can be a story climax, gameplay test, or have some kind of intensity that separates it from the rest of the game, it can be considered a boss battle. Well, in game design terms, at least.
Examples: Manfred von Karma from Phoenix Wright, final levels of puzzle games like Super Monkey Ball or of music games like Elite Beat Agents. If you want to stretch the definition, you can even count the question of which of your companions do you save when you can only save one in Mass Effect as a boss battle. As long as you actually care about them both, that is.
No Boss Battle
Some games just don't need boss battles, or would actively be made worse by them. They might break the flow, or perhaps the game itself is already an increasingly difficult test. Maybe they just don't make sense.
Examples: Tetris, Animal Crossing, Mario Kart.
Conclusion
I'd say the most important aspect of a "boss battle" is that it feels like a separate part of the game that has more tension and difficulty, in story and/or gameplay, than the rest. It is a climax, or otherwise a rise in intensity, that tests the player, and sometimes also the character. It is different from a purely story climax, because the player must be tested in one form or another. They're not used, or even useful, in every game. As with anything else in game design, use it if it makes sense to use it, otherwise don't.