I'd say higher numbers tend to make players happy. It's better to get $1000 instead of $10, right? Big numbers also go better together with effects. Imagine a particle explosion and sound-effects every time you achieve something in the game. That would certainly look better with an accompanying 14'500, than some low number like 34. Big numbers give the impression of something, well, big, that warrants an audiovisual firework.
Why do big numbers feel like they are more of an achievement? In our daily life, we deal a lot with low numbers. Our daily purchases are usually not in the thousands of dollars, also we don't have thousands of friends or a hundred pair of shoes etc.
Numbers from one to ten are probably most common and are very normal to us. It's also the amount we can show using just our fingers. So are numbers from 10 to 50. Higher numbers are more and more rare in our daily life and therefore become special or more valuable.
On the other hand, you shouldn't use numbers for your total score that are so high that your average person doesn't even know how to pronounce it. Anything above several million is too high and too hard to decipher. This is readable: 1'400'000, this is not: 1'400'000'000'000. This means that your score distribution should be in a way that your total score won't go higher than about 6-8 digits.
Beware though, that if you start throwing really high numbers at the user right from the start, it might feel as if there's no room for improvement. A good example are RPGs, where your hero starts really weak and deals low damage at around 1-10, but at a high level with "uber" gear, the damage goes into the thousands. That really gives an impression of progress and achievement.
That being said, it's also a matter game-type and player expectations. As Joe Wreschnig pointed out in his comment, pinball machines started to work with very high scores quite early and this has established as some sort of standard. Doing anything else might dampen the users enjoyment. Eg. "What, only 1'041 points? I got a highscore of 1'300'000 on that other machine".
When you're creating a simulation type-of game, your scores (eg. currency) should be somewhat realistic. Realistically You won't get $1000 per visitor in your theme-park (unless the value of the dollar keeps sinking, but that's another story).
A lot of answers have covered technical reasons for high scores. One that you should keep in mind is screen-size and readability. If your game runs on a mobile device, you simply don't have the space for really high numbers. Also on devices that run on a TV, you're limited by the lowest possible resolution, where readability can become an issue. I'd rather have a very well readable score of 450 than an unreadable 450'000'000.
TL;DR: When designing your scores, think about it as a reward and never forget the context and player expectations. Will 10 points feel good in the context of a pinball game? Probably not. Also try to give the player some feel of progress by increasing scores, maybe giving higher scores more oomph by adding audiovisual effects. So that it really feels rewarding getting better scores.