Allright so I've been doing some searches, here what I got:
SDL: It is a standard, very mature but very old. (I got posts of late 2009 where they were still waiting for the 1.3 to come up...we are still waiting 3 years later). Also it seems abandoned and old style (it's around from the '98). And also it's made in C (for this there's no downside nor upside. Someone prefers C over C++ nowadays for some reason and C can run on C++ but not viceversa).
SFML: It's a new born in the libraries. It's not very mature and lack of a lot of documentation. It was buggy till 1.6 but now the 2.0 is quitely good (it's the one I've tried). It has a lot of things built in (even the possibility to easily use OpenGL). It's in C++ and has the OOP paradigm.
Allegro: I haven't heard much about it. It gives me the idea of a toy, but it's not an opinion I count on. It is C based (like SFML) but doesn't have that large community (like SFML).
I'm considering only cross platform libraries. Now I'd like to make few questions about this situation:
Is SDL falling apart?
Why isn't SDL being update to the so awaited 1.3 (which will, they say, support iPhone and phones) in 3 years?
Is SFML gonna take SDL place?
Why people say that SFML is more abstract then SDL? It seems pretty much the same to me. You have likely all the configurations you want on both.
Allegro had a troubled story. It was only for DOS, then it was ported for other platforms etc... Is it really cross platform?
If I had to choose from that list, I'd definitely gun from SFML as it has a simple API and allows you to implement much of it yourself, a great learning experience. I'd like to just chime in and say SFML is simpler and sticks to what it needs to do. A base framework for rendering things.
I think you should also consider the new contender MonoGame (http://monogame.codeplex.com/)
OpenGL and written on .NET, it runs on Mono pretty much everywhere =) It has a beautiful API based off of XNA and does a lot of the work for you such as game loops and the really redundant stuff we all end up writing ourselves, anyway.
I currently use Allegro 4.x, it has some flaws that I had to manually work around, but since then they've released 5.x that overhauled the entire API and fixed all the problems I was having to fix.
Allegro 4.x is very stable (as in, no longer being developed) because the entire community suggests that if you are just starting with it that you start with 5.x which fixes all the problems with 4.x and is still cross-platform.
Allegro 5 is quite adept - I like it personally. The initialization code is simple to use, and easy to stuff away in a class or an Init() function. Really, if you write a wrapper function or class, you won't even have to directly call anything from any of these libraries.
Are you interested in game graphics or game development? Because if you want to make games, you should just pick one of these libraries and run with it. I'd only sweat these small steps if you're particularly interested in learning the nitty gritty of graphics development.