I used both SDL and SFML for my game projects. I used SDL for a finished game, a tetris game, and I mostly used SFML as a windowing layer for shader experiments. From my experience, I'd have to say that SDL is what it says. It's a simple direct-media layer, which abstracts the data your media (sound and video) works with to a level you can interact with accross different platforms. That's "all" it does.
Because of that, you generally have to mix it with a few other libraries, most notably SDL_mixer, SDL_gfx, and SDL_image, which provide functionality everyone needs to create games.
All of these libraries are also meant for the C programming language, and they follow it's conventions. Not that there is anything wrong with that, but if you want an object oriented approach to how you interact with io, then you basically have to build that layer yourself.
SFML comes with everything right out of the box. It has sound, it has video, it has mixing, it has frame-capping, and it is object oriented. If that's what you want, I'd recommend you go for that. SFML also has an abstraction layer for some of the basic functionality in opengl (specifically beginShape/EndShape and pixel shaders). I don't think SDL has anything like that, just a method to merge with it.
P.S. Not to drive away the conversation from C/C++, but if you're not using that language for any particular reason, and want an easier one, there's always pygame. Which is basically SDL+ for Python.