There are many places where HDR makes a difference besides providing source data to base a bloom filter on. Essentially, anyplace where lighting values are scaled or added together is going to look different, and will be more realistic / physically correct when done in HDR.
One of the most obvious places where it matters is with environment reflections. As you may know, most non-metal materials have a very low specular reflection at normal incidence - like 2% to 5%. So the reflection will only be 2% to 5% as bright as the environment it's reflecting. If you don't have an HDR environment (including sky, light sources, etc. with realistic brightness levels), this will lead to unnaturally dark, washed-out-looking reflections. With an HDR environment, the reflection looks natural and maintains contrast despite the low overall specular level.
Here's an example I rendered using a gray sphere with a 5% spec and an environment map from one of Paul Debevec's HDR light probes. There is no tone mapping or bloom in this image.
This problem shows up with any shiny surface; cars, glass windows, and water are common cases. You can hack around this a bit by scaling or otherwise altering an LDR cubemap, but the benefit of an HDR cubemap is that it "just works" regardless of material or lighting environment.
Aside from reflections, HDR is important just for light accumulation to function correctly. When you have a scene with multiple sources of light, perhaps ambient and directional, or several converging spotlights, etc., there can be places where the total amount of light goes over 1.0, even if none of the individual lights has a brightness over 1.0. It improves the visuals if you use tone mapping to reign in the excess brightness smoothly, rather than simply saturating to 1.0 and clipping anything over that.
Here's an example from Uncharted 2 (this image can be found on slide 126 of John Hable's GDC talk). As you can see, without tone mapping, the areas on the character's cheek and forehead where the lighting gets too bright show an unpleasant yellow tinge (due to the red and green channels saturating before the blue) as well as complete loss of detail in the saturated region.
Finally, one more place where HDR shows up is postprocessing effects like motion blur and depth of field. Although it can be expensive, doing your whole postprocess chain in HDR leads to much more realistic results. Here is an example of what happens with motion blur:
When doing a blur, bright areas of an HDR image will remain bright and hard, while in an LDR image they will soften and dim. As shown above, the HDR postprocessing is a better simulation of what would happen in a real camera, so it looks more "filmic".
So there's much, much more to HDR lighting and rendering than just bloom.
