how does HDR work?

Question

I'm trying to understand what HDR is and how it works.

I understand the basic concepts and have an slight idea of how it is implemented with D3D/hlsl.

However it's still pretty foggy.

Say I'm rendering a sphere with a texture of the earth and a small point list of vertices to act as stars, how would I render this in HDR ?

Here are a few things I'm confused about:

• I'm guessing, I can't use just any basic image format for the texture as the values would be limited to [0, 255] and clamped to [0, 1] in a shader. Same goes for the back buffer, I take it the format needs to be a float point format ?

• What are the other steps involved ? Surely there has to be more than just using floating point formats to render to a render target and then apply some bloom as a post process ? (considering the output will be 8bpp anyway)

Basically, what are the steps for HDR ? How does it work ? I can't seem to find any good papers / articles that describe the process, other than this one, but it seems to skim over the basics a little, so it's confusing.

Answer 1

HDR techniques allows you to simulate a greater range of detail than you can view on screen than with traditional lighting/textures. You can compare it to how the eye behaves when exposed to different amounts of light - when there's too much light the eye lets less light in so things are still in your visible range. When there is not enough light, the iris opens more so more details can be seen.

The right hand side of this image is using HDR. It makes more use of the range of colors, the darks are darker and the bright areas are brighter. In comparison the left hand side of the image looks a little flat.

The basic steps are:

1. Render your scene to floating point texture (with a format such as A16B16G16R16F) using other floating point textures on your models, and/or lights that may have a brightness greater than 1.0f.

2. To display this texture the range of visible colors needs to be converted to something displayable on your screen - this process is called tone mapping, and a variety of different tone mapping equations can be used to get different effects. This is a must since monitors can't display the full range of colors or luminescence that we can store in floating point textures (it would be cool if it was possible, but it would also be a blinding hazard...).

3. Bloom and other after effects are added to further exaggerate the difference in luminescence of the things rendered. The bloom is calculated from the floating point buffer and combined with the tone mapped image.

Hope that helps

Answer 2

Technically, HDR merely means using a larger range of possible values for your graphics. Usually you're restricted to 256 discrete values for the red, green, and blue channels, which means that if you have 2 items, one twice as bright as the other, and a 3rd which is 10,000 times brighter than the first, there's no way you can represent all 3 in the same scene correctly - you either make the bright object only 256x brighter than the first instead, or you make both the dull objects completely black (losing the contrast between them) and then the bright object is infinitely brighter than them both.

This is easy to fix by using floating point values for the red/green/blue values - but now you have the problem of how to display that on a graphics device that only handles a fixed number of discrete values per channel (eg. 256). So the second part of the problem is how to map your floating point values back to the limited range. The trivial solution is to scale all the values proportionately into the discrete range, but this would mean that 1 very bright pixel could make the rest of the screen black, etc. Sometimes this is what you want, sometimes it's not - see CiscoIPPhone's tone mapping link for examples of how you can approach this.

It's generally not your textures that need to be stored in a new format - it's when the lighting is applied to them that you need to be able to accommodate larger values. Obviously however if you have light sources baked into a texture - eg. a starry background - you might want a higher resolution format there. Or just have the shader scale the values of such materials up when the time comes to render them.

Answer 3

Computers traditionally represented each pixel on screen as only 24 bits in memory: 8 for red, 8 for green, and 8 for blue. This is almost enough bits that a human wouldn't notice if you added more, and the 8-bit byte is very convenient for microprocessors, so that's what stuck.

While 8 bits is almost enough precision to display an image, it is definitely not enough precision to compute an image. At various points while computing an image, at least 32 bits of precision is required.

This is why pixel shaders compute colors in 32-bit precision, even when you are rendering to an 8-bit precision image. Otherwise, you couldn't for example divide a value by 1000, and then later multiply it by 1000, because dividing any 8-bit value by 1000 results in zero.

There has been a trend in realtime 3D graphics toward keeping all graphics in >8 bit precision until the last possible moment, at which time the >8 bits of red are downsampled to 8 bits, and so forth for green and blue.

HDR refers to the act of rendering to images which have higher than 8-bit precision. In contemporary TV videogames, 16-bit precision is the norm, and this may be "enough" in videogames for years to come.

Answer 4

One aspect of which I think is key for HDR is the correct application of monitor gamma.

The monitor you are looking at produces light as a function of the input pixels. You might expect that a pixel with value 255 will produce (approximately) 255 times more light than a pixel with value 1. This is not the case. With a standard monitor gamma of 2.3, it is 255 ^ 2.3 times brighter, or about 340000!

Everybody producing content (camera manifacturers) knows this, or (if you're a designer) you implicitly compensate for it.

This is all fine if you just render bitmaps (well, most of the time), but if you use them as textures in a 3D scene it is a different story. If you want to model interaction with light correctly, you should use linear light calculations throughout the rendering pipeline. This means

• correcting your textures for gamma

• render everything with linear light (where you need lots of precision due to the High Dynamic Range of light),

• apply the inverse gamma transform of the monitor as the last thing before you put the image on screen.

When you make this change to an existing scene, with existing artwork, lights, etc, you probably have to fix a lot of your light intensities and textures, because they were chosen to look nice when rendering with non-linear light. So it is not a feature you can just "turn on" and expect everything to look better just like that.