For HDR (high dynamic range) rendering, you would want to use more than 8 bits usually - since you are not only encoding color, but also intensity of the light much more precisely (and thus over a greater range) than 8 bit RGB can do.
Of course, your monitor can likely only display 8 bit RGB (if even that), so this only matters if you are actually doing some processing on the data before displaying it using ordinary 8 bit RGB - as is often done with HDR rendering (applying bloom, tone mapping, more realistic lighting...)
So if you use HDR rendering, you sometimes want more bits to more accurately represent the color over a wide range of intensities.
For the CPU side, operating on floating point color values is probably more convenient than working with integers when it comes to operations other than addition/subtraction, as well.
Even if you didnt use HDR rendering, you could use the higher color range to implement dithering on the GPU. This would reduce the color banding effect you get due to 8 bits not being enough for creating visually smooth gradients of color in some cases.
For 'normal' uses of textures, you wont be doing anything that would significantly benefit from more than 8 bits per component.