# RGB to xyY color space conversion and luminance

The luminance calculated by following GLSL functions (fragment shaders - tonemap) has different value:

``````float GetLuminance (vec3 rgb)
{
return (0.2126 * rgb.x) + (0.7152 * rgb.y) + (0.0722 * rgb.z);
}

vec3 RGB2xyY (vec3 rgb)
{
const mat3 RGB2XYZ = mat3(0.4124, 0.3576, 0.1805,
0.2126, 0.7152, 0.0722,
0.0193, 0.1192, 0.9505);
vec3 XYZ = RGB2XYZ * rgb;

return vec3(XYZ.x / (XYZ.x + XYZ.y + XYZ.z),
XYZ.y / (XYZ.x + XYZ.y + XYZ.z),
XYZ.y);
}
``````

I used a glm library to calculate an example result. For glm::vec3(2.0f, 3.0f, 8.0f) GetLuminance returns 3.1484. RGB2xyY returns glm::vec3 which z component is equal 3.8144. What is wrong ?

-
I'm not sure what's wrong with your implementation, but here's another one you can try. – Byte56 Apr 3 '13 at 18:00
I think that both luminances should have the same value. – Irbis Apr 3 '13 at 18:46
I understand that something is wrong, I just don't know where the problem lays. So I was giving you another implementation should you could implement and see if that fixes it. – Byte56 Apr 3 '13 at 20:16
Well what is the answer you should be getting? That should tell you which one is "wrong"? – Tetrad Apr 3 '13 at 21:21

Either the matrix is transposed (i.e. rows and columns swapped) in `RGB2xyY`, or the equation in `GetLuminance` uses the row vector instead of the column vector. The two values just happen to be close by accident.

Here is a Matlab trace showing whats going on. The quote is used to indicate matrix transposition. (The multiplication below is swapped around, of course you could also do `s*v'` and `s'*v'`).

BTW, using tools like Matlab (or Octave) are great for quickly prototyping these types of algorithms and finding bugs like these fast.

``````>> s = [0.4124, 0.3576, 0.1805;
0.2126, 0.7152, 0.0722;
0.0193, 0.1192, 0.9505] %assign matrix to s

s =

0.4124    0.3576    0.1805
0.2126    0.7152    0.0722
0.0193    0.1192    0.9505

>> v = [2 3 8] % assign vector to v

v =

2     3     8

>> v*s         % multiply

ans =

1.6170    3.8144    8.1816

>> v*s'        % multiply with transposed matrix

ans =

3.3416    3.1484    8.0002
``````
-