# Pixel Shader, YUV-RGB Conversion failing

I am tasked with playing back a video hthat comes in in a YUV format as an overlay in a larger game. I am not a specialist in Direct3d, so I am struggling. I managed to get a shader working and am rendering 3 textures (Y, V, U). Sadly I am totally unable to get anything like a decent image. Documentation is also failing me. I am currently loading the different data planes (Y,V,U) in three different textures:

m_Textures = new Texture[3];
// Y Plane
m_Textures[0] = new Texture(m_Device, w, h, 1, Usage.None, Format.L8, Pool.Managed);
// V Plane
m_Textures[1] = new Texture(m_Device, w2, h2, 1, Usage.None, Format.L8, Pool.Managed);
// U Plane
m_Textures[2] = new Texture(m_Device, w2, h2, 1, Usage.None, Format.L8, Pool.Managed);


When I am rendering them as R, G and B respectively with the following code:

float4 Pixel(float2 texCoord: TEXCOORD0) : COLOR0
{
float y = tex2D (ytexture, texCoord);
float v = tex2D (vtexture, texCoord);
float u = tex2D (utexture, texCoord);

//R = Y + 1.140 (V -128)
//G = Y - 0.395 (U-128) - 0.581 (V-128)
//B = Y + 2.028 (U-128)

float r = y; //y + 1.140 * v;
float g = v; //y - 0.395 * u - 0.581 * v;
float b = u; //y + 2.028 * u;

float4 result;
result.a = 255;
result.r = r; //clamp (r, 0, 255);
result.g = g; //clamp (g, 0, 255);
result.b = b; //clamp (b, 0, 255);

return result;
}


Then the resulting image is - quite funny. I can see the image, but colors are totally distorted, as it should be.

The formula I should apply shows up in the comment of the pixel shader, but when I do it, the resulting image is pretty brutally magenta only. This gets me to the question - when I read out an L8 texture into a float, with

float y = tex2D (ytexture, texCoord);


what is the range of values? The "origin" values are 1 byte, 0 to 255, and the forum I have assumes this. Naturally I am totally off when the values returned are somehow normalized. My Clamp operation at the end also will fail if for example colors in a pixel shader are normalized 0 to 1. Anyone an idea how that works? Please point me also to documentation - I have not found anything in this regard.

• Don't know HLSL, but I'd expect the values read from the texture to be between 0 and 1, not 0 and 255. Try y *= 256 and see what it gets you? Second point, why not just convert to RGB when decoding the video, rather than doing it in a shader? Or put the YUV values into the 'RGB' labelled channels of a single texture and do the conversion in the shader, but avoid three separate layers on the GPU side? – Will Aug 13 '12 at 15:12
• Did and that actually worked ;) The conversion is out of my hand in this case - the format is fixed and the image is life and the decoder can not do additional transpositions. The layers also have different sized (U and V are quarter images, half width and height). Please add as answer and I will approve ;) – TomTom Aug 13 '12 at 16:26
• Eh which bit of my advice worked? The 256 bit, right? Yeah, I've played with some pretty split up YUV plane decoding myself in the past and usually your hands are pretty tied. (What SO needs is a way to mark a *comment as the accepted Answer) – Will Aug 14 '12 at 5:52
• Yeah. Multiplied with 256. Still looking for a better way to debug Shaders in Visual Studio ;) I would have seen this immediately. – TomTom Aug 14 '12 at 9:30

tex2D(sampler,pos) return values between 0 and 1.