1
\$\begingroup\$

The following code gives me a completely black colored model, while it should be a model shaded in greyscale:

float4 SimplePixelShader(VertexShaderOutput input) : COLOR0
{
    //some lambertian shading done in the code I left out here

    return color;
}

sampler TextureSampler = sampler_state
{
    Texture = <ScreenTexture>;
};

float4 GrayscalePixelShader(float2 TextureCoordinate : TEXCOORD0) : COLOR0
{
    // Get the color.
    float4 color = tex2D(TextureSampler, TextureCoordinate); 

    // Turn pixel to grayscale.
    float grayscale = dot(color.rgb, float3(0.3, 0.59, 0.11));
    color.r = grayscale;
    color.g = grayscale;
    color.b = grayscale;
    color.a = 1.0f;

    // Return the result.
    return color;
}

technique Simple
{

    pass Pass0
    {
        VertexShader = compile vs_2_0 SimpleVertexShader();
        PixelShader  = compile ps_2_0 SimplePixelShader();
    }

    pass Pass1
    {
        PixelShader  = compile ps_2_0 GrayscalePixelShader();
    }
}

As far as I could find out the problem lies here:

// Get the color.
float4 color = tex2D(TextureSampler, TextureCoordinate);

The whole lambertian coloring works, the grayscale seems to work properly by itself after some testing, but what doesnt seem to work is getting the red lambertian shaded model colors. Anybody any clue what I'm doing wrong?

Edit: Thanks to a lot of help in the chat from LuisW I now have a working greyscale shader, I didnt have to change anything about my GreyscalePixelShader but I did have to remove the second pass of the simple techique and put it into a seperate technique, as such:

technique Greyscale
{
    pass Pass0
    {
        PixelShader = compile ps_2_0 GrayscalePixelShader();
    }
}

I also had to update my XNA/C# Draw method to create a fullscreen quad that I rendered my scene with the first technique to so I could use my greyscale technique on that quad as an input. Mind you, the method Im using, with spritebatch, is known to be slow (for me that didnt matter, but if you need fast, then drawing the quad directly is supposably faster) My updated draw method:

protected override void Draw(GameTime gameTime)
{
    this.device.Clear(ClearOptions.Target | ClearOptions.DepthBuffer, Color.DeepSkyBlue, 1.0f, 0);

    PresentationParameters pp = this.device.PresentationParameters;
    RenderTarget2D renderTarget;
    renderTarget = new RenderTarget2D(this.device, pp.BackBufferWidth, pp.BackBufferHeight, true, pp.BackBufferFormat, pp.DepthStencilFormat);
    this.device.SetRenderTarget(renderTarget);
    this.device.Clear(ClearOptions.Target | ClearOptions.DepthBuffer, Color.DeepSkyBlue, 1.0f, 0);
    Rectangle screenRectangle = new Rectangle(0, 0, screenWidth, screenHeight);

    // Get the model's only mesh
    ModelMesh mesh = this.model.Meshes[0];
    Effect effect = mesh.Effects[0];

    effect.CurrentTechnique = effect.Techniques["Simple"];

    //<left out some noninteresting worldmatrix calculations here not relevent for this question/answer> 

    device.DepthStencilState = DepthStencilState.Default;
    device.BlendState = BlendState.Opaque;

    mesh.Draw();

    this.device.SetRenderTarget(null);

    effect.CurrentTechnique = effect.Techniques["Greyscale"];

    spriteBatch.Begin(0, BlendState.Opaque, null, null, null, effect);
    spriteBatch.Draw((Texture2D)renderTarget, screenRectangle, Color.White);
    spriteBatch.End();

    base.Draw(gameTime);
}
\$\endgroup\$
  • \$\begingroup\$ Did you check the data ordering in the texture? Maybe the byte order is swapped. For a quick test, try: float4 color = tex2D(TextureSampler, TextureCoordinate).abgr; (if that fixes the problem, I’ll switch this comment to an answer) \$\endgroup\$ – sam hocevar Aug 11 '13 at 16:25
  • \$\begingroup\$ @SamHocevar This doesnt seem to fix the problem, but it does give a change: the model is now grey instead of black. (My model is a dark red so that makes sense but there's also a spotlight on the model and that should give a lighter shade of grey which doesnt happen) \$\endgroup\$ – Hasse Iona Aug 11 '13 at 18:01
  • \$\begingroup\$ @HasseIona What if you remove the grayscale code and just use the texture color? Does the texture show up correctly then? If not, you've got to fix that (which could be a byte order issue as Sam said) before worrying about the grayscale part. \$\endgroup\$ – Nathan Reed Aug 11 '13 at 21:11
  • \$\begingroup\$ I just want to add here that in case your texture is filled entirely with (r=0,g=0,b=0,a=1) reversing the Byte order will lead to what you observe because it will suddenly be interpreted as red and with your greyscale code obecome a monotonic (0.3,0.3,0.3,1) Color which is probably what you observe. You might also want to check on your texture coordinates, might be that you allways fetch a pixel at (0, 0) because of wrong UV and that one might allways be black. \$\endgroup\$ – Luis W Aug 12 '13 at 8:55
  • \$\begingroup\$ @NathanReed removing the greyscale and just using the texture color doesnt give me the correct colors. I've fiddled a bit with the byte order and basically what happens is the model is black if the a is at the end of the byte order and bright red if it is at the front, the ordering of the rgb doesnt matter for the color. Which would automatically suggest that its a (0, 0, 0, 1) (or reversed if the a is in front). \$\endgroup\$ – Hasse Iona Aug 12 '13 at 9:10
1
\$\begingroup\$

We had a chat conversation about it because it turned out out that there was a bigger problem than just an issue with the shader.

Heres my summary:

Multipass rendering using a single technique is very limited and it´s impossible to access the result of the first pass in the second. Inorder to get the first passes results you have to:

  • render the first pass into a texture
  • apply the second passes effect
  • render a fullscreen quad textured with the texture from the first pass

that way you can read the current pixels result of the first pass throught a texture lookup in the second pass and apply the effect on it.

Implementation details got edited into the question text, so i wont post them again.

\$\endgroup\$
  • \$\begingroup\$ @Byte56 this means that the chatroom can be closed, and thanks a lot for arranging the chatroom in the first place :) \$\endgroup\$ – Hasse Iona Aug 14 '13 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.