1
\$\begingroup\$

I should start his question off by saying that i am a begginer in HLSL. I've borrowed some shader code for flat shading from a website i've long since lost. It's works brilliantly for textured models but i cannot seem to get it functioning using vertex colored models. it just seems to always be grey.

My HLSL shader is as follows:

float4x4 World;
float4x4 View;
float4x4 Projection;

float4 AmbientColor = float4(1, 1, 1, 1);
float AmbientIntensity = 0.3f;

float3 DiffuseLightDirection = float3(1, 0, 0);
float4 DiffuseColor = float4(1, 1, 1, 1);
float DiffuseIntensity = 1.0f;

texture ModelTexture;
sampler2D textureSampler = sampler_state {
    Texture = (ModelTexture);
    MinFilter = Linear;
    MagFilter = Linear;
    AddressU = Clamp;
    AddressV = Clamp;
};

struct VertexShaderInput
{
    float4 Position : SV_Position0;
    float4 Normal : NORMAL0;
    float4 Color : COLOR0; //Added in by me
    float4 TextureCoordinate : TEXCOORD0;
};

struct VertexShaderOutput
{
    float4 Position : SV_Position0;
    float4 PositionWorld : TEXCOORD0;
    float4 Color : COLOR0; //Added in by me
    float4 TextureCoordinate : TEXCOORD1;
};

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
{
    VertexShaderOutput output;

    float4 worldPosition = mul(input.Position, World);
    float4 viewPosition = mul(worldPosition, View);
    output.Position = mul(viewPosition, Projection);
    output.PositionWorld = worldPosition;
    output.Color = input.Color; //Added in by me
    output.TextureCoordinate = input.TextureCoordinate;

    return output;
}

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
    float3 Normal = cross(ddy(input.PositionWorld.xyz), ddx(input.PositionWorld.xyz));
    Normal = normalize(Normal);

    float lightIntensity = dot(Normal, DiffuseLightDirection);
    float4 lightColor = lightIntensity * DiffuseColor * DiffuseIntensity + AmbientColor *     AmbientIntensity;
    lightColor.a = 1;

    float4 textureColor = tex2D(textureSampler, input.TextureCoordinate);
    textureColor.a = 1;

    //return saturate(textureColor * lightColor);
    return saturate(input.Color * lightColor); //Added in by me
}

technique FlatTextured
{
    pass Pass1
    {
        VertexShader = compile vs_4_0 VertexShaderFunction();
        PixelShader = compile ps_4_0 PixelShaderFunction();
    }
}

As you can see I've temporarily commented out the line which uses the texture color for shading and tried to replace it with vertex color. I've also highlighted in the code which lines i've changed to attempt to use vertex coloring.

At the moment this only produces grey shaded models. I know there is vertex data on the models because i can use MonoGame's built in BasicEffect to see the coloring.

What am i doing wrong? How can i modify the existing shader to use vertex color instead of texture color?

Edit: Additional information

I've tried using the BasicEffect in monogame to draw my objects and the vertex colors are coming through fine. It seems that i should be able to see the colors with my shader but i am not.

With the BasicEffect BasicEffect shading

With My Effect MyEffect shading

Edit: Additional Information

Here is the code in the draw method i am using. The un-commented part is using my shader, while the commented part is the BasicEffect that is set up when the model is loaded.

flatShadedEffect.CurrentTechnique = flatShadedEffect.Techniques["Flat"];

        foreach (ModelMesh mesh in modelobject.Model.Meshes) {

            foreach (ModelMeshPart part in mesh.MeshParts) {
                part.Effect = flatShadedEffect;
                part.Effect.Parameters["View"].SetValue(camera.GetView());
                part.Effect.Parameters["Projection"].SetValue(camera.GetProjection());
                part.Effect.Parameters["World"].SetValue(modelobject.GetWorld());
            }

            //foreach (BasicEffect effect in mesh.Effects) {
            //    effect.World = modelobject.GetWorld();
            //    effect.View = camera.GetView();
            //    effect.Projection = camera.GetProjection();
            //}


            mesh.Draw();
        }
\$\endgroup\$
  • \$\begingroup\$ From github.com/mono/MonoGame/blob/master/MonoGame.Framework/… - MonoGame seems to be using a different vertex data input structure. Perhaps you could using their struct (VSInputNmTxVc, with the same data types and member order) and see if that helps? \$\endgroup\$ – snake5 Jul 19 '15 at 14:51
  • \$\begingroup\$ @snake5 Unfortunately this did not work, i tried copying the structures, vertex shader and pixel shader of the effect (BasicEffect_VertexColor_NoFog) that was the technique the BasicShader was using with the same result. I'm stumped, something else is going on in the Draw series of methods for the basic effect, but i cannot seem to even find where the effect parameters are being set for the BasicShader in the source code. \$\endgroup\$ – Lex Webb Jul 20 '15 at 0:14
  • \$\begingroup\$ What are the changes performed in C# code to switch from your effect to BasicEffect? Also, what's the associated C# code for using your effect? Since the shader seems to be fine, I'm guessing there might be some issues in the usage code. \$\endgroup\$ – snake5 Jul 20 '15 at 8:00
  • \$\begingroup\$ I've updated the question with the information. \$\endgroup\$ – Lex Webb Jul 20 '15 at 9:50
1
\$\begingroup\$

So after a few days of digging i finally found this question which is asking the exact same thing as me. He fixed the issue by using in my opinion a dirty hack.

This involves getting the vertex colors from the original BasicShader when the model is created. The when it comes to draw, that color is passed in a parameter to the shader which in-turn uses it as the parts diffuse color. This produces the image i want, but is a bit hack-ish..

I will keep the question open in case a better answer present itself.

The Result: enter image description here

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.