0
\$\begingroup\$

UPDATE 3:

I have fixed the issue by reordering my effects (see my answer for details.) Anyone who wants to post more information based on that answer (either in an answer or in the comments) would be greatly appreciated.

UPDATE 2:

I think the issue may be that I've been using the default Magenta as transparent rather than having actual alpha values. It's hard to tell through Monogame what magic is happening to make the Magenta transparent (note: I'm in a 3d camera and not using spritebatch.)

I will try this again tonight or tomorrow with a set of identical textures using an alpha value instead of magenta.

Looks like this was unrelated to my issue. I now have images that have proper alpha values rather than replacement color values and am still having the same issue.


I've been able to apply my shader to multiple layered textures, with the issue that I can't seem to get the alpha to detect in my shader.

enter image description here

As you can see in the picture, the Alpha values for the Turret and the Tank-body are not being multiplied in properly ("derp.a") meaning that the lighting pass is hitting certain pixels 2x-3x based on how the textures overlap.

Full shader pasted below:

//Texture2D currTexture : register(s0);
sampler currSampler : register(s0);
/*{
    Texture = (currTexture);
    AddressU = clamp; 
    AddressV = clamp;
};*/

float4x4 xViewProjection;
float3 tankpos;
float4 shotpos;


struct VertexToPixel
{
    float4 Position : POSITION; // Can't use in pixel shader, but required for it
    float4 Color : COLOR0;      // color for the vertex?
    float2 rPos : TEXCOORDS0;   // world position of the texture
};

struct PixelToFrame
{
    float4 Color : COLOR0;
};

VertexToPixel SimplestVertexShader(float4 inpos : POSITION)
{
    VertexToPixel output = (VertexToPixel) 0;
    output.Position = mul(inpos, xViewProjection);
    output.rPos = inpos.zx;
    return output;
}

PixelToFrame LightAroundAShot(VertexToPixel PSIn)
{
    PixelToFrame output = (PixelToFrame) 0;
    output.Color = PSIn.Color;
    float x = (PSIn.rPos.x * 0.02 - shotpos.z * 0.02) * 1 / (1 + shotpos.w);
    float z = (PSIn.rPos.y * 0.02 - shotpos.x * 0.02) * 1 / (1 + shotpos.w);
    float4 derp = tex2D(currSampler, float2(PSIn.rPos.x, PSIn.rPos.y));

    output.Color.g += (0.75 - sqrt(x * x + z * z)) * shotpos.w * derp.a;
    output.Color.br += (0.35 - sqrt(x * x + z * z)) * shotpos.w * derp.a;

    return output;
}


technique Simplest
{
    pass Pass0
    {
        VertexShader = compile vs_4_0_level_9_1 SimplestVertexShader();
        PixelShader = compile ps_4_0_level_9_1 LightAroundAShot();
    }
}

I've attempted to use curr texture at the top in a couple ways and can't seem to find the voodoo to get it working. Any help understanding the relationship (especially in monogame) between Textures, Samplers, and how to get that darn "derp" alpha to apply correctly would be great.

Note: When I tried using curr texture I was setting it before each call of the graphics device so that it'd be the appropriate texture. I've also attempted to register them in various ways to s1, different ways of setting the texture in graphics device. I'm truly stuck at the moment.

Thank you for any help!

\$\endgroup\$
  • \$\begingroup\$ Can you explain what you mean by getting the "alpha to detect"? What's the desired / expected output, and how does it differ from what you've shown above? \$\endgroup\$ – DMGregory May 29 '18 at 17:07
  • \$\begingroup\$ The Tankbody and Tankturret Alphas should be 0 in those areas that they don't have any yellow/orange coloring. The issue is that the lightning is being applied to those areas multiple times because when I multiple "derp.a" (the alpha of the texture), it's still 1 instead of 0; meaning I'm probably talking to the wrong texture resource via the sampler. The hope is that the lighting pass would only be applied once (by having those alpha's be 0) and thus not apply 2x-3x lighting in those odd little green boxes you see around the tank. I'll add this to the question as well for clarification \$\endgroup\$ – blurry May 29 '18 at 17:11
  • \$\begingroup\$ Note it seems you are using the legacy Effects system for Direct3D 11. Be sure to use the latest version from GitHub if using C++. \$\endgroup\$ – Chuck Walbourn May 30 '18 at 16:54
  • \$\begingroup\$ I'm using C#/Monogame. What do you mean the Legacy Effects system? Are you saying that the Effect class is not the appropriate way to load shaders in Monogame? \$\endgroup\$ – blurry May 30 '18 at 19:49
0
\$\begingroup\$

I ended up having a (hacky) fix of reordering Effects. It seems that what really wants to occur here is part of the Alpha Test/zbuffering that I was letting an AlphaTestEffect do for me. At the moment, ordering the effects so that they all apply to the object at the same time and then move on, and doing it oriented by my Depth axis (y in this case) seems to get rid of the boxes (and thus only light once per object, rather than twice.)

The proper way to fix this in the coming week or two is to write my own alpha testing and z-buffering shader, now that I'm learning shaders, and then have these effects that rely on transparency perform a similar operation to alpha-testing and z-buffering when applying their color modification.

There is probably a more efficient way than repeating those steps in multiple shaders, but I guess I'll find out in time. Regardless, I got lucky enough to stumble into a working animation.

To those who end up with this issue; it seems that by letting Monogame perform some magic with AlphaTestEffect and then writing your own effect for lighting requires you to go back and write your own Alpha-Test OR draw things in Depth-order; even though you have an alpha-test drawing their initial sprites correctly.

Anyone with more information based on the above may still be able to help my understanding.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.