Why aren't my 2D primitives visible using a custom effect?

Question

I'm working with Monogame and rendering a triangle using the following code.

vertices = new []
{
    new VertexPositionColor(new Vector3(100, 200, 0), Color.White), 
    new VertexPositionColor(new Vector3(200, 100, 0), Color.White), 
    new VertexPositionColor(new Vector3(300, 200, 0), Color.White), 
};


graphicsDevice.DrawUserPrimitives(PrimitiveType.TriangleList, vertices, 0, 1);

Ordinarily I'd apply a BasicEffect before the draw call, but this time I'm using a custom effect. For testing, I wrote the following shader code to return solid green for each pixel within the triangle.

struct VertexShaderInput
{
    float4 Position : POSITION;
    float4 Color : COLOR;
};

struct VertexShaderOutput
{
    float4 Position : POSITION;
    float4 Color : COLOR;
};

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
{
    VertexShaderOutput output;
    output.Color = input.Color;
    output.Position = input.Position;

    return output;
}

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR
{
    return float4(0, 1, 0, 1);
}

technique Technique0
{
    pass Pass0
    {
        VertexShader = compile vs_4_0_level_9_1 VertexShaderFunction();
        PixelShader = compile ps_4_0_level_9_1 PixelShaderFunction();
    }
};

The problem is that, although the shader compiles successfully, nothing shows up.

Why is my triangle invisible using this shader?

Based on my understanding of vertex and pixel shaders, the above HLSL code should work correctly. In the vertex shader, since (for the moment) I'm working in strictly 2D without a camera, no transformations need to be done on the vertices (i.e. they're already in screen space). From there, the pixel shader simply returns solid green for each pixel. I've also ruled out potential backface culling issues by swapping the order of vertices, and I've tried using different semantics on my HLSL structure variables. No luck.

Shaders still feel uncomfortable to me, but I feel my logic is correct based on what I've read. If that's the case, there must be some weird quirk I'm not aware of in having these HLSL shaders work at all, apart from the logic. What am I missing? Is there a better way to debug these problems short of changing random variables and hoping for the best?

Answer 1

The problem you are having is that 'no transform' coordinates are not in pixels. They are in normalized device coordinates which range from -1 to 1. If you want to specify your vertices in pixel coordinates, you need a transformation which converts that to normalized device coordinates with a scale and offset.

For example in DirectX Tool Kit's SpriteBatch, this function computes a matrix to do just that:

// Generates a viewport transform matrix for rendering sprites using x-right y-down screen pixel coordinates.
XMMATRIX SpriteBatch::Impl::GetViewportTransform(_In_ ID3D11DeviceContext* deviceContext, DXGI_MODE_ROTATION rotation )
{
    // Look up the current viewport.
    if ( !mSetViewport )
    {
        UINT viewportCount = 1;

        deviceContext->RSGetViewports(&viewportCount, &mViewPort);

        if (viewportCount != 1)
            throw std::exception("No viewport is set");
    }

    // Compute the matrix.
    float xScale = (mViewPort.Width  > 0) ? 2.0f / mViewPort.Width  : 0.0f;
    float yScale = (mViewPort.Height > 0) ? 2.0f / mViewPort.Height : 0.0f;

    switch( rotation )
    {
    case DXGI_MODE_ROTATION_ROTATE90:
        return XMMATRIX
        (
             0,       -yScale,  0,  0,
             -xScale, 0,        0,  0,
             0,       0,        1,  0,
             1,       1,        0,  1
        );

    case DXGI_MODE_ROTATION_ROTATE270:
        return XMMATRIX
        (
             0,       yScale,   0,  0,
             xScale,  0,        0,  0,
             0,       0,        1,  0,
            -1,      -1,        0,  1
        );

    case DXGI_MODE_ROTATION_ROTATE180:
        return XMMATRIX
        (
            -xScale,  0,       0,  0,
             0,       yScale,  0,  0,
             0,       0,       1,  0,
             1,      -1,       0,  1
        );

    default:
        return XMMATRIX
        (
             xScale,  0,       0,  0,
             0,      -yScale,  0,  0,
             0,       0,       1,  0,
            -1,       1,       0,  1
        );
    }
}