What causes this InvalidOperationException when drawing primitives?

Question

I'm working on implementing a 2D masking effect in my current game. To do so, I have two textures, one for the source image and one for the mask. The source image is just a regular texture, so that's easy. The mask I'm generating by rendering to a render target through the use of graphicsDevice.DrawUserIndexedPrimitives, but when I try, the following error pops up.

This is the function that attempts to render to the mask's render target. Vertices are of type VertexPositionColor.

private void RenderMaskTarget(SpriteBatch sb)
{
    graphicsDevice.SetRenderTarget(maskTarget);
    graphicsDevice.Clear(Color.Transparent);

    sb.Begin(SpriteSortMode.Deferred, null, null, null, null, basicEffect, camera.Transform);

    graphicsDevice.DrawUserIndexedPrimitives(PrimitiveType.LineStrip, vertices, 0, vertices.Length, indexData, 0, primitiveCount);

    sb.End();
}

And this is the function that creates my basic effect.

private void CreateBasicEffect()
{
    Matrix projection = Matrix.CreateOrthographicOffCenter(0, Constants.ScreenWidth, Constants.ScreenHeight, 0, 0, 1);
    Matrix halfPixelOffset = Matrix.CreateTranslation(-0.5f, -0.5f, 0);

    basicEffect = new BasicEffect(graphicsDevice)
    {
        World = Matrix.CreateOrthographicOffCenter(0, graphicsDevice.Viewport.Width, graphicsDevice.Viewport.Height, 0, 0, 1),
        View = Matrix.Identity,
        Projection = halfPixelOffset * projection,
        VertexColorEnabled = true,
        LightingEnabled = false
    };
}

I've searched everywhere I can on this site and Google in general, but nothing I've tried has worked. I think my problem is a fundamental misunderstanding of what the exception above actually means. Based on its wording, the current vertex declaration doesn't include all required elements, and it then goes on to list the elements I do have, but it doesn't tell me what the required ones are. Further, where is the current vertex declaration defined? Is it VertexPositionColor that defines it or the BasicEffect?

I realize that similar questions to this have been asked before, but like I said, no solutions I've seen have worked. I think part of that is XNA vs. Monogame, where an XNA solution might not work for Monogame. I apologize if this is a true duplicate question.

Thank you :)

Answer 1

This section is not correct:

sb.Begin(SpriteSortMode.Deferred, null, null, null, null, basicEffect, camera.Transform);

graphicsDevice.DrawUserIndexedPrimitives(PrimitiveType.LineStrip, vertices, 0, vertices.Length, indexData, 0, primitiveCount);

sb.End();

You're using a spriteBatch and then bypass it by calling the DrawUserIndexedPrimitives directly. I'm not entirely sure but it seems to me that your drawcall simply has no shader active at the time you call it. Since the spriteBatch starts drawing at the sb.End().

So what I think happens here is

1. you start a spriteBatch.
2. attempt to draw something without the batch.
3. close the spriteBatch- but it is empty so nothing happens.

At the point 2, no shader is actively set.

Replace the draw section by:

private void RenderMaskTarget()
{
    graphicsDevice.SetRenderTarget(maskTarget);
    graphicsDevice.Clear(Color.Transparent);

    basicEffect.CurrentTechnique.Passes[0].Apply();
    graphicsDevice.DrawUserIndexedPrimitives(PrimitiveType.LineStrip, vertices, 0, vertices.Length, indexData, 0, primitiveCount);

}

Answer 2

Your active vertex declaration includes SV_Position0 and COLOR0 elements, as you have surmised. Your active shader expects a different set of elements, and thus cannot be used to render the geometry you are sending.

You need to make your vertex declaration and your shader match (you also need to make sure you actually send the type of data the vertex declaration asserts).

You're using MonoGame's BasicEffect, which I believe is a "built in" shader that expects positions, colors and texture coordinates. Your vertex declaration doesn't contain texture coordinates and you may not actually be sending them in the vertex data, either.

The simplest thing you can probably do is set TexturesEnabled to false on your effect, which should stop it from requiring texture coordinates. Alternatively, provide texture coordinates in your vertex data and declaration.