Short version: What can happen to the output of a vertex shader that leads to no(or very few) pixels being drawn, especially in relation to a perspective matrix.

Long version:

My problem refers to the fact that when using a projection matrix very few pixels are actually drawn, more often than not, no pixels are drawn. I have used the built-in debugger in VS2012 and inspecting the output of the vertex shader reveals that the transformations are most likely correct, it is only when it is passed to the pixel shader that things start to go astray.

What I tried:

• use different built in matrices for calculating perspective
• check the code for indirect modifications
• set "CULLING_NONE" as a rasterizer state, that only increased the chance of something actually being drawn
• set the "w" component of the SV_POSITION vector to 1, that also seemed to help to a certain degree
• checked winding of triangles

My implementation performs the View-Projection multiplication first (of their transposes) and then multiplies the position with the result of the former. I believe that's not where the problem lies due to the associative property of matrix multiplication.

I'd prefer it if I wouldn't have to resort to all sorts of hacks that can only increase the chance of something being drawn at all (never in the right way) as it seems clear that the problem lies in my implementation.

• I don't know if there are any major differences with VS 2012 and 2010 on the debugging side, but for debugging hlsl and directX graphics, the standalone program PIX is largely recommended. But looking from the screenshots, it looks like some pipeline debugging has been put into VS 2012 as well. – ChrisC Feb 28 '13 at 23:49

Well, a lot of stuff happens between the Vertex and Pixel shaders, but before we get into that, I'm worried that you may be a bit confused about what the Vertex and Pixel shaders do. So let's quickly go through the entire Direct3D 9 programmable pipeline again. This nice chart shows the entire pipeline, and you should print it out for reference.

(feel free to imagine I'm saying this with the voice of the "how it's made" videos)

It all begins at your program. There are several ways you can start drawing stuff on the screen. But regardless of whether you're using Indexed Primitives, Regular Primitives, Rect/Tri Patches (which are automatically tesselated for you) or directly with DrawPrimitiveUp, you will be feeding the pipeline with a lot of vertices.

Each vertex simply defines a point in space, maybe with some custom information for texture coordinates, or normals, or whatever you want to send with your vertices. But remember that they're just vertices. Triangles and Pixels don't even exist at this stage.

Then, a program called the Vertex Shader is applied to each one of these vertices indiscriminately. The main job of the Vertex Shader is to transform each vertex and convert it to viewport coordinates. Nothing is yet drawn at this stage (the white lines you see are put in there by the debugger for your debugging pleasure, but don't exist in the GPU at all).

Afterwards, all vertices go through Primitive Assembly, which takes vertices in groups of three, and makes triangles out of them.

If you're using DirectX 10 or above, triangles (or whatever your primitive topology is) can get sent to another program called the Geometry Shader, where you can create more triangles from the current ones if you wish. This is mostly used for tesselation and for particle systems.

Each triangle then goes through Face Culling if it's enabled. In Face Culling, triangles not facing the camera are discarded. Generally, half of all triangles are expected to be discarded in this stage.

Even more triangles are discarded in the Plane Clipping stage. Triangles too close or too far from the camera are discarded.

If this were too little, one additional round of triangle discarding goes on at the Frustum Clipping stage, which discards triangles that are outside of the viewport, and therefore not visible.

The next step consists of performing a Homogenous Divide (or "Division by w") to normalize the position of all triangles in the viewport.

Finally, all the triangle coordinates are transformed to those of the viewport (actual pixels) in the Viewport Mapping stage.

Next comes the Rasterization and Component Interpolation stage. Each triangle is disassembled into the fragments that make it up (one fragment for each pixel in the screen). For each fragment, all the components you passed into the original vertices, such as texture coordinates, normals, or your custom data, are interpolated throughout the entire triangle, and assigned to each fragment.

Each fragment is then passed to a program called the Pixel Shader, whose function is to make a pixel out of each fragment. You're supposed to use the interpolated texture coordinates, normals or whatever custom data you originally passed, to figure out what the color of each pixel is. In simpler terms, this is where "drawing happens".

Afterwards, pixels outside of a region you define called the Scissor Rect are discarded by the Scissor Test, if you enabled it.

Then each pixel goes through the Alpha Test, if you have enabled it. Pixels that fail the Alpha Test are discarded.

If there are triangles that overlap in the viewport, there will be more than one pixels meant to be drawn for each actual pixel in the screen. If you enable the Depth Test, you can make sure pixels belonging to triangles behind those that have already been rendered are discarded (no point in drawing what won't be seen)

The Stencil Test, if enabled, is similar to the Depth Test, and is used for effects such as volume textures.

Finally, Alpha Blending, if enabled, blends pixels in the same location with a function and parameters you can define.

The pixels can then be sent to the Dithering stage, if enabled (I've never enabled it myself). Afterwards, a Channel Mask can be applied, and everything gets sent to the Render Target.

At this point, the scene is considered to be rendered. No further per-pixel operations are performed. If the render target is the actual screen, the image can then be sent to the display.

Now, onto your specific problem, I'm not sure exactly where in the pipeline is each of the debugger views probing, but my guess is that you're using the w coordinate incorrectly, because after the Vertex Shader, the only thing that can change the location of your vertices is the homogeneous divide.

In general, your vertices should be sent without a w coordinate. Make sure you use the right world and view matrices, and if after multiplying each vertex by your world and view matrices (you can do this by hand for debugging), your X and Y coordinates fall between -1 and 1, and your Z coordinate falls between your near and far clipping planes, your vertices will appear on the screen.

• Don't you think w whould be 1.0? [the default value in GLES for vec4 attribute is (0.0, 0.0, 0.0, 1.0);] Anyway, excellent overview! – Aki Suihkonen Mar 1 '13 at 6:26
• @AkiSuihkonen You're completely right. I usually send vertices in groups of three, and have never needed to send a custom w, but the math only makes sense if w is 1. – Panda Pajama Mar 1 '13 at 6:56
• +1 for "(feel free to imagine I'm saying this with the voice of the "how it's made" videos)". It may sound shallow as an argument, but besides that, it really got close to that kind of presentation. Might be nice to put the image straight into your answer for an even easier understanding. – teodron Mar 1 '13 at 8:28
• You didn't really touch my subject, probably I didn't make myself understood, but I will check for "Homogenous Divide", as that seems to be where things go wrong. – Taikand Mar 1 '13 at 16:13
• @Taikand post your shader code and the matrices you're using and I might be able to help you. By the way, the title of your question (which I tried to answer) doesn't match the actual content. – Panda Pajama Mar 1 '13 at 16:18