Debugging .FX files in Visual Studio not working

Question

So I have been learning how to use DirectX 11 for a while now and I have chosen to use the .FX framework rather than seperate .vs & .ps files. (I know it's deprecated but I found it a little easier to use for learning, I will use in seperate .vs and .ps files in the near future).

In order to get my engine to compile I have to keep the effects.fx file in the same file directory as the engine project but I cannot include it in the VS2015 project directly (This works fine). At the moment whenever I am writing some code in my .FX file and it creates an error I get an error from the C++ side of things and nothing from my shader. Which leaves me clueless on what is causing the issue.

How can I get the HLSL compiler error/warning output to the debug window?

Answer 1

First you will have to set up a custom build step like so: right click on project -> build customization-> then add the fx(targets, props).

Looks like this:

Have all the shaders under a filter in VS then right click the shader(.fx) file-> propertys-> set item type on the left to HLSL compiler-> hit apply-> change HLSL compiler settings to effect.

I also use an extension so I have real time compellation of shaders as I type and it hooks into the MSDN docs so u get some info on what u are doing its called HLSL tools for Visual Studio, once I change some shader code I hit compile on the shader file in VS and see what I get in the output window\error window.

Answer 2

You have to enable debugging in the initialize of the shader.

DWORD dwShaderFlags = D3DCOMPILE_ENABLE_STRICTNESS |
        D3DCOMPILE_IEEE_STRICTNESS | D3DCOMPILE_DEBUG; //Debug yay =D
hr = D3DCompile(shader_code.data(),
        shader_code.size(),
        NULL,
        pDefines,
        D3D_COMPILE_STANDARD_FILE_INCLUDE,
        pEntrypoint,
        pTarget,
        dwShaderFlags,
        NULL,
        pCompiledShader,
        &pErrorBlob);