Compiled shader not working?

Question

I did a simple pixel shader that makes the texture monochrome, and it works perfectly when it is precompiled, but not when compiled (returns E_FAIL). The effect was compiled using fxc.exe with these arguments: fxc /T ps_2_0 /Fo BasicHLSL.fxc BasicHLSL.hlsl

Here's the code

ID3DXEffect* monochromeShader = nullptr;

void initEffects(GameWindow& w)
{
    long errorID =
    D3DXCreateEffectFromFile
    (
        w.getD3DDevice(),
        "data/shaders/BasicHLSL.fxc\0",
        NULL,
        NULL,
        D3DXFX_NOT_CLONEABLE,
        NULL,
        &monochromeShader,
        NULL
    );

    std::cout << std::hex << errorID;

    if (monochromeShader == nullptr)
        showErrorMsg(std::to_string(errorID).c_str());
}

And here's my shader before compiling with fxc.exe:

// Pixel shader input structure
struct PS_INPUT
{
    float4 Position   : POSITION;
    float2 Texture    : TEXCOORD0;
};

// Pixel shader output structure
struct PS_OUTPUT
{
    float4 Color   : COLOR0;
};

// Global variables
sampler2D Tex0;

PS_OUTPUT ps_main( in PS_INPUT In )
{
    // Output pixel
    PS_OUTPUT Out;

    Out.Color = tex2D(Tex0, In.Texture);
    float4 f4 = Out.Color * float4(0.299f, 0.587f, 0.114f, 1.0f);
    float f = f4.r + f4.g + f4.b;
    Out.Color.rgb = f;

    return Out;
}

// Effect technique
technique tech_main
{
    pass P0
    {
        // shaders
        VertexShader = NULL;
        PixelShader  = compile ps_2_a ps_main();
    }  
}

Thanks for reading.

Answer 1

The problem here is that you compiled the shader with the ps_2_0 profile which means it's a pixel shader binary blob and not an "Effect" binary blob. It does not contain "techniques" or "passes".

You can use that binary blob to create a pixel shader directly without using the optional "Effects" system:

Here's a simple 'binary blob loader':

inline std::vector<uint8_t> ReadData(const wchar_t* name)
{
    std::ifstream inFile(name, std::ios::in | std::ios::binary | std::ios::ate);

    if (!inFile)
        throw std::exception("ReadData");

    std::streampos len = inFile.tellg();
    if (!inFile)
        throw std::exception("ReadData");

    std::vector<uint8_t> blob;
    blob.resize(size_t(len));

    inFile.seekg(0, std::ios::beg);
    if (!inFile)
        throw std::exception("ReadData");

    inFile.read(reinterpret_cast<char*>(blob.data()), len);
    if (!inFile)
        throw std::exception("ReadData");

    inFile.close();

    return blob;
}

Then you create the shader:

std::vector<uint8_t> blob = ReadData(L"data/shaders/BasicHLSL.fxc");

HRESULT errorID = w.getD3DDevice()->CreatePixelShader(blob.data(), &monochromeShader);

If you want to use the Effects system, you need to use the fx_2_0 profile:

fxc /T fx_2_0 /Fo BasicHLSL.fxc BasicHLSL.hlsl

See this post

Please note that Direct3D 9 is legacy, D3DX is deprecated along with the DirectX SDK itself, the fx profiles are deprecated in the modern versions of the HLSL compiler, the use of the "Effects" system generally is discouraged. See MSDN, Where is the DirectX SDK (2015 Edition)?, and Effects for Direct3D 11 Update

Answer 2

D3DXCreateEffectFromFile Documentation

The last parameter is to a LPD3DXBUFFER that holds compilation errors, which you can use to determine exactly why this compile (and any future similar compiles you may do) failed.

LPD3DXBUFFER errBuff = NULL;

D3DXCreateEffectFromFile (..., &errBuff);

if (errBuff)
{
    // contains the error string
    char *errStr = (char *) errBuff->GetBufferPointer ();

    // contains the size (in bytes)
    int errSize = errBuff->GetBufferSize ();

    // clean up
    errBuff->Release ();
}