Run Simple HLSL Pixel Shader with DirectX 9

Question

I'm trying to run HLSL Pixel Shaders with DirectX 9 where I feed a texture in, run the HLSL code and copy the resulting texture back from memory. I think I'm getting pretty close, although a LOT of time could be wasted trying to figure out the details. It's kind of pointless to study the whole DirectX SDK just for doing this basic task; I just want to get it done. Could you take a look at this code, give pointers as to what's missing, and answer a few questions?

Fist here are my questions, then followed by the code I have so far

1. Am I on the right track?
2. Am I better to pre-compile the shaders with fxc.exe or compile them on-the-fly? If I pre-compile, how do I load a pre-compiled shader?
3. In Diff.hlsl shader below, certain data are defined with "register" (sampler s0 : register(s0);) What are those and how do I configure them?
4. Can HLSL shaders alter the size of the texture? If so, how do I get the size back to parse the texture data accordingly?
5. How do I configure the parameters and run the shader?
6. Will I run into performance issues running this code? It gets called to process a video with AviSynth, where GetFrame gets called for each video frame. 8 instances will run on 8 parallel threads.
7. Anything else I should know?

I'm copying the texture data back and forth between AviSynth and the shader texture. The only difference in the memory structure is that AviSynth has a padding to the right of each line which has to be removed when copying to the shader and added back when copying back to AviSynth.

Here are some of the shaders I'm trying to run

https://github.com/zachsaw/MPDN_Extensions/blob/master/Extensions/RenderScripts/SuperRes/Diff.hlsl

https://github.com/zachsaw/MPDN_Extensions/blob/master/Extensions/RenderScripts/SuperRes/SuperResEx.hlsl

Here's my code: Shader.cpp

#include "Shader.h"

Shader::Shader(PClip _child, IScriptEnvironment* env) :
GenericVideoFilter(_child) {
    if (vi.IsPlanar() || !vi.IsRGB32())
        env->ThrowError("Source must be float-precision RGB");

    InitDirectX(env);
}

void Shader::InitDirectX(IScriptEnvironment* env) {
    dx = Direct3DCreate9(D3D_SDK_VERSION);

    // Initialize DirectX9
    D3DDISPLAYMODE dxMode;
    dx->GetAdapterDisplayMode(D3DADAPTER_DEFAULT, &dxMode);
    D3DPRESENT_PARAMETERS dxSettings;
    ZeroMemory(&dxSettings, sizeof(dxSettings));
    dxSettings.Windowed = TRUE;
    dxSettings.SwapEffect = D3DSWAPEFFECT_DISCARD;
    dxSettings.BackBufferFormat = dxMode.Format;
    dxSettings.EnableAutoDepthStencil = TRUE;
    dxSettings.AutoDepthStencilFormat = D3DFMT_D16;
    dxSettings.PresentationInterval = D3DPRESENT_INTERVAL_IMMEDIATE;
    dx->CreateDevice(D3DADAPTER_DEFAULT, D3DDEVTYPE_NULLREF, NULL, NULL, &dxSettings, &dxDevice);

    // Create pixel shader
    LPD3DXBUFFER dxShaderCode;
    LPD3DXBUFFER errors = NULL;
    if (FAILED(D3DXCompileShaderFromFile("F:\AVSMeter\MPDN_Extensions-master\Extensions\RenderScripts\SuperRes\Diff.hlsl", NULL, NULL, "main", "ps_3_0", 0, &dxShaderCode, &errors, &dxShaderConstantTable))) {
        env->ThrowError((const char*)errors->GetBufferPointer());
        errors->Release();
    }
    dxDevice->CreatePixelShader((DWORD*)dxShaderCode->GetBufferPointer(), &dxShader);
    dxShaderCode->Release();
}

Shader::~Shader() {
    if (dxDevice != NULL) {
        dxDevice->Release();
        dxDevice = NULL;
    }
    if (dx != NULL) {
        dx->Release();
        dx = NULL;
    }
    if (dxShader != NULL) {
        dxShader->Release();
        dxShader = NULL;
    }
    if (dxShaderConstantTable != NULL) {
        dxShaderConstantTable->Release();
        dxShaderConstantTable = NULL;
    }
}

PVideoFrame __stdcall Shader::GetFrame(int n, IScriptEnvironment* env) {
    PVideoFrame src = child->GetFrame(n, env);

    // Copy data to texture buffer
    unsigned char* textureBuffer = (unsigned char*)malloc(vi.width * vi.height * 16);
    CopyToTexture(src->GetReadPtr(), src->GetPitch(), textureBuffer);

    // Create texture
    LPDIRECT3DTEXTURE9 dxTexture = NULL;
    HANDLE* dxTextureHandle = (HANDLE*)textureBuffer;
    dxDevice->CreateTexture(vi.width, vi.height, 1, 0, D3DFMT_A32B32G32R32F, D3DPOOL_SYSTEMMEM, &dxTexture, dxTextureHandle);

    // Run shader on texture
    dxDevice->SetTexture(0, dxTexture);
    dxDevice->SetPixelShader(dxShader);

    // Copy data back to frame
    CopyFromTexture(textureBuffer, src->GetWritePtr(), src->GetPitch());

    // Release texture
    dxTexture->Release();
    dxTexture = NULL;
    free(textureBuffer);
    return src;
}

void Shader::CopyToTexture(const byte* src, int srcPitch, unsigned char* dst) {
    for (int i = 0; i < vi.height; i++) {
        memcpy(dst, src, vi.width * pixelSize);
        src += srcPitch;
        dst += vi.width * pixelSize;
    }
}

void Shader::CopyFromTexture(const byte* src, unsigned char* dst, int dstPitch) {
    for (int i = 0; i < vi.height; i++) {
        memcpy(dst, src, vi.width * pixelSize);
        src += vi.width * pixelSize;
        dst += dstPitch;
    }
}

Shader.h

#include <windows.h>
#include <d3d9.h>
#include <d3dx9.h>
#include <cstdio>       //needed by OutputDebugString()
#include "avisynth.h"   //version 5, AviSynth 2.6 +

class Shader : public GenericVideoFilter {
public:
    Shader(PClip _child, IScriptEnvironment* env);
    ~Shader();
    PVideoFrame __stdcall GetFrame(int n, IScriptEnvironment* env);
private:
    void InitDirectX(IScriptEnvironment* env);
    void Shader::CopyToTexture(const byte*, int srcPitch, unsigned char* dst);
    void Shader::CopyFromTexture(const byte* src, unsigned char* dst, int dstPitch);
    LPDIRECT3D9 dx = NULL;
    LPDIRECT3DDEVICE9 dxDevice = NULL;
    LPDIRECT3DPIXELSHADER9 dxShader = NULL;
    LPD3DXCONSTANTTABLE dxShaderConstantTable = NULL;
    const int precision = 4;
    const int pixelSize = 16;
};

Answer 1

Since this question has been viewed 1000 times, I might as well post a follow-up.

This is the code I ended up with, processing Avisynth video scripts through pixel shaders. https://github.com/mysteryx93/avisynthshader

2. I'm precompiling shaders because some shaders (Bicubic.hlsl) take a long time to compile, and you get all the errors at compile-time.

3. If I remember correctly, shader output size is set to the size of the output texture we're rendering to.

4. The performance bottleneck is in transfering GPU memory back to the CPU, as graphic cards are not designed for that. I got best performance by lining up all the shaders into a processing chain, and transfer only the result after processing them all at once. I'm also getting best performance by running 8 threads on 2 DirectX engines.