I'm trying to use NVENC to encode some frames that I'm capturing from a game. I've hooked EndScene() and I can get the data from the backbuffer, which is in ARGB format.
Unfortunately, NVENC only supports NV12 buffers ("resources") as input. I checked the SDK demo source and I've found that they create their NV12 surfaces using the fourcc code like so:
IDirectXVideoProcessorService * directx_services;
DXVA2CreateVideoService(d3ddev, IID_PPV_ARGS(&directx_services));
directx_services->CreateSurface(desc.Width, desc.Height, 0, (D3DFORMAT)MAKEFOURCC('N', 'V', '1', '2'), D3DPOOL_DEFAULT, 0, DXVA2_VideoProcessorRenderTarget, &NV12_surface[0], NULL);
This works. The problem now is that i need to convert my ARGB buffer into NV12. StretchRect() does not work, and CheckDeviceFormatConversion() returns D3DERR_NOTAVAILABLE, so it doesn't seem like this is possible using high-level API's.
One idea I had was to copy the backbuffer to a texture surface, and then use that texture as an input to a pixel shader when doing some dummy draw operation, and then inside the shader i can do some math to do the conversion from ARGB to NV12. But then i would have NV12 formatted data inside an ARGB render target again. Would i be able to "cast" that to a NV12 buffer, or just use it as an input to my encoder even though the format technically doesn't match, but the data does?
Any ideas?
