1
\$\begingroup\$

I'm currently working on an image processing project that pans and zooms on multi-layer high resolution images.

The goal is colorize (including color, gain and noise) n layers (n having a minimum of 5) and merge them via shaders. On top of that, there are 5–6 layers of 2D geometry each colored and merged as well.

The normal image size is 7440 x 7440.

These are then merged and rendered on a single Quad. I've been able to get this working very well and the results are good.

This is being processed on a Xeon based workstation with a M4000 Nvidia Quadro or a Xeon based Lenovo p51 with M2200 Nvidia Quadro.

The issue comes when I need to process a very large image (34,992 x 34,992) that requires splitting the image over 9 quads on the same systems.

These images must go through the same process of colorization and merging. The images are 11,664 x 11,664 per quad.

When zoomed in, we are still processing areas we can't see—can I apply some form of frustum culling here?

And, when zoomed out, is there a way to process a lower resolution image so the GPU is not pegged and none responsive?

\$\endgroup\$
1
\$\begingroup\$

Instead of merging everything into a large temporary texture if you render all the temporaries and 2D mesh to a frame buffer the same size and render-offset as your final window the GPU would be culling all the unseen pixels on its own.

But I recommend switching to OpenCL. You can better split down the tasks into smaller GPU commands (kernel executions) and then add artificial software-controlled delays to let the GPU do other work by using clFinish and putting your thread to sleep for a hundred milliseconds or so before issuing more commands to the queue. This would give the rest of the system a chance to access the GPU.

You can fine-tune this by profiling the time those OpenCL kernel executions take on a particular system to balance the pause and execution queue length.

But with OpenCL you will need to calculate the frustum culling yourself from the zoom factor and scrolling offset.

With the cl_khr_gl_sharing extension or cl_khr_dx9_media_sharing, cl_khr_d3d10_sharing, or cl_khr_d3d11_sharing you can put the result directly into a GPU texture without doing a round-trip over the PCIe bus, leaving everything in GPU RAM.

With this more complex system you can even use OpenCL to calculate into one texture (or set of smaller tile textures) while the GPU is also drawing using another previous result texture-set during the software-controlled OpenCL pauses making the app feel more responsive.

As an extra step, you can watch for mouse movements and app focus to dynamically re-tune the pause/work ratio for maximum performance or improved responsiveness.

A side-effect of using OpenCL is that GPU drivers are more likely to be tuned to give priority to render commands (OpenGL or DirectX) than OpenCL commands. Using draw calls to do your processing puts those commands on the same priority (user-responsiveness) as other graphic draw calls. It is not guaranteed but using OpenCL can help the GPU drivers maintain system responsiveness.

\$\endgroup\$
5
  • \$\begingroup\$ OpenCL is not an option. SharpDX is use through out the existing code base, so SharpDX it is. Could you please elaborate on the "frame buffer." \$\endgroup\$
    – user96556
    Oct 24 '17 at 18:56
  • \$\begingroup\$ To render to an offscreen frame buffer (surface / texture) create a texture using BindFlags.RenderTarget you should then be able to use d3dContext.OutputMerger.SetTargets to render to this texture. But it is difficult to manage GPU usage and responsiveness with pure DirectX calls. You can also mix OpenCL and SharpDX. Use OpenCL to blend your texture buffers and do everything else with SharpDX. They are not exclusive to one another. \$\endgroup\$ Oct 24 '17 at 20:37
  • \$\begingroup\$ Another issue with using the GPU's graphic API exclusively is that long shaders can trigger the windows driver watchdog timer. Shaders are very limited in execution time and if they hang graphics for too long windows will kill and reload the graphic drivers. \$\endgroup\$ Oct 24 '17 at 20:41
  • \$\begingroup\$ docs.microsoft.com/en-us/windows-hardware/drivers/display/… \$\endgroup\$ Oct 24 '17 at 20:43
  • \$\begingroup\$ I'm looking at switching to compute shader or a more restrictive and less resource heavy model. In a super tight window of time... So my time to come up to speed on OpenCL is very limited. \$\endgroup\$
    – user96556
    Oct 24 '17 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.