Best way of writing pixel manipuliting-intensive applications with OpenGL/Direct3D

Question

Recently I have been making little experiments with engines similar to old Wolfenstein 3D, Doom and Build, engines where the 3D rendering is entirely done in software and therefore you need full access to the screen buffers at pixel level.

In the days of DOS and before we had current GPUs it was relatively straightforward to get a surface to plot pixels. You had to devise the fastest way to do your calculations and the fastest way of updating the framebuffer. In fact, it is still possible by using libraries like SDL, which still give you a reasonable speed when you manipulate surfaces at pixel level.

So I wonder which would be the best way of dealing with pixel-intensive applications like the ones I suggest using current GPUs, by using OpenGL or Direct3D.

I understand this sounds absurd, but I have been looking at API documentation and it seems that it is a feature that we have somewhat lost in current hardware. I see suggestions as using glDrawPixels or using pixel shaders, but no solid explanation or theory which can help me to make the right decisions.

Answer 1

I understand that you are at a loss. I also liked to hack pixels when I was young, and did some games on calculator devices, or dos era this way.

This can still be done today, but, because of various hardware history of evolutions, it has become more involved.

CPU pixel pushing

Notably, we can still push pixels one by one, but it would be incredibly slow because the frontbuffer is a memory that is very far away nowadays. In current PC architectures we can't just write to some memory zone and get pixels displayed, because of hardware reasons. We have double buffering, we have write combining buffers, we have synchronization issues between chips, we have bus frequency and width issues, driver interposition...

What you need to do to push pixels today is accept that fact or hardware life, and draw into your own personal surface in the general heap memory of your process, make it a new texture every frame and upload it to the graphic card. This is how movie playing works and it works well enough.

If you use OpenGL 3+ core, or Direct3D 10+, you will be forced to use a technique that is hardware happy, which means, write combining safe, and pipelinable.
It takes the form of multiple buffering, your surface in RAM will get copied to a "shared" surface on some special VRAM location that the CPU can lock to access without disturbing the rendering by the GPU (D3D10_USAGE_DYNAMIC), of the current or previous frame (glMap/glUnmap). In the next frame, the data copy hopefully will be finished and the gpu will flush the new frame command buffer, inside of which will be contained the instruction to copy the shared zone to a hard zone in order to use as a texture (d3dcontext::CopyResource or glCopyTexSubImage2D). Which will be streamed on GPU by the texture units into a screen quad (tex2d in a pixel shader.).

some link to consider carefully: http://eatplayhate.me/2013/09/29/d3d11-texture-update-costs/.

This is how you can create a texture streaming pipeline that will not take much resource, and your CPU power can be used to push the pixels. Actually it will be limited by RAM bus speed, which is about 20/30 GiB/s nowadays, so you can push 4k*3k res at about 670 FPS.

There are wonderful articles about how to optimize the creation of software renderers, particularly rasterizers, its fascinating. With the history of scanline techniques and perspective correction issues (division was a slow hardware operation, and still is to some extent). Get the whole story here, after reading it all you will be a much better graphics engineer: https://fgiesen.wordpress.com/2013/02/17/optimizing-sw-occlusion-culling-index/
The man is Fabian Giesen, the guy from Farbrausch, a big name.

GPU pixel pulling

The second big paradigm that you can apply here, is forget totally about the Amiga500 way of drawing things, and use the GPUs to their full processing power, and code everything as part of a pixel shader. If you don't use unordered access writes, you will be forced into the mold of "parallel gather" collective operation. (MPI terminology)

This is a much more powerful paradigm because of the non-dependency on your neighbors, which means the hardware can be parallel, and schedule units to pixels freely how it sees fit. (of course there are some contrived details when going deeper (like inter-fragment-quad approximate communication using ddx ddy). But basically you are given a main function into which you can decide the color of the pixel that is given to you, but you don't decide where to write this pixel, it has fixed coordinate. So you gather data around you to decide this color. Its the other way around. You cannot make an iterating Bresenham algorithm for instance. Rather, to draw a line, you'd have to determine if your pixel is on the line and write black it yes, discard if no.)

This has its constraints but its a happy playground up shaderland. I recommend :)

Compute

Finally you can use compute, in computeland you can use the GPU as a multi core CPU and write where you want, to the cost of interlocking operations, this can get quite difficult to understand without serious experience in concurrent computations. This is why doing shaders first is a good idea to get used to it progressively.

The freedom you'll gain compared to shader is access to non filtered memory, and more hardware control on the low level. Where you put data, registers, unit, global...

Answer 2

I'm also from a pixel-banging background. PEEK and POKE anyone?

Using a GPU is a different mind-set entirely. (Well, it's still code, so just "more of the same...)

The GPU is really good at certain things: Processing individual pixels (through textures and fragment shaders), and filling triangles with pixels.

Much effort (on the programmer's part) is needed to get things set up. Much of the code involves creating buffers and textures to send to the GPU.

A trivial starting step would be to continue drawing your pixels, and pushing it to the GPU as one big textured rectangle, or rather, two triangles.

As you wrap your head around fragment shaders (which push individual pixels) and vertex shaders (which transform triangles mathematically around the screen) you'll begin to see opportunities to move some of your rendering work into these shaders.

There's plenty of example code around for open GL; here's one set of tutorials:

http://www.opengl-tutorial.org

To get your first rectangle of pixels on the screen through OpenGL, you'll use a combination of "Drawing a triangle" and "Drawing a textured cube".

OpenGL has a sort-of long runway, but it's pretty fun and powerful once you're airborne.

Hope that helps a little!