How can I load raw Bayer images using OpenGL?

Question

I want to load a raw Bayer format image and convert it to RGB using OpenGL.

So far I have played around with glTexImage2D() and loaded a Bayer image as a monochrome texture. My question is: how can I do per-pixel operations on this texture so that I can convert it to an RGB image?

Answer 1

This isn't a trivial task for someone new to graphics programming, I'm assuming you know how to write fragment shaders and sample textures.

To convert the Bayer image to an RGB texture you'll want to point sample the monochrome Bayer image, and do the interpolation in the fragment shader.

For each destination pixel, you'll know the value of one channel - the one for the colour of the filter over that pixel. For the other two channels you'll need to sample some of the surrounding pixels and interpolate. This gets more complicated for edge pixels, and for that reason the easy option is to throw away a pixel or two along each edge of the result.

Let's say the filter looks like this, where R, G and B represent the colour of the filter over the pixel:

RGRGRGRG
GBGBGBGB
RGRGRGRG
GBGBGBGB

For each R pixel, one way to calculate the G value is to take the average of the four pixels each side of it, as they all have a green filter. To calculate the B value you can do the same thing diagonally.

In pseudocode a basic conversion to RGB would look something like this:

if (x % 2 == 0 && y % 2 == 0)
{
  // Red pixel in filter
  result.r = bayer[x,y];
  result.g = (bayer[x+1,y] + bayer[x-1,y] + bayer[x,y+1] + bayer[x,y-1]) * 0.25;
  result.b = (bayer[x+1,y+1] + bayer[x+1,y-1] + bayer[x-1,y+1] + bayer[x-1,y-1]) * 0.25;
}
else
// TODO: Handle the other cases