OpenGL FBO to OpenCV image

Question

I am trying to figure out the best way to share an image between OpenGL and OpenCV libraries. I perform a render to texture on OpenGL, so I have and FBO/texture that then I want to pass it OpenCV, where (ideally) I will be some GPU stuff with it.

I know a naive way would be using glReadPixels, but this is obviously way to slow. It requires to copy the data from GPU memory to memory and then load it through cvLoadImage or similar in OpenCV.

I am pretty sure there must be a way to bind or map the memory from OpenCV, so I can "load" an image by just accessing to its FBO pointer or something like that... but I cannot figure out how.

I have goggled a lot about it, and so far I could just find this answer that I don't quite understand. What do they mean by glbuffer.bind();?

Any guess? Thanks.

Answer 1

The answer you are referring to seems to be this one:

glbuffer.bind();
unsigned char *dest = (unsigned char*)glbuffer.map(QGLBuffer::ReadWrite);
// creates an openCV image but the pixel data is stored in an opengl buffer
cv::Mat opencvImage(rows,cols,CV_TYPE,dest);  
.... do stuff ....
glbuffer.unmap(); // pointer is no longer valid - so neither is openCV image

What is done here is a mapping of the GL buffer memory into the virtual memory of the application. Thus you can write to the buffer or read from it as if it were any other memory you have a pointer to. Note that the code above uses Qt, but you can do this without Qt also.

By creating the OpenCV matrix (not the header, only the internal image data) within the mapped region, you can effectively read/write the region via OpenCV. There may be problems with stride, so be careful.

In general if you would want to transfer the buffer to OpenCV, you would first map the region, then build a OpenCV Mat header around it (like in the example code above) and finally call Mat::copyTo() to another Mat object. After releasing the mapping, you would still have the data from the buffer in that other object. It might very well be that for this operation you would need to set the CV_TYPE to sth. like 4-channel unsigned char (or whatever you have in your buffer) and then convert to BGR.