4
\$\begingroup\$

I am trying to figure out the best way to share an image between OpenGL and OpenCV libraries. I perform a render to texture on OpenGL, so I have and FBO/texture that then I want to pass it OpenCV, where (ideally) I will be some GPU stuff with it.

I know a naive way would be using glReadPixels, but this is obviously way to slow. It requires to copy the data from GPU memory to memory and then load it through cvLoadImage or similar in OpenCV.

I am pretty sure there must be a way to bind or map the memory from OpenCV, so I can "load" an image by just accessing to its FBO pointer or something like that... but I cannot figure out how.

I have goggled a lot about it, and so far I could just find this answer that I don't quite understand. What do they mean by glbuffer.bind();?

Any guess? Thanks.

\$\endgroup\$

1 Answer 1

2
\$\begingroup\$

The answer you are referring to seems to be this one:

glbuffer.bind();
unsigned char *dest = (unsigned char*)glbuffer.map(QGLBuffer::ReadWrite);
// creates an openCV image but the pixel data is stored in an opengl buffer
cv::Mat opencvImage(rows,cols,CV_TYPE,dest);  
.... do stuff ....
glbuffer.unmap(); // pointer is no longer valid - so neither is openCV image

What is done here is a mapping of the GL buffer memory into the virtual memory of the application. Thus you can write to the buffer or read from it as if it were any other memory you have a pointer to. Note that the code above uses Qt, but you can do this without Qt also.

By creating the OpenCV matrix (not the header, only the internal image data) within the mapped region, you can effectively read/write the region via OpenCV. There may be problems with stride, so be careful.

In general if you would want to transfer the buffer to OpenCV, you would first map the region, then build a OpenCV Mat header around it (like in the example code above) and finally call Mat::copyTo() to another Mat object. After releasing the mapping, you would still have the data from the buffer in that other object. It might very well be that for this operation you would need to set the CV_TYPE to sth. like 4-channel unsigned char (or whatever you have in your buffer) and then convert to BGR.

\$\endgroup\$
4
  • \$\begingroup\$ It's possible that this will still end up creating two copies of the data in video memory, and it might even end up transferring the data from the GPU to the CPU and back again. OpenCV uses CUDA internally, so it depends whether the graphics and CUDA drivers are smart enough to talk to each other and realize that the data is already in video memory. \$\endgroup\$ May 11, 2013 at 23:22
  • 1
    \$\begingroup\$ OpenCV does not use CUDA internally. OpenCV comes with a CUDA module, but you have to explicitely use it using gpu::Mat instead of cv::Mat. \$\endgroup\$
    – ypnos
    May 11, 2013 at 23:33
  • \$\begingroup\$ The OP said "I want to pass it OpenCV, where (ideally) I will be some GPU stuff with it." Presumably that means using the CUDA module. I'm not aware of any other GPU support in OpenCV, though admittedly I'm not that familiar with it. \$\endgroup\$ May 12, 2013 at 0:14
  • \$\begingroup\$ Right, I overlooked that. He would need to check a direct conversion to gpu::Mat. Relevant: answers.opencv.org/question/9512/how-to-bind-gpumat-to-texture \$\endgroup\$
    – ypnos
    May 12, 2013 at 2:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .