4
\$\begingroup\$

I am looking to include images captured from the users webcam into my application so I've been looking into using OpenCV to capture from the webcam and then render it in Irrlicht. I've created a basic app to test and I'm doing this in my main loop to capture and convert to ITexture* to render onto a cube.

    if( cv::waitKey(50) >= 0 ) break;
    capture >> camera_frame;

    if( ! camera_frame.empty() )
    {
        unsigned char *tex_buf = (unsigned char*)frame_tex->lock();
        unsigned char *frame_buf = camera_frame.data;
        // Convert from RGB to RGBA
        for(int y=0; y < camera_frame.rows; y++) {
            for(int x=0; x < camera_frame.cols; x++) {
                *(tex_buf++) = *(frame_buf++);
                *(tex_buf++) = *(frame_buf++);
                *(tex_buf++) = *(frame_buf++);
                *(tex_buf++) = 255;
            }
        }
        frame_tex->unlock();

        //cube->setMaterialTexture(0, driver->getTexture("../../media/t351sml.jpg"));
        cube->setMaterialTexture(0, frame_tex);
        cube->setMaterialFlag(video::EMF_LIGHTING, false);

    }

...

It works.... BUT... the framerate drops to about 4fps.

Yes, it probably could have been done slightly more efficiently, but it's a massive drop in framerate for something so simple.

Is there another way to do this that won't kill the framerate?

Improve this question
\$\endgroup\$
4
  • \$\begingroup\$ Is it necessary for your texture to be RGBA since it will always be opaque (based on what I see in the provided code)? If not, you could get rid of the slow conversion. \$\endgroup\$
    – fastinvsqrt
    Jan 23, 2015 at 6:02
  • \$\begingroup\$ I tried memcpy but got distorted rubbish as the output so I went back to doing this conversion. It's my understanding that it's the lock() and unlock() methods that cause the slowness. \$\endgroup\$
    – Tim
    Jan 23, 2015 at 7:03
  • 2
    \$\begingroup\$ Assuming lock() gets the texture's pixels, then yes. Retrieving texture data from the graphics card is extremely slow. Is there some way for you to set the data for the texture without having to first retrieve it? Does Irrlicht have some kind of equivalent to a Pixel Buffer Object in OpenGL? \$\endgroup\$
    – fastinvsqrt
    Jan 24, 2015 at 5:51
  • \$\begingroup\$ What @fastinvsqrt said - round-tripping from video memory to system memory back to video memory will never be fast, and no amount of loop optimization is going to change that. You need to keep this operation as a video memory to video memory copy; not being familiar with the tools you use I can't advise further, but I expect that there must be a way. \$\endgroup\$
    – Maximus Minimus
    Apr 14, 2017 at 9:52

1 Answer 1

Reset to default
1
\$\begingroup\$

Try this (the suggested modifications are in the comments in the code):

if( cv::waitKey(50) >= 0 ) 
    break;
capture >> camera_frame;

if( ! camera_frame.empty() )
{
    unsigned char *tex_buf = (unsigned char*)frame_tex->lock();
    unsigned char *frame_buf = camera_frame.data;
    // Convert from RGB to RGBA
    for(int j=0; j < camera_frame.rows*camera_frame.cols; j++) {

        *(tex_buf) = *(frame_buf);
        *(tex_buf+1) = *(frame_buf+1);
        *(tex_buf+2) = *(frame_buf+2);
        *(tex_buf+3) = 255;
        frame_buf +=3;
        tex_buf +=4;
        // remove accumulator because when product with row*cols it huge time 
        // and merge x loop & y loop in one that remove more asm count operation  
    }
    frame_tex->unlock();

    //cube->setMaterialTexture(0, driver->getTexture("../../media/t351sml.jpg"));
    cube->setMaterialTexture(0, frame_tex);
    cube->setMaterialFlag(video::EMF_LIGHTING, false);

}
Improve this answer
\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .