# OpenGL : sluggish performance in extracting texture from GPU

I'm currently working on an algorithm which creates a texture within a render buffer. The operations are pretty complex, but for the GPU this is a simple task, done very quickly.

The problem is that, after creating the texture, i would like to save it. This requires to extract it from GPU memory.

For this operation, i'm using glGetTexImage(). It works, but the performance is sluggish. No, i mean even slower than that. For example, an 8MB texture (uncompressed) requires 3 seconds (yes, seconds) to be extracted. That's mind puzzling. I'm almost wondering if my graphic card is connected by a serial link...

Anyway, i've looked around, and found some people complaining about the same, but no working solution so far. The most promising advise was to "extract data in the native format of the GPU". Which i've tried and tried, but failed so far.

Edit : by moving the call to glGetTexImage() in a different place, the speed has been a bit improved for the most dramatic samples : looking again at the 8MB texture, it knows requires 500ms, instead of 3sec. It's better, but still much too slow. Smaller texture sizes were not affected by the change (typical timing remained into the 60-80ms range).

Using glFinish() didn't help either. Note that, if i call glFinish() (without glGetTexImage), i'm getting a fixed 16ms result, whatever the texture size or complexity. It really looks like the timing for a frame at 60fps.

The timing is measured for the full rendering + saving sequence. The call to glGetTexImage() alone does not really matter. That being said, it is this call which changes the performance.

And yes, of course, as stated at the beginning, the texture is "created into the GPU", hence the need to save it.

Edit 2 : I've also tried to use glReadPixels(), instead of glGetTexImage(). But it's worse unfortunately. Approximately twice slower.

Edit 3 : Here is the code. Note that it uses a "framework" that i've not developed. So some of the function calls here are not OpenGL standard. But they are nonetheless quite self-expressive.

glDisable(GL_BLEND);

// Simple construction, for test : the resulting texture is output into render buffer
renderbufx1.bind();
clear(0,0,0,0);
img.tex->drawPart(-1,-1,2,2,0,0,1,1);
renderbufx1.unbind();

// Not really useful, but just in case, the result is copied into another render buffer.
// renderbufx1 is going to be displayed, and i want to avoid any access conflict
savebufx1.bind();
clear(0,0,0,0);
renderbufx1.getTexture()->bind();
renderbufx1.getTexture()->drawPart(-1,-1,2,2,0,0,1,1);
renderbufx1.getTexture()->unbind();
savebufx1.unbind();

savebufx1.getTexture()->bind();
glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_BYTE, t.data);  // <-- This is the line
savebufx1.getTexture()->unbind();

glEnable (GL_BLEND);

renderbufx1.getTexture()->bind();
renderbufx1.getTexture()->drawPart(tex2x,-1,w,h,texSpan,texSpan,fact,fact);
renderbufx1.getTexture()->unbind();


Timing measured encompasses all this (and a few other things, which do not really contribute to the total).

Edit 4 : (on course for the most edited question of the week)

The native texture format of the GPU is supposed to be retrieved through the function glGetTexLevelParameter. More precisely, i've used this line of code :

glGetTexLevelParameteriv ( GL_TEXTURE_2D, 0, GL_TEXTURE_INTERNAL_FORMAT, &iTexFormat);


The result of the function (within iTexFormat) is 32856. 32856, decimal, is 0x8058, and within gl.h there is :

#define GL_RGBA8 0x8058

• How are you measuring that time? Remember that OpenGL calls are normally asynchrounous, but glGetTexImage() must wait until processing is done so that the result it reads back is correct. – Anton Dec 1 '12 at 11:39
• try to call glFinish, before glGetTexImage and then measure time of glGetTexImage – kravemir Dec 1 '12 at 15:22
• "after creating the texture, i would like to save it." Unless you're rendering to this texture, there's no reason to "save it". It stores exactly and only what you gave it. – Nicol Bolas Dec 1 '12 at 16:15
• @Cyan If you keep believing it’s “pretty obvious” you will never know what’s going on. Is the time spent doing userland computations? Flushing a framebuffer? Waiting for the GPU to finish something? In the kernel driver? A profiling tool will tell you that. – sam hocevar Dec 1 '12 at 20:57

First of all, it's worth mentioning that you should ensure you have the most up-to-date video drivers for your GPU installed.

You might be able to get better performance by using a Pixel Buffer Object (PBO) to read back your render target. A PBO can be created with a hint to the driver to locate the buffer in main memory; the GPU then transfers the render target contents to the PBO. Strangely enough, on some (many?) systems it's faster for the GPU to write to main memory than for the CPU to read video memory.

Details can be found in the article linked above, as well as the ARB_pixel_buffer_object extension spec, but in general, you'd do the following:

1. Create a PBO (as shown in the article) using the hint GL_STREAM_READ, which tells OpenGL you intend to use this PBO for reading back a render target to the CPU.
2. Bind it for "pack" and then do your glGetTexImage() call, which will copy the contents of the render target into the PBO.
3. Call glMapBuffer() to get access to the PBO memory.

Note also that with PBOs the glGetTexImage() call is asynchronous. That is, you can do additional work on the CPU or GPU between steps 2 and 3, while the memory transfer runs in the background. When you call glMapBuffer() it will wait for the transfer to finish, if it's not already done.

However, all this being done, you may or may not see a performance improvement. This is all down to the driver and how it decides to allocate memory, schedule work, etc. Unfortunately, working on a PC you don't have much control over this stuff.

• Wow, i initially thought that i was already doing that, but then, looking at the documentation you point at, it seems quite much more complex. Note also that the way performance is being benchmarked is not exactly friendly with this methodology. But well, let's have a look at it anyway. – Cyan Dec 1 '12 at 23:40
• @Cyan If you want to properly benchmark GPU operations (as opposed to CPU calls that queue up GPU operations), you should look into ARB_timer_query. – Nathan Reed Dec 2 '12 at 0:00
• @ Nathan Thanks, good point – Cyan Dec 2 '12 at 0:08
• Well, i may be out of luck. The website you recommend to read (and which is indeed very good) also has a test program to demonstrate and measure the effect of PBO. On my test system, turning PBO on or off doesn't change anything. Same speed. And btw, the speed is confirmed as being miserable, at just 1.2MPixels/s. That's well below my expectation. – Cyan Dec 2 '12 at 0:11
• Follow up : my dev station was just updated, from WinXP-32 to Win7-64 (corporate managed). I immediately tried the PBOTest application above. The results are radically changed : now texture transfer is down to 0.4ms, while it was 20ms previously. It's a 50x increase. Both drivers were up to date, but apparently, only Win7 ones were properly optimised. It's also interesting to note that, in both cases, PBO or not PBO, it made no difference at all. – Cyan Dec 18 '12 at 23:56
glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_BYTE, t.data);


This reads the image data back as a series of signed bytes. So unless your image is using the GL_RGBA8_SNORM internal format, this is going to require CPU conversion. And you don't want that.

You should be reading it back as GL_UNSIGNED_BYTE if your image format is GL_RGBA8. Also, try changing the pixel transfer format to GL_BGRA to see if that helps.

• GL_BYTE is indeed, what i want. Anyway, i also made some tests with GL_UNSIGNED_BYTE, but it had no impact on speed. – Cyan Dec 2 '12 at 16:30
• I also tried your suggestion to ask for GL_BGRA instead of GL_RGBA. Same result : speed remains unaffected. – Cyan Dec 2 '12 at 16:31
• @Cyan: "GL_BYTE is indeed, what i want." Why? Do you really want to do CPU-conversion of normalized unsigned bytes to normalized signed bytes? – Nicol Bolas Dec 2 '12 at 16:33
• because the value created can be negative, and the property of "division towards zero" is useful for my use case. Anyway, that's not really the point here. I've tried both, signed and unsigned (by modifying the fragment shader part). They both feature the exact same speed. – Cyan Dec 2 '12 at 16:48
• @Cyan: What is your texture's internal format? Because there's no way that values in a GL_RGBA8 image can be negative. Hence why reading signed values from an unsigned format makes no sense. – Nicol Bolas Dec 2 '12 at 16:48