Why loading ~250mb of compressed texture data spills Out of memory error on Windows

Question

I am making SDL2/OpenGL 2D game for Windows with a lot of pre-rendered sprites. When testing on my laptop I got SDL_Surface creation failed: out of memory error during loading assets.

Windows' Application Manager on other machine shows that my game in stress uses less then 170 mb of ram memory and MSI Afterburner shows that vram usage for entire system is below 800 mb.

My laptop has 8 gb of RAM and 2048 mb of vram (GeForce 840 M).

I am obviously doing something terribly wrong, but I don't know wheres my ram usage counting technique is wrong, or maybe for some reasons I have less memory then available on system, or something completely else is happening. But I don't know where to search for.

I will gladly post some code if I need to, but I don't know which parts will be useful in diagnosing the problem.

One more thing, I am using GL_COMPRESSED_RGBA_ARB for all textures

Edited

It's not exactly the same situation as what's described here. I've already measured vram usage, and numbers are way below available count, yet still I get out of memory error. So, the question is: "How measure properly vram usage", not "How measure vram usage", as in other question. I am not a native English speaker, so maybe there is some cloud in my title - I will gladly accept correction for title

Edited

I reckon the image load procedure might be useful here, since it is heavily related to issue.

Game::Texture * Game::Asset::GetTex(const UString & path)
{
    Texture * ret = _assetsTex[path];
    if (ret)
        return ret;

    const UString & absolutePath = GetAbsolutePath(path);
    SDL_Surface * surface = IMG_Load(absolutePath.GetCStr());

    if (!surface)
        ERR(_U("Failed to load surface: #1#"), UString::FromA(IMG_GetError())); // Here's my problem!

    int bpp = surface->format->BytesPerPixel;
    UInt8 * pixels = (UInt8 *)surface->pixels;

    if (bpp != 3 && bpp != 4)
    {
        WARN(_U("Texture '#1#' is not 24 bpp or 32 bpp! Aborting load..."), path);
        SDL_FreeSurface(surface);
        return nullptr;
    }

    GLuint tex;
    glGenTextures(1, &tex);
    glBindTexture(GL_TEXTURE_2D, tex);

    if (bpp == 3)
        glTexImage2D(GL_TEXTURE_2D, 0, GL_COMPRESSED_RGBA_ARB, surface->w, surface->h, 0, GL_RGB, GL_UNSIGNED_BYTE, pixels);
    else if (bpp == 4)
        glTexImage2D(GL_TEXTURE_2D, 0, GL_COMPRESSED_RGBA_ARB, surface->w, surface->h, 0, GL_RGBA, GL_UNSIGNED_BYTE, pixels);

    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

    ret = new Texture(tex, surface->w, surface->h);

    SDL_FreeSurface(surface);

    _assetsTex[path] = ret;

    return ret;
}

Edited

I moved from SDL_Image to stb_image for image load, but I still got out of memory error message

Edited

I've installed VS Studio 2015 on my laptop and found out:

• Task Manager shows application eats up 650 mb in the moment of Out of memory casted
• GetProcessMemoryInfo returns PeakWorkingSetSize ~720 mb and WorkingSetSize 650 mb as well
• But Visual 2015 UI (Debug Diagnostic, memory usage) shows that application actually consumed 1.7 gb. Even if this is true and both former are not, there is still plenty of room for allocations (application is 32 bit, there are 8 gb of on-board memory, not counting page file)
• I tried to convert png -> tga and load them up, to check if png uncompressing is eating memory, but the result was exactly the same, the same Out of memory error
• Out of memory is spit in png load function which has nothing to do with OpenGL and GPU at all. I am retagging and retitleing the post

Answer 1

Firstly, why aren't you using GL_COMPRESSED_RGB_ARB for your textures which don't have an alpha channel? I'm not that familiar with OpenGL, but I think that should reduce their size in memory significantly.

By default on Windows 32-bit applications only get 2GB of address space. You can improve that so that when run on a 64-bit operating system they get 4GB of address space by using the /LARGEADDRESSAWARE linker option.

That won't solve your problem when your game is running on a 32-bit version of Windows though, and there's no magic setting to fix that. 2GB is the limit.

To get access to more than 4GB of address space you'll need to compile your program as as 64-bit.

You can use the VMMap tool to get some idea of where your address space is being used. Try using it before and after a texture is created for example, to make sure it's actually getting compressed. There's also a "Memory - Commit Size" column you can enable in Task Manager to show you the total address space usage.

If you're only loading 250MB of texture data, then that really shouldn't be consuming so much address space. Do you have any memory leaks in your code? Do you have more than one copy of any of the textures loaded?