Quickly Running Out Of Memory

Question

Over the past months I have been developing an application using Vulkan. Recently I have implemented loading 3d models from .fbx files and their associated .png textures. At this point there appears to be no performance issues until you look at the memory usage.

A single, 4096x4096 10mb .png file after loaded is spiking memory usage over 500mb. Loading an 80,000 polygon .fbx file is spiking memory usage over 2GB! during the loading process and then it levels down around 800mb. That is still 800mb for loading 1 model and 1 texture and this is just on the CPU.

On the GPU, which as considerably less available ram than the CPU, just by looking at task manager, is showing about the same amount of usage. If I load in more model+texture pairs I can see memory usage on the GPU's graph take large steps up after transfer operations complete.

At this rate I wont be able to have but only a small handfull of objects in an entire scene.

So what am I doing wrong here? Just what important factor have I completely overlooked when designing this application?

This is a learn-as-you-go project, and i'm proud of myself for getting this far, but this memory issue is absolutely dumbfounding and I haven't a clue.

If anyone want's to have a rummage around in my code the project is up on GitHub.

Answer 1

Generic advices.

Don't use png. It's good for intermediate image processing, acceptable in some test runs, but it's quite slow for loading and needs extra ram for decompressing.

Use the texture file formats that's very close to the data format that will be actually consumed by gpu - dds/ktx/pkm - whatever that supports your target hardware's texture compression (bc1/bc3/bc5/etc) and will allow you do read the binary dump directly without processing. You can pack that later with zlib as part of the game's resources archive.

Same applies to 3d model formats. If something is good for exchanging geometry between different modelling software - like FBX or Collada - then it's overly generic, it will require some (or lots) processing upon loading, and will fragment your memory.

And don't judge about your app's consumption by looking in the task manager. Use some profiling tool to actually track the allocations done by the app.