Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

I'm developing some algorithms in Lua which are run mostly in Lua (few calls to C++) and I'm noticing a huge slowdown when I'm running them from my app rather than the default Lua interpreter. The script returns in about 11 seconds with the interpreter, and about 5+ minutes with my program.

I don't think it is a problem with calling the C++ function, overruling that function with an empty function in the script has no noticeable effect for time.

Both Lua and the program are compiled with Visual Studio 2010 (I created a new solution for Lua, with projects for both the interpreter and static library)

I've run a profiler on the code (the Very Sleepy profiler, can't get a Lua profiler working but haven't tried to much) and found that in my application around 50% of the time is spend on malloc.. and 40% on free, both called from the Lua garbage collector (tried disabling that to check, but that crashes with using to much memory (understandably), the Lua interpreter peeks at about 4mb ram usage).

I'm new at Lua, so its very possible I've done something wrong somewhere, anyone got a few hints to try?

share|improve this question
    
malloc! The root of all evil! We ran into a similar problem when first investigating Lua and simply abandoned the effort, so I'm curious to see if there's a solution. –  Crashworks Oct 15 '10 at 22:32
    
Wasn't there a suggestion to change the default memory handler for lua to one that uses realloc ... if I remember correctly this was said to be much faster in some cases. But that was ~3 years ago. How do you bind Lua? If you use Luabind, except quite a significant function call overhead compared to directly binding methods, or probably compared to toLua as well. –  LearnCocos2D Oct 15 '10 at 23:03
    
Have you tried the recommendations in optimizing lua's GC? lua-users.org/wiki/OptimisingGarbageCollection –  David Young Oct 15 '10 at 23:46
    
@DavidYoung I've tried optimising it a bit, got it about 15% faster, on both, so nothing that would really explain the large difference in speed –  Kevin van der Velden Oct 16 '10 at 10:17
    
@GamingHorror looking back at the call stacks it would seem that malloc is getting called from realloc (luaV_execute->luaM_realloc_->l_alloc->realloc->malloc), I also use the Lua C api directly, with a bit of light user data for the one class that needs to be called. –  Kevin van der Velden Oct 16 '10 at 10:22
add comment

1 Answer

up vote 3 down vote accepted

As you say, Lua does memory allocations like crazy by default. You should consider writing a custom allocator that has better knowledge of the allocation patterns typically used by your Lua programs, or plugging in something like tcmalloc for better allocation performance all around.

share|improve this answer
    
I've tried that, it does save a bit of time (couple of seconds, still nowhere near the standalone interpreter), maybe the problem (or one of them) is that allocation time isn't constant and depends on how much memory has been reserved before? That would be solvable by using a pool of memory just for Lua, I'll give that a shot (anyone got some hints on implementing that?). –  Kevin van der Velden Oct 17 '10 at 17:36
    
What saves a couple of seconds, tcmalloc? The reason malloc is generally slow isn't because allocation time is not constant (though it often is not) - it's because allocation has horrible cache locality, lots of locking overhead, tons of consistency checking in debug builds, etc. Basically, you should never be calling real malloc (and especially not real free!) during script execution - always use an existing pool of memory as much as you can. –  user744 Oct 17 '10 at 17:57
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.