Why do separate loops run faster than one loop that does multiple things?

Question

My game had some lag, so I tried to improve the performance.

I found that if I write my loops separately like this:

for (...){
    for (...){
        // Do operation A 
        // (Fill int[][] with 1s)
    }
}

for (...){
    for (...){
        // Do operation B 
        // (Work with Sprite Renderers & Spritebatch)
    }
}

for (...){
    for (...){
        // Do operation C
        // (Work with Shape Renderers with alpha)
    }
}

then the result runs a whole lot faster than if I try to do all the operations in one loop:

for (...){
    for (...){
        // Do operation A 
        // Do operation B
        // Do operation C
    }
}

Can anyone explain this?

I'm running this game on Android OS!

Answer 1

This is not an unexpected result actually. It's a little like how a person's efficiency plummets when they try to multi-task, compared to when they can focus on doing one specific job.

Modern CPUs in computers and phones are vastly faster than the memory they're connected to, so there's elaborate caching and pre-fetching machinery set up to try to stuff the cache full of what the CPU might need soon, so that it doesn't have to wait on data to arrive all the way from main memory. This article on data locality from Game Programming Patterns explains the situation very well.

When you're in a tight inner loop that does just one thing, you make it very easy for these caching systems to predict what the CPU needs and keep ahead of it. For instance, iterating over an integer array in order, your CPU will work through sequential memory addresses, working on every bit of data in each cache line before moving onto the next one. That's the ideal access pattern for these systems

As your loop grows more complicated, your CPU has to switch between doing multiple different tasks, asking for stuff from different areas of memory, and possibly evicting data or instructions from the cache that it will need again when it returns to "operation A" to make room for data/instructions it needs to work on "operation C." This makes it harder for the caching systems to stay ahead of the CPU, and it might stall waiting for main memory from time to time.

So sometimes, letting the CPU do a batch of one type of work uninterrupted, then do a batch of a second kind of work, can help it do both faster than if it were trying to interleave the two.

The order of the nested loops matters too, for the similar reasons. Iterating over a 2D/jagged array [x][y] in the order for ( y...) { for ( x...) { ... } } means grabbing the first entry of the first inner array, then jumping way down the memory address space to the first entry of the second inner array, instead of working on the rest of the data from the first array that got pulled in on the same cache line. That data will probably be evicted from the cache by the time we come back to look at the second entry from the first inner array, forcing us to request it from main memory again...

This is one scenario that could lead to the profiling results you're seeing. But we won't know for sure unless you do some deeper profiling to count things like cache misses. As mentioned in the comments above, this could also be due to coincidence - profiling is tricky, and it takes controlled setups and a lot of repetition to be sure of the data.

Answer 2

After some deep research and a lot of testing finally found out that the problem wasnt the loop as because it works on a simple maths. but the problem was in the rendering batch, the more batch there is, java will likely to share more memory to each batch as because batch is a collection of vertices. i divided them because i have to batch.begin() and batch.end() every batch, and it runs better. thanks all in helping me today. P.s sorry for bad english XD