Frameskipping in Android gameloop causing choppy sprites (Open GL ES 2.0)

Question

I have written a simple 2d platform game for Android and am wondering how one deals with frame-skipping? Are there any alternatives? Let me explain further.

So, my game loop allows for the rendering to be skipped if game updates and rendering do not fit into my fixed time-slice (16.667ms). This allows my game to run at identically perceived speeds on different devices. And this works great, things do run at the same speed.

However, when the gameloop skips a render call for even one frame, the sprite glitches. And thinking about it, why wouldn't it? You're seeing a sprite move say, an average of 10 pixels every 1.6 seconds, then suddenly, there is a pause of 3.2ms, and the sprite then appears to jump 20 pixels. When this happens 3 or 4 times in close succession, the result is very ugly and not something I want in my game.

Therfore, my question is how does one deal with these 'pauses' and 'jumps' - I've read every article on game loops I can find (see below) and my loops are even based off of code from these articles. The articles specifically mention frame skipping but they don't make any reference to how to deal with visual glitches that result from it.

I've attempted various game-loops. My loop must have a mechanism in-place to allow rendering to be skipped to keep game-speed constant across multiple devices (or alternative, if one exists)

I've tried interpolation but this doesn't eliminate this specific problem (although it looks like it may mitigate the issue slightly as when it eventually draws the sprite it 'moves it back' between the old and current positions so the 'jump' isn't so big.

I've also tried a form of extrapolation which does seem to keep things smooth considerably, but I find it to be next to completely useless because it plays havoc with my collision detection (even when drawing with a 'display only' coordinate - see extrapolation-breaks-collision-detection)

I've tried a loop that uses Thread.sleep when drawing / updating completes with time left over, no frame skipping in this one, again fairly smooth, but runs differently on different devices so no good. And I've tried spawning my own, third thread for logic updates, but this, was extremely messy to deal with and the performance really wasn't good. (upon reading tons of forums, most people seem to agree a 2 thread loops ( so UI and GL threads) is safer / easier).

Now if I remove frame skipping, then all seems to run nice and smooth, with or without inter/extrapolation. However, this isn't an option because the game then runs at different speeds on different devices as it falls behind from not being able to render fast enough.

I'm running logic at 60 Ticks per second and rendering as fast as I can.

I've read, as far as I can see every article out there, I've tried the loops from My Secret Garden and Fix your timestep.

I've also read:

Against the grain

deWITTERS Game Loop

Plus various other articles on Game-loops. A lot of the others are derived from the above articles or just copied word for word.

These are all great, but they don't touch on the issues I'm experiencing. I really have tried everything I can think of over the course of a year to eliminate these glitches to no avail, so any and all help would be appreciated.

A couple of examples of my game loops (Code follows):

From My Secret Room

    public void onDrawFrame(GL10 gl) {

        //Rre-set loop back to 0 to start counting again
       loops=0;

       while(System.currentTimeMillis() > nextGameTick && loops < maxFrameskip) {
           SceneManager.getInstance().getCurrentScene().updateLogic();
           nextGameTick += skipTicks;
           timeCorrection += (1000d / ticksPerSecond) % 1;
           nextGameTick += timeCorrection;
           timeCorrection %= 1;
           loops++;
       }

       extrapolation = (float)(System.currentTimeMillis() + skipTicks - nextGameTick) / (float)skipTicks;
       render(extrapolation);
}

And from Fix your timestep

double t = 0.0;
double dt2 = 0.01;  
double currentTime = System.currentTimeMillis()*0.001;
double accumulator = 0.0;
double newTime;
double frameTime;

@Override
public void onDrawFrame(GL10 gl) {

    newTime = System.currentTimeMillis()*0.001;
    frameTime = newTime - currentTime;
        if ( frameTime > (dt*5)) //Allow 5 'skips'
            frameTime = (dt*5);
    currentTime = newTime;

        accumulator += frameTime;

        while ( accumulator >= dt )
        {
            SceneManager.getInstance().getCurrentScene().updateLogic();
            previousState = currentState;
            accumulator -= dt;


        }

        interpolation = (float) (accumulator / dt);

        render(interpolation);
}

Answer 1

From what I see, the game loops you are trying to use are somewhat specialized. I don't know why you chose those ones, but in my opinion you are trying to solve problems that don't exist in the first place.

Instead of copy-pasting a game loop from some site, I suggest you think about how your game is made and what your priorities are, and then create a game loop based on that.

In general terms, I like to visualize a game as composed of three elements:

• The game state is all the data in your game. Positions, angles, files and so on. No programs, just data.

• The update logic is a program that takes inputs (such as time, user input, network input and so on), and modifies the game state based on such inputs.

• The display logic is a program that takes a game state and without modifying it, displays it to the user.

If you can split your game into these three elements, then you're pretty much on the other side.

It is now depending on what does each one of those elements look like, and what your priorities are, to make a game loop that works for you. But first, the bad news:

The only way to guarantee that your game will run at a desired framerate all the time, is to make sure that:

• It takes much less to do an iteration of the update logic, than the time it is supposed to simulate (for example: simulating one second of game time takes much less than one second)

and

• The display logic takes much less than the framerate you want to display the game at.

If you can't make these guarantees, then you will have to make compromises.

A common idea in modern games is that it is more acceptable for the display logic to fall behind, than for the update logic to fall behind. That means that, if for example a frame takes longer to display than usual, the game state should still be running in real time, even if some frames are dropped.

A starting point for a game loop that does this could look like this (in pseudocode)

current_time = 0
previous_time = 0
game_logic_step = 1 / 100 -- doesn't really have to be the same as the drawing framerate

while running do
    current_time = time()

    while previous_time <= current_time 0 do
        update(game_logic_step) -- one step of update logic
        previous_time = previous_time + game_logic_step
    end
    display() -- display logic. As long as the game state doesn't get changed here, we're okay
end

With this, you're drawing as fast as you can, and you're making sure that the game state is always up to date.

This game loop is pretty robust, and is more than enough for 90% of the games I've ever worked on, and I am willing to guess that it will be more than enough for your game.

However, if showing all of the frames is more important than keeping the game state up to date (as in most older console games), then your game loop could look more like this:

current_time = 0
next_time = 0
draw_logic_step = 1 / 60

while running do
    while current_time <= next_time do
        wait_some_time()
        current_time = time()
    end
    next_time = current_time + draw_logic_step
    update(draw_logic_step)
    display()
end

Notice that we are fixing one update() call for each draw() call. With such a game loop, you will be showing all frames, while capping the frame rate. However, if one iteration of the loop takes more than draw_logic_step, then the game will slow down (but no frames will be dropped).

Those two game loops should suffice for the vast majority of games. Variations with interpolations, extrapolations, multiple threads and the like, are in my experience used only to optimize for corner cases where you want to push the update logic to the limit, yet keep the game running as smooth as possible.

Doing such optimizations though, is complex, and will make the game much more difficult to write and debug. In my opinion, you should start with a simple game loop as above, and you will see that it will work pretty well.

In fact, if your game is simple enough so both the update and display logics are really fast, then either of those two loops will work perfectly with no dropped frames or game slowdowns.

If your game is simple enough, and you are still dropping frames regardless of the game loop you use, then I would be inclined to think that the problem is not within the game loop but on your actual processing. What I would do is remove the fancy gameloop, use a simple one, and focus on understanding why are you skipping frames and fix that instead. Unfortunately, this changes the discussion from gameloops to how to debug and fix your code.

Some things that may help you are:

• On a comment, you mentioned that your update logic is taking 4ms. For 60fps, that is quite a lot, as it's about 1/4th of all the time you have for each frame. 4ms is not borderline high, but you may want to ask yourself why is it taking so much.

• You also mention vsync. In practical terms, Vsync means that your present call (probably swapBuffers()) will stall until it reaches a refresh point. I wouldn't worry about vsync too much, because:

  • If vsync is making your program stall for a whole frame, it is because you already missed the previous frame drawing deadline. Removing vsync won't solve this, and you still have to make your program run faster.

  • If your logic is slightly more than 1/60th of a second, vsync will manifest itself as a sudden framerate drop from exactly 60fps to exactly 30fps. That is not your case, so I wouldn't focus on vsync.

• Remember that your program is written in Java, which is a garbage collected interpreted language. Even if you do everything right, garbage collection cycles will stop your program completely for a very noticeable length of time, and you have no control over this. If you want to have more predictable performance, you should consider writing your game with the JNI in C. I've heard Xamarin.Android has better performance, if C# is your thing.

• But even so, remember that your program is running on a mobile processor in a multitasked environment. The OS will throttle the CPU, and will yield the CPU to other processes, meaning that the performance of your game is even less predictable. If you want complete predictability, write games for consoles instead.

• In the grand scale, I wouldn't worry too much about the problem you're facing. I would dedicate much more time to making sure the game is fun, because that's what really matters. I don't think that a fun game will stop being fun because of an occasional dropped frame. In fact, with so many things going on in the screen at the same time, I doubt people will even notice dropped frames, but that is just my humble opinion.

Answer 2

Some suggestions to consider:

1. Reduce your target frame-rate below 60: Some devices are just so poor in performance that you cannot expect 60fps. There are many examples of commercial games which run at or below 30 fps, and most people don't notice the difference. 60 is ideal, but you just can't expect many mobile devices to be able to handle 60fps for any serious game.

2. Code Optimizations: If there are things that you are doing every frame, which you could do less frequently, then you should consider having a secondary update function which occurs only every X frames. Look for bottlenecks. Do some benchmarking and determine what in your loop is taking the most time, and try to determine if there are faster algorithms for what you are doing. Try other algorithms and see if the benchmarks improve.

3. Reduce the game quality: Reduce the number of game objects; Do fewer draw calls; use faster shaders; Reduce shadow and lighting quality; Use lower polygons objects; etc. You could even allow the player to customize their own quality settings.

4. Release two different versions. A low-end version with 30fps and a high-end version with 60fps. Make it clear to the players that they are not the same. You could even consider removing features from the low-end version to make it even more obvious. ('Limited' and 'Full' versions.)

5. Don't support low-end devices: You just can't expect all devices to be able to handle 60fps. If you absolutely have to have 60fps, then drop support for any devices that cannot keep up. Try to optimize as much as you can and are willing to, but at some point you just have to say: "This device is not worth the effort to support it." The big game companies do this for good reasons!

Answer 3

I have a similar problem with jerky frames using android GL 2.0.

When traced on android studio GL profiler I saw that the glcompileshader function is called multiple times during game play. This is an expensive call but accounts for the jerkiness.

What causes these multiple calls to glcompilershader? I can only guess happens because the shader is invalid but I get no errors at creation. Not sure if accessing attributes in the shader causes a recompile of the shader. Still trying to debug this nightmare.