In Android, neat way to send input info from GLSurfaceView to onDrawFrame in GLSurfaceView.Renderer?

The input is pinch/drag for camera movement, which modifies a matrix to scale/translate, and sent to a vertex shader. It's working, but ugly.

What's the proper way to do it?

I've noticed:

• GLSurfaceView gets onTouch callbacks; but Renderer has access to GL.

• Shaders and textures can only be loaded after the onSurfaceCreated() callback, so should be loaded there?

• Touch and graphics are on different threads, so should use queueEvent() or similar.

Is there a game/app on github that's a good example to follow? Thanks for any help!

General Architecture

If you had input and rendering share a thread, that would allow you to mix input and output(rendering) code, making it easier to make messy code.

What I suggest you to do is to have an Input Model. That is some object that contains a representation of the input of the user. Then you update it from your event handlers (listeners? virtual methods?). By the way, a well done Input Model does not have details of the input devices (for example, if touching a certain part of the screen is how you input a jump, then you store that the user inputs jump in the Input Model, not the details of the touch event).

The Input Model can be queue, if that is what fits your game. Another common option is a HashMap.

Let me step back further. Your game has an entry point. This entry point creates an object that serves as Composition Root for your game. What does the Composition Root do? In simple terms, it plugs stuff. This is basically setting up event handlers.

Some event handlers will be used to initialize stuff, such as the Input and Game Models. Yes, load things into GL in onSurfaceCreated. Other event handlers will receive input events, and these event handler will have code to update the Input Model. etc... Yes, this is how you would probably do it anyway.

After the Composition Root plugged all the stuff. The game loop can begin.

In your update/game loop you can read the Input Model and use it to update your world/scene (Domain Game Model). And finally your render method (where you access GL) can read your Game Model and output.

It is OK for the update/game loop to update the Input Model (as long as encapsulation is not broken). If we go with the queue idea, then update/game loop can dequeue. In fact, it is OK to have update/game loop swap the Input Model, similar to how double buffering works.

By the way, in your Game Model is where you would do space partitioning. If you want to use an ECS, you make it part (or the totality) of the Game Model.

These are the guidelines in general:

• There is a Composition Root, created on the entry point, that plugs everything, and starts the game loop.
• Input handlers will depend on the Input Model (they write it). The Input Model should not depend on the input device details.
• Output handlers will depend on the Game Model (they read it).
• The game loop contains logic that updates both Input and Game Models.
• Your game loop should not depend on any device details.

You can further evolve this architecture by breaking the Game Model into multiple sub-models. After all, the above does not have a strong separation between output and logic concerns. For example, one of the sub-models could be only what you need for rendering, and another could be just what you persist when the game is saved, perhaps another for AI, etc. Yes, this can make ECS appealing. If you are picking an ECS, I would like to encourage picking one that (allows configuration that) makes the kind of queries you need easy and efficient.

Android Particular

For Android, by default the Composition Root is the Activity. It is OK to have it delegate or even to use a different Composition Root. For OP constraints, we will use an object of type GLSurfaceView as Composition Root. Remember, your Composition Root is your glue.

This means that we will have a class MyGLSurfaceView which extends GLSurfaceView and its responsibility is to setup everything. However, GLSurfaceView has the methods to deal with I/O (touch, render, etc...). How do we separate that?

The architectural line that we have to draw is that any code that does I/O should not be in MyGLSurfaceView. Instead we put that in other classes, let us call them Adapters. And the Adapters need a reference to GLSurfaceView for I/O.

MyGLSurfaceView creates the Adapters, then it can call setOnTouchListener on itself and set the Adapter as listener. Remember that the Adapter has to update the Input Model.

As alternative, you can have MyGLSurfaceView pass itself (as GLSurfaceView) in the constructor of the Adapters (dependency injection), and then the Adapters manipulate that reference. That is an inversion of control. No special container needed.

For rendering, there is the Renderer. It will be a separate class, that implements Renderer. MyGLSurfaceView will instantiate it.

As you know, when you call setRenderer, the GL thread will start, and that thread will set up the GL context, and enter a loop. This thread will call the methods of the Renderer.

Yet, the Renderer needs to know what to render, and that depends on the state of the game/scene/world. We want the Renderer to pull this information.

First, add to the Renderer interface a method to set a callback. This callback must return an object that represent the state of the game (or at least the part of the state of the game that is relevant for rendering※). The Renderer can call it to get what it has to draw on onDrawFrame.

※: I would expect the state of the game to have an space partitioning structure where you can query the objects that are nearby the camera, so that the Renderer can limit itself to those.

Second, make Game Logic class. This class is responsible for maintaining the state of the game. It has to take the Input Model, update the state of the game, and return it. All game logic, devoid of I/O, belongs there.

To reiterate, the Input Model is in terms of the game, not in terms of the input device.

Finally, since MyGLSurfaceView is our Composition Root. It is its responsibility to set up the callback. The callback will get the Input Model from the Input Adapter, pass it to the Game Logic class, and return the state of the game to the Renderer.

The above approach should not have a race condition, given that the Game Logic runs on demand. The object that the Renderer gets from the Callback will not change while the Renderer is using it.

Let us say you want to change it so that the state of the game is updated as soon as input happens.

In that case, we need the Input Adapter to push the input events. That is, add to the Input Adapter a method to set a callback that will run every time the Input Model is updated. The Input Adapter is still responsible of updating the Input Model. Next, MyGLSurfaceView will setup the callback, making it call the Game Logic.

However, when we do this, the state of the game can update in the middle of rendering.

First notice that it is trivial to cache the state of the game. Have MyGLSurfaceView store the sate in a field, and then in the callback for the Renderer, it can pass it. This means that the Game Logic does not need to return the same object every time.

Taking advantage of that, we can solve the threading problem by buffering the state of the game. What? You will have two state of the game. One that is being used by the Renderer, and one that is being updated. Then have a method in the Game Logic to swap them. The MyGLSurfaceView will tell the Game Logic to swap the state of the game when the Renderer calls.

You may, of course, make a queue of states instead.

Please notice that having two states of the game do not imply to use double the memory. If you make everything in the state objects immutable, there is no problem in both of them referencing the same objects. That means that you only need extra memory for what changed from one state to the other.

What matters is that the state of the game that the Renderer is using does not change. If you know that the objects in that state of the game will not change, you can reference them in the other state of the game... just remember to replace them (instead of updating them). Making everything immutable is just the safest way to do that.

It is also possible to make the read only instead of immutable, and pool them.

• I realize now how specific my question is to Android's GLSurfaceView, whereas the "proper" way that you've addressed is cross platform. I've read your answer several times, and researched "ECS" (which I think is not AWS's Elastic Container Service, but Entitty Component System - that was confusing for a while!), but find it hard to relate to my codebase. You've mentioned some specific aspects of GLSurfaceView in passing - could you please bring them out just a little more? Mar 26 '20 at 6:35
• BTW: in my codebase, my GLSurfaceView subclass is the "Composition Root" (because of other constraints); and I also want to be able to plug in graphics code, so I can reuse the basics for other projects. Maybe an interface with initInput() for touch event handlers andinitGraphics() for shaders and textures? I think I need to get on top of these issues, even if I did go for an ECS architecture! Mar 26 '20 at 6:44
• Sorry, forgot to say thannks for taking the time for your thoughtful answer! It will me more useful to me when I'm a little more advanced! Mar 26 '20 at 7:01
• @hyperpallium Yes ECS is Entity Component System. By default the Composition Root in an Android application would be an Activity. However, please note it is a root, it is OK to delegate part of the setup work. I'm not sure what you would do with those interfaces. In my mind you should be able to refactor your code by extracting methods, extracting classes, and perhaps some inversion of control. I have to say it has been a long time since I developed for Android, so I might be missing something. Which reminds me, perhaps you can find softwareengineering.stackexchange.com useful. Mar 26 '20 at 14:28
• @hyperpallium Alright, trying to think about your code... there is a touch event, onTouch runs. onDrawFrame has to update an uniform that holds a matrix computed based on the touch event. You can compute the matrix on either end, what matters is that you pass the relevant information. Updating the uniforms should not mean to change the shader program. Yes, of course, you can load programs and textures afterwards, however you want to front load as much as possible. Threading problems? Simplest solution is to pass an immutable object. Or, at least a read only interface. So no race condition. Mar 26 '20 at 14:39