Separating Game Engine from game code in similar games, with versioning

Question

I have a finished game, that I want to decline in other versions. These would be similar games, with more or less the same kind of design, but not always, basically things might change, sometimes little, sometimes big.

I would like the core code to be versioned separately from the game, so that if say I fix a bug found in game A, the fix will be present in game B.

I am trying to find the best way to manage that. My initial ideas are this:

• Create an engine module/folder/whatever, that contains everything that can be generalized and is 100% independent from the rest of the game. This would include some code, but also generic assets that are shared among games.
• Put this engine in its own git repository, which will be included in the games as a git submodule

The part I am struggling with is how to manage the rest of the code. Let's say you have your menu scene, this code is game-specific, but also most of it tends to be generic and could be reused in other games. I can't put it in the engine, but re-coding it for each game would be inefficient.

Maybe using some sort of git branches variation could be effective to manage that, but I don't feel like this is the best way to go.

Anyone has some ideas, experience to share or anything about that?

Answer 1

Create an engine module/folder/whatever, that contains everything that can be generalized and is 100% independent from the rest of the game. This would include some code, but also generic assets that are shared among games.

Put this engine in its own git repository, which will be included in the games as a git submodule

That's exactly what I do and it works very well. I've an application framework and a rendering library, and each of these are treated as submodules of my projects. I find SourceTree is useful when it comes to submodules, as it manages them well and won't let you forget anything, e.g. if you updated the engine submodule in project A, it will notify you to pull the changes in project B.

With experience comes the knowledge of what code should be in the engine vs. what should be per-project. I suggest that if you are even mildly unsure, that you keep it in each project for now. As time goes by, you will see amongst your various projects what remains the same and then you can gradually factor that out into your engine code. In other words: duplicate code until such time as you are close to 100% certain that it is not changing discretely per project, then generalise it.

Note on Source Control and Binaries

Just remember that if you are expecting your binary assets to change often, you may not want to put these in text-based source control like git. Just sayin'... there are better solutions for binaries. The simplest thing you can do for now to help keep your "engine-source" repository clean and performant is to have a separate "engine-binaries" repository that holds only binaries, which you also include as a submodule in your project. This way you mitigate the performance-damage done to your "engine-source" repository, which is changing all the time and on which you thus need fast iterations: commit, push, pull etc. Source control management systems like git operate on text deltas, and as soon as you introduce binaries, you introduce massive deltas from a text perspective - which ultimately costs you dev time. Another solution may be GitLab Annex. Google's your friend.

Answer 2

At some point an engine MUST specialise and know stuff about the game. I will go off on a tangent here.

Take resources in an RTS. One game may have Credits and Crystal another Metal and Potatoes

You should use OO concepts properly and go for max. code-reuse. It's clear that a concept of Resource exists here.

So we decide resources have the following:

1. A hook in the main loop to increment/decrement themselves
2. A way to get the current amount (returns an int)
3. A way to subtract/add arbitrarily (players transferring resources, purchases....)

Notice that this notion of a Resource could represent kills or points in a game! It isn't very powerful.

Now lets think about a game. We can sort of have currency by dealing in pennies and adding a decimal point to the output. What we cannot do are "instantaneous" resources. Like say "power grid generation"

Lets say you add a InstantResource class with similar methods. You're now (starting to) pollute your engine with resources.

---

The Problem

Lets take the RTS example again. Suppose player whatever donates some Crystal to another player. You want to do something like:

if(transfer.target == engine.getPlayerId()) {
    engine.hud.addIncoming("You got "+transfer.quantity+" of "+
        engine.resourceDictionary.getNameOf(transfer.resourceId)+
        " from "+engine.getPlayer(transfer.source).name);
}
engine.getPlayer(transfer.target).getResourceById(transfer.resourceId).add(transfer.quantity)
engine.getPlayer(transfer.source).getResourceById(transfer.resourceId).add(-transfer.quantity)

However this is really quite messy. It's general purpose, but messy. Already though it imposes a resourceDictionary which means now your resources have to have names! AND it is per player, so you cannot have team resources any more.

This is "too much" abstraction (not a brilliant example I'll admit) instead you should hit a point where you accept that your game has players and crystal, then you can just have (for example)

engine.getPlayer(transfer.target).crystal().receiveDonation(transfer)
engine.getPlayer(transfer.source).crystal().sendDonation(transfer)

With a class Player and a class CurrentPlayer where CurrentPlayer's crystal object will automatically show the stuff on the HUD for the transfer/sending of donations.

This pollutes the engine with crystal, the donating of crystal, the messages on the HUD for current players and all that. It is both faster and easier to read/write/maintain (which is more important, as it isn't significantly faster)

---

Final remarks

The resource case isn't brilliant. I hope you can still see the point though. If anything I have demonstrated that "resources do not belong in the engine" as what a specific game needs and what is applicable to all notions of resources are VERY different things. What you will usually find are 3 (or 4) "layers"

1. The "Core" - this is the textbook definition of engine, it's a scene graph with event hooks, it deals with shaders and network packets and an abstract notion of players
2. The "GameCore" - This is pretty generic to the type of game but not to all games - for example resources in RTS or ammunition in FPSs. The game logic starts to seep in here. This is where our earlier notion of resources would be. We've added these things that make sense for most RTS resources.
3. "GameLogic" VERY specific to the actual game being made. You'll find variables with names like creature or ship or squad. Using inheritance you'll get classes that span all 3 layers (for example Crystal is a Resource which is a GameLoopEventListener say)
4. "Assets" these are useless to any other game. Take for example the combine AI scripts in half life 2, they're not going to be used in an RTS with the same engine.

---

Making a new game from an old engine

This is VERY common. Phase 1 is to rip out layers 3 and 4 (and 2 if the game is a TOTALLY different type) Suppose we are making an RTS from an old RTS. We still have resources, just not crystal and stuff - so the base classes in layers 2 and 1 still make sense, all that crystal referenced in 3 and 4 can be discarded. So we do. We may however check it as a reference for what we want to do.

---

Pollution in layer 1

This can happen. Abstraction and performance are enemies. UE4 for example provides a lot of optimised cases of composition (so if you want X and Y someone wrote code that does X and Y together really fast - it knows it is doing both) and as a result is REALLY quite large. This isn't bad but it is time consuming. Layer 1 will decide things like "how you pass data to shaders" and how you animate things. Doing it the best way for your project is ALWAYS good. Just try and plan for the future, reusing code is your friend, inherit where it makes sense to.

---

Classifying layers

LASTLY (I promise) don't be too afraid of layers. Engine is an archaic term from the old days of fixed function pipelines where engines pretty much worked the same way graphically (and as a result had a lot in common) the programmable pipeline turned this on its head and as such "layer 1" became polluted with whatever effects the developers wanted to achieve. AI was the distinguishing feature (because of the myriad of approaches) of engines, now it is AI and graphics.

Your code shouldn't be filed in these layers. Even the famous Unreal engine has MANY different versions each specific to a different game. There are few files (other than like data structures maybe) that would have gone unchanged. This is fine! If you want to make a new game from another it'll take longer than 30 minutes. The key is to plan, to know what bits to copy and paste and what to leave behind.

Answer 3

My personal suggestion for how to handle the content that's a mix of generic and specific is to make it dynamic. I'll take your menu screen as an example. If I misunderstood what you were asking for, let me know what it was you wanted to know and I'll adapt my answer.

There are 3 things that are (nearly) always present on a menu scene: the background, the game logo and the menu itself. These things are usually different based on the game. What you can do for this content is make a MenuScreenGenerator in your engine, which takes 3 object parameters: BackGround, Logo and Menu. The basic structure of these 3 parts are also part of your engine, but your engine does not actually say how these parts are generated, just what parameters you should give them.

Then in your actual game code, you create objects for a BackGround, a Logo and a Menu, and you pass this to your MenuScreenGenerator. Again, your game itself does not handle how the menu is generated, that's for the engine. Your game only needs to tell the engine what it should look like and where it should be.

Essentially, your engine should be an API that the game tells what to display. If done properly, your engine should do the hard work and your game itself should only tell the engine what assets to use, what actions to take and what the world looks like.