How can I make message passing between threads in a multithreaded engine less cumbersome?

Question

The C++ engine I'm working on currently is split up into several large threads- Generation (for creating my procedural content), Gameplay (for AI, scripts, simulation), Physics, and Rendering.

The threads communicate with one another through small message objects, which pass from thread to thread. Before stepping, a thread processes all of its incoming messages- updates to transforms, adding and removing objects, etc. Sometimes one thread (Generation) will create something (Art) and pass it to another thread (Rendering) for permanent ownership.

Early in the process and I've noticed a couple things:

1. The messaging system is cumbersome. Creating a new message type means subclassing the base Message class, creating a new enum for its type, and writing logic for how the threads should interpret the new type of message. It's a speed-bump for development and is prone to typo-style errors. (Sidenote- working on this has me appreciating how great dynamic languages can be!)

   Is there a better way to do this? Should I use something like boost::bind to make this automatic? I'm worried that if I do that I lose the ability to say, sort the messages based on type, or something. Not sure if that kind of management will even become necessary.

2. The first point is important because these threads do so much communicating. Creating and passing messages is a big part of making things happen. I'd like to streamline that system, but also be open to other paradigms that might be just as helpful. Are there different multithreaded designs I should think about to help make this easier?

   For example, there are some resources which are infrequently written, but frequently read from multiple threads. Should I be open to the idea of having shared data, protected by mutexes, that all threads can access?

This is my first time designing something with multithreading in mind from the ground up. At this early stage I actually think it's going really well (considering) but I'm worried about scaling, and my own efficiency at implementing new stuff.

Answer 1

To your broader problem, consider trying to find ways to reduce inter-thread communication as much as possible. It's better to avoid synchronisation issues altogether, if you can. This can be achieved by double buffering your data, introducing a single-update latency but greatly easing the task of working with shared data.

As an aside, have you considered not threading by subsystem, but instead using thread spawning, or thread pools to fork by task? (see this in regards to your specific issue, for thread pooling.) This short paper outlines the pool pattern's purpose and usage concisely. See these informative answers also. As noted there, thread pools improve scalability as a bonus. And it is "write once, use anywhere", as opposed to having to get subsystem-based threads to play nice each time you write a new game or engine. There are plenty of solid third party thread pooling solutions out there, too. It would be simpler to start out with thread spawning and work your way to thread pools later, if the overhead of spawning and destroying threads needs to be mitigated.

Answer 2

You asked about different multi-threaded designs. A friend of mine told me about this method that I thought was pretty cool.

The idea is that there would be 2 copies of every game entity (wasteful, I know). One copy would be the present copy, and the other would be the past copy. The present copy is strictly write only, and the past copy is strictly read only. When you go to update, you assign ranges of your entity list to as many threads as you see fit. Each thread has write-access to the present copies in the assigned range and every thread has read-access to the all past copies of the entities, and thus can update the assigned present copies using data from the past copies with no locking. Between each frame, the present copy becomes the past copy, however you want to handle the swapping of roles.

Answer 3

We've had the same problem, only with C#. After thinking long and hard about ease (or lack thereof) of creating new messages, the best we could do was to make a code-generator for them. It's a bit ugly, but usable: given only a description of message contents, it generates message class, enums, placeholder handling code etc - all this code that is almost the same each time, and really typo-prone.

I'm not entirely happy with this, but it's better than write all that code by hand.

Concerning shared data, the best answer is of course "it depends". But generally, if some data is read often, and needed by many threads, sharing it is worth it. For thread safety, your best bet is making it immutable, but if that's out of the question, mutex might do. In C# there's a ReaderWriterLockSlim class, specifically designed for such cases; I'm sure there is a C++ equivalent.

One other idea for thread communication, that probably solves your first problem, is to pass handlers instead of messages. I'm not sure how to work this out in C++, but in C# you can send a delegate object to another thread (as in, add it to some kind of message queue), and actually call this delegate from the receiving thread. This makes it possible to create "ad hoc" messages on the spot. I only toyed with this idea, never actually tried it in production, so it might turn out bad in fact.

Answer 4

I'm only in the design phase of some threaded game code, so I can only share my thoughts, not any actual experience. With that said, I'm thinking along the following lines:

• Most game data should be shared for read-only access.
• Writing data is possible using a kind of messaging.
• To avoid updating data while another thread is reading it, the game loop has two distinct phases: read and update.
• In the read phase:
• All shared data is read-only for all threads.
• Threads can calculate stuff (using thread-local storage), and produce update requests, which are basically command / message objects, placed in a queue, to be applied later.
• In the update phase:
• All shared data is write-only. Data is to be assumed in an unknown / unstable state.
• This is where update request objects are processed.

I think (though I'm not sure) in theory this should mean that during both the read and update phases, any number of threads may run concurrently with minimal syncing. In the read phase no one is writing the shared data, so no concurrency issues should arise. The update phase, well, that's trickier. Parallel updates on the same piece of data would be a problem, so some synchronization is in order here. However, I could still run an arbitrary number of update threads, as long as they are operating on different data sets.

In all, I think this approach would lend itself well to a thread pooling system. The problematic parts are:

• Syncing the update threads (make sure no multiple threads try to update the same data set).
• Making sure that in the read phase no thread can accidentally write shared data. I'm afraid there would be too much room for programming mistakes, and I'm not sure how much of them could be easily caught by debug tools.
• Writing code in a way such that you can't rely on your intermediate results being available for reading right away. That is, you can't write x += 2; if (x > 5) ... if x is shared. You need to either make a local copy of x, or produce an update request, and only do the conditional in the next run. The latter would mean a whole lot of additional thread-local state preserving boilerplate code.