What operations should be multithreaded?

Question

I wonder what game operations should be multithreaded, and which should always run on the main thread.

The most expensive operations I could come up with are:

• loading a level
• loading resources (for each level or at game initialization)
• updating game world (weather, liquids, chain reactions, lots of small entities, etc.)
• downloading from game server (level files, custom resources, player saves, object states, etc.)
• occasional heavy methods (large screenshots, per-pixel calculations, full world update)

Which of those operations should be threaded, and which should not? Which of them can be threaded most effectively by using as many threads as the host supports?

Or is it better to split some of these operations into blocks and execute each frame, bit by bit, without threading, and without causing frame drops?

Answer 1

Threading implies concurrency. Think of a logical flow. What would you like to happen in parllel and which actions should only occur AFTER something happens.

Modern computer architectur benefits from parallel computing. Either on CPU cores or the GPU.

In my humble opinion :

• I don't know if multithreading disk usage improves perfomance and loading times a lot (please try and let me know :D )

• On the other hand using data from the Internet should be absolutely parallelised. One doesn't depend on the other(much ;) )

• "updating game world" this is where parallelism shines. Like I said before, paralelised computation is better
• "occasional heavy methods" same. Sometimes you might wish to distribute the tough computation among a couple of frames if you can. You don't always need to keep everything as fast as the FPS

Answer 2

All operations that are performed over a large set of data, with each operation working independently from one another will benefit from parallelization. Here are some common examples of such tasks:

• Collision checking
• Pathfinding
• Constraint solving
• Particle simulation
• Joint pose interpolation

Some tasks, like pathfinding, are better suited to the CPU because algorithms like A* involve a lot of dynamic branching. When you are performing particle simulation or joint pose interpolation, you can offload this work to the GPU because each operation is identical except for the data.

Answer 3

It really depends. You're likely to gain the most benefit by performing operations with no data interdependence on each other in parallel, but as with anything you should profile beforehand (and afterwards) to make sure you're getting the most benefit from your efforts.

For example, to give some concrete advice regarding one of your examples, loading data:

If you can factor your resource loading into independent chunks, this can be beneficial. However, you'll want to investigate whether or not you're already disk-bound. If the slow part of your loading operations is reading from the disk (more likely with non-SSDs, such as platter drives or optical media) you may not improve your situation. Similarly, if the ultimate destination of the data you read from the disk isn't independent, you should verify that the synchronization cost of installing all the loaded data into it isn't a bottleneck.

At work we all have SSDs in our development machines, and content containers store their data independently in memory, so switching our toolchain to a multithreaded loading system did have significant performance implications (a 600% improvement in load times). But your mileage may vary.

Similar caveats apply to other operations, such as game logic updates. If you can organize the problem into discrete chunks of computation that don't depend on eachother, you can probably see some benefit from a concurrent model. But if you find yourself having to employ lots of locks or other thread synchronization primitives, chances are you're actually trying to parallelize the serial part of the task and will introduce more overhead than you'll remove.

You want to approach concurrency at the large scale, not the small one. Think about splitting operations into multiple concurrent tasks on the scale of "loading data" and "per frame game updates." Generally, looking at the scale of "this big screenshot function" is too small to be useful -- thus, your "occasional heavy method" category of operations is not likely to gain much benefit. Screenshots are inherently serial, except for the actual disk IO portion (covered earlier in our discussion of loading), per-pixel operations are already best handled by the GPU, and I don't think a "full world update" is an "occasional heavy method" but rather a synonym for what was discussed previous regarding updating the game world.