In Unity, how do I use the extra time in every frame to do some extra processing?

Let's say, for example, that my target framerate is 60 frames per second.

Now, I know that I have a maximum of 16.6 milliseconds to "do everything" every frame in order to maintain that framerate.

Let's say that I also have a background task that I wish to run. It will return some results. I don't need them urgently; I just wish to run it and have its results when it's done.

Here's my point. I know when I'm done rendering everything; WaitForEndOfFrame() in a coroutine tells me so. Let's say my last frame took only 13 milliseconds to render instead of 16.6, so I know I have 3.6 milliseconds to spare during which I would like to run my task for exactly those 3.6 miliseconds or less, never going over my 16.6 milliseconds maximum total, and then continue with the next frame, on which I will do the same if I get some spare time.

Could I achieve this using Unity's coroutines (or some other way)? If so, how?

• I don't think there is anyway you can actually plan for something like that. You can't expect a code to run at exactly the same time on the insane different CPUs and platforms out there. Making assumptions on the time you have left will lead to chaos. – Kamalen Jan 17 '18 at 17:02
• For science, you can get the time of your Update process using C# native DateTime and TimeSpan (because Unity Time.time is set at start of the frame and won't change during this frame), then somehow put in place a piece of code. But unless your secondary task is made of many very fast little tasks, or allows you to cut out the execution when no time is left, you have a risk of missing the exit time. – Kamalen Jan 17 '18 at 17:09
• Have you looked into the new Unity Jobs System? – DMGregory Jan 17 '18 at 17:45
• I haven't, I'll be sure to check it out. I see it's a new feature in the works, available in preview builds. Thank you. – flatterino Jan 18 '18 at 8:07

1 Answer

This (general) approach is used in several AAA engines, but isn't very compatible with the way current Unity (2017) works.

To do what you've described, you'd have to build your own task system that doesn't immediately conflict with Unity's own process. Doing so is generally ineffective as the extra overhead negates the benefits of extra time.

For further knowledge, I suggest reading up on the fiber architecture. There are several great GDC talks on it, though they are currently locked behind the vault.

Here are the slides for Christian Gyrlin's GDC presentation on Naughty Dog's engine architecture. It can get a little low level, but it goes over how this kind of process can be achieved.

That's all well and good, but what if you still want to build it?

While there's no good way to do exactly what you want (specifically, timed processing in unity), there are plenty of ways to split your tasks into chunks, assuming the right data architecture.

So how do you build that data architecture? Well you need these metrics (Not comprehensive, hopefully a good start)

• Task Intensity : You need to know how much time / resources will the task requires.
• Order of Operations : Clearly defined task prerequisites and dependencies. These help you determine how to allocate your current resources. Small tasks with lots of dependencies should get high priorities.
• Pausing : Similar to above, when and where can a task sleep to free processing power for critical elements
• Granularity : The more granular your process, the easier it will be to take advantage of any left over resources.

Once you have these, it's pretty easy to grab something applicable using a job queue or query.

What's my preferred alternative?

Rather than intentionally calling tasks to use up leftover time, I recommend filling up that time with background processes / coroutines.

Coroutines are wonderful for dispatching tasks that don't need to be done same frame and are easy enough to split / pause. While not exactly the same context, I explain how to use them to split a task across multiple frames here

In general, if I'm at or above my target frame rate, I don't worry about squeezing out even more performance. Leave the buffer there for when future you wants to do something cool.

As DMGregory mentioned, Unity has plans to implement a proper Job System in the near future. If this is a feature you require, There is a unite talk on how it will work.

• You can actually access the Job System now in the early preview build. I haven't had a chance to experiment with it just yet though, so I can't speak from experience on how it works. – DMGregory Jan 17 '18 at 21:31