0
\$\begingroup\$

We are working in a context where multithreading is not an option, so we're using coroutines to handle our loads. Often there are multiple things that need to be downloaded, leading to a pattern like this:

// set up multiple downloads
dataToDownload = 3;
StartCoroutine(ReadDataCoroutine1(OnDataRead));
StartCoroutine(ReadDataCoroutine2(OnDataRead));
StartCoroutine(ReadDataCoroutine3(OnDataRead));

// handle each of the loads completed
private void OnDataRead()
{
    // nasty and error prone way to check completion of all data
    dataToDownload--;
    if (dataToDownload > 0) return;

    DoStuffNowThatDataIsAllLoaded();
}

So I thought I would build a helper class to batch the downloads:

public BatchDownloader(
    IEnumerable<Func<IEnumerator>> downloadFuncs,
    Action<int, int> onDownloadComplete = null,
    Action<int> onAllDownloadsComplete = null)
{
    this.downloadFuncs = downloadFuncs;
    this.onDownloadComplete = onDownloadComplete;
    this.onAllDownloadsComplete = onAllDownloadsComplete;
    totalDownloads = downloadFuncs.Count();
}
public IEnumerator DownloadBatch()
{
    completedDownloads = 0;
    foreach (var downloadFuncs in downloadFuncs)
        yield return ProcessDownload(downloadFuncs);
    onAllDownloadsComplete?.Invoke(totalDownloads);
}
private IEnumerator ProcessDownload(Func<IEnumerator> downloadFunc)
{
    yield return downloadFunc();
    onDownloadComplete?.Invoke(totalDownloads, ++completedDownloads);
}

Used something like this:

public IEnumerator LoadAllTheThings()
{
    Func<IEnumerator>[] downloadFuncs = new Func<IEnumerator>[]
    {
        ReadDataCoroutine1,
        ReadDataCoroutine2,
        ReadDataCoroutine3
    };

    BatchDownloader batchDownloader = new BatchDownloader(downloadFuncs);

    yield return batchDownloader.DownloadBatch();

    DoStuffNowThatDataIsAllLoaded();
}

Now that is much cleaner, more flexible, and it all works a treat, except that no load starts until the previous finishes. This obviously has a huge effect on load time. So in an attempt to have the coroutines run simultaneously, I'm trying the following:

public IEnumerator DownloadBatch()
{
    completedDownloadsCount = 0;

    var activeDownloads = new List<IEnumerator>();
    foreach (var downloadFunc in downloadFuncs)
    {
        activeDownloads.Add(downloadFunc());
    }

    while (activeDownloads.Count > 0)
    {
        yield return null;
        for (int i = activeDownloads.Count - 1; i >= 0; --i)
        {
            var download = activeDownloads[i];
            if (!download.MoveNext())
            {
                onDownloadComplete?.Invoke(totalDownloadsCount, ++completedDownloadsCount);
                activeDownloads.RemoveAt(i);
            }
        }
    }

    onAllDownloadsComplete?.Invoke(totalDownloadsCount);
}

Now while this correctly kicks off the coroutines, and runs them "in parallel", any code after the last yield in a coroutine is not executed. So in a method like the following:

private IEnumerator ReadDataCoroutine1()
{
   yield return WaitForStuffToDownload();

   DoPostDownloadStuff();
}

DoPostDownloadStuff() is never called.

What am I misunderstanding here about stepping through these IEnumerators? Is there a way to do what I'm trying to, without introducing StartCoroutine into the batch downloader? (that is my solution for now, but idealy I'd like to not have to)

\$\endgroup\$
  • \$\begingroup\$ Can you clarify why you want to avoid StartCoroutine? \$\endgroup\$ – DMGregory Apr 22 at 11:14
  • \$\begingroup\$ @DMGregory because I want to be able to yield return the downloader IEnumerator out of another IEnumerator. Essentially this allows me to write code in a way very similar to async/await, where I can keep a series of related actions together. I can have a very high level object doing the StartCoroutine call \$\endgroup\$ – FlintZA Apr 22 at 13:04
  • \$\begingroup\$ You can still do that with StartCoroutine. There's no restriction on calling StartCoroutine only once in the execution tree. \$\endgroup\$ – DMGregory Apr 22 at 13:07
  • \$\begingroup\$ Sure, but I would need to pass the unity object down the chain to be used to call StartCoroutine (there is no reason for the batch class itself to be a unity object) \$\endgroup\$ – FlintZA Apr 22 at 13:27
  • \$\begingroup\$ Sounds like you have a solution then. Does it differ from your desired behaviour in any way? \$\endgroup\$ – DMGregory Apr 22 at 13:36
1
\$\begingroup\$

For completeness sake, here is the version that uses a passed in MonoBehaviour to run the downloads.

[Edit]: I've settled for this solution, I haven't had any success with a version that does not use StartCoroutine. [/Edit]

Note also the fix in ProcessDownload, for the case where no onDownloadComplete delegate was provided (whoops).

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using UnityEngine;

namespace MB.Other
{
    public class BatchDownloader
    {
        private readonly IEnumerable<Func<IEnumerator>> downloadFuncs;
        private readonly Action<int, int> onDownloadComplete;
        private readonly Action<int> onAllDownloadsComplete;
        private readonly int totalDownloadsCount;

        private int completedDownloadsCount;

        public BatchDownloader(
            IEnumerable<Func<IEnumerator>> downloadFuncs,
            Action<int, int> onDownloadComplete = null,
            Action<int> onAllDownloadsComplete = null)
        {
            this.downloadFuncs = downloadFuncs;
            this.onDownloadComplete = onDownloadComplete;
            this.onAllDownloadsComplete = onAllDownloadsComplete;
            totalDownloadsCount = downloadFuncs.Count();
        }

        public IEnumerator DownloadBatch(MonoBehaviour coroutineRunner)
        {
            completedDownloadsCount = 0;

            foreach (var downloadFuncs in downloadFuncs)
                coroutineRunner.StartCoroutine(ProcessDownload(downloadFuncs));

            while (totalDownloadsCount != completedDownloadsCount)
                yield return null;

            onAllDownloadsComplete?.Invoke(totalDownloadsCount);
        }

        private IEnumerator ProcessDownload(Func<IEnumerator> downloadFunc)
        {
            yield return downloadFunc();

            completedDownloadsCount++;

            onDownloadComplete?.Invoke(totalDownloadsCount, completedDownloadsCount);
        }
    }
}
| improve this answer | |
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.