Dependency injection, in some form, is almost always the solution I would reach for first. DI approaches to providing access to objects have some clear benefits:
they are usually fast: the consuming code is directly given a reference to the dependencies and can usually cache those references, so no further lookups need to be done in the consuming code.
they are self-documenting: the dependencies of the consuming code are clearly listed in the code itself as part of the structure of the code, which usually makes it obvious when you've broken something via a refactor, and helps indicate to a future reader of the code what that code is supposed to have jurisdiction over (and by exclusion, what that code should not be able to do).
they do away with the assumption of globality of the dependency objects: DI approaches, unlike your other two cited methods (usually), don't assume that there is one global version of some dependency, which can give the consuming code some flexibility for future expansion or for isolation (in the case of unit tests, for example).
An oft-cited downside to DI approaches is that they involve passing "some thing" to "every other thing." This can be a downside, especially when refactoring large existing codebases that rely on some other method of access to dependencies. However, on balance it is also usually a strong indicator of a potential design flaw (the fact that "everything" depends on some object is an implication that code may not be well-factored).
I tend to use DI approaches to pass around dependencies in game code: giving certain systems access to other systems, or certain objects access to sibling objects (e.g., rendering components that depend on reading data from physical components get a reference to the corresponding component via the constructor).
Singletons are about enforcing the existence of a single object and providing global access to it. They have the advantage of being easy to use, and easy to proliferate: if you need to access some object at some point in the code, you don't have to worry about plumbing a reference to that object all the way through from where it originates to where you need it.
On the other hand, one could argue that that ease creates problems: by not having to think about how to get an object to where you want to use it, you don't have to think about whether or not that is creating a bad, overly-coupled design. Further, any such dependency is harder to find, as it's buried randomly in the consuming code where-ever somebody chose to access to the singleton, and isn't always apparent by simply browsing the interface of that consuming code.
Finally, depending on the language you are using and the method you are using to implement a singleton, they can have some technical issues since they are a global object: multithreaded access can present some issues that in some cases cannot be solved except by taking a lot of locks, and initialization order of the object can require some careful planning in languages like C++ which have interesting static object initialization ordering caveats.
I would generally never recommend a singleton, personally, and do not use them.
Find-via-tag (or in general, lookup via some well-known identifier) is an interesting middle-of-the-road approach. Depending on how the lookup is done, it could be more costly than the simple direct access usually offered by DI solutions. Depending on the implementation of the backing storage for the objects, it may not be possible for consuming code to cache the looked-up reference, thus requiring constant re-fetches.
It is also possible for lookups to fail, since they are generally done at runtime, and thus it is harder to verify the compile-time correctness of any consuming code. Such code often requires more protection and error-checking scaffolding to ensure robustness; removing an object entirely from the code base may not immediately present a compile-time error in code that looked up that object at runtime, if the code is only checking for existence of that object and not actually calling any methods or whatever on that object's interface.
But find-via-tag can be used as an bridging mechanism allowing it to steal some advantages from the other two approaches. For example, an "object locator" object could be passed via DI to consuming code, which then uses it to look up the specific objects it needs. This gives you a place to insertion some isolation into the code, freeing you from assumptions that there is one instance of every dependency object, but also retains some of the ease-of-use of a singleton.
I often use this approach in editor framework code to provide a service-locator object that can be used for plugins or modules to request access to the editor services (transaction or undo service, notification service, et cetera).