UI/Menu that works with and without VR (Unity, WMR)

Question

I'm working on an app that I want to be useable without and with VR (with a Windows Mixed Reality HMD like the Samsung Odyssey).

For non-VR I'm simply using a canvas in "Screen Space - Camera" and a second camera, which is a child of the Main Camera and can only see the UI (the Main Camera can see everything but the UI). This way the menus are displayed as transparent overlays that are always visible to the player (with the game scene in the background).

For VR you have to use World Space. This makes sense because I want to be able to look around without having the menu move too.

How do you now make an e.g. pause menu work in VR AND non-VR - depending on if the player plugs in his headset before starting the app or not?

• Do you change the Screen Space to World Space in code at runtime if there's an HMD?
• Do I have to move the whole canvas at runtime to where the player is? You'd want the menu to be in front of the player, no matter where he is but it also has to be in front of the other objects in the scene, so you can interact with it.

I'm using Unity 2017.3 with OpenVR/SteamVR.

Answer 1

This is a question of platform management. Using one Unity project, you can create an application that runs in Desktop (Windows/Mac), VR (Vive/WMR), MR (HoloLens/Magic Leap) and Mobile devices (Android/iOS).

A simple solution would be to do as you say, change a UI canvas from screen space to world space.

However, there are other considerations that you may run into as your UIs get more advanced, that simply changing the space of the canvas can't account for. You may wish to have a UI follow the user in VR/MR, or display different options to mobile users than VR users. All of this gets very complex to manage in a single canvas.

I'd suggest a platform manager. It will know the platform it's running on, and enable/disable scripts and objects accordingly.

You'd arrange things so that you have a parent object that holds a canvas. That parent object will get enabled or disabled by whatever turns on or off your UI. Under the parent object, you have one or more canvas objects for your platforms. For UIs that are common among all platforms, you'd just have one canvas. For UIs that are more complex, you'd have multiple canvases that are turned on or off by your platform manager.

As an example, your menu UI would be turned on at the parent level. In this case, when running in VR, the only canvas under that parent is the VR canvas. This VR canvas is in world space and also has scripts that lerp its position to be in front of the user, 2 meters away. The desktop canvas (that's disabled in this case) would just have a screen space canvas, without the need for movement scripts. In this case, the content of the two UIs might be very similar, but you may have options available only to desktop users (like a display options menu where they can change the resolution).

This will likely result in some duplication of UI elements. Typically this is pretty easy to handle. If you're willing to bump up your Unity version, you can take advantage of the prefab improvements to use nested prefabs, so any duplicated elements are automatically updated across multiple canvases.

Additionally, this platform manager can also handle other things you need to enable/disable when switching between platforms. Like scripts that do spatial mapping on a MR device, or different camera objects for each platform you support.