Is it feasible to playback 4K video in Unity?

Question

I'm starting work on a 360 VR video player, aimed at mobile devices (iOS/Android Cardboard + GearVR) and I'm evaluating whether it is possible to get video quality from 1080p to 4k in Unity or if I should instead just do the project natively (aka, OpenGL).

I've been trying to find information on the state of video in Unity but I haven't arrived to a clear answer. I'm going to explain the info I'm running on as of now, hopping that someone can add to this or provide an answer or at least show that I am erring on the side of ignorance :)

Assume a video in mp4, H264 at 4K resolution (equirectangular mapping):

OpenGL Approach:

If I were to do this in OpenGL, a (somewhat simplified) workflow would entail decoding each video frame, updating a GL_TEXTURE_2D with the pixel contents, and rendering to a sphere with glCullFace(GL_FRONT) as if it where a regular skybox.

With the above setup, the limiting factors would be:

• Does the GPU in my device support 4K textures? what marketing calls 4k is usually 3840 horizontal, and, an iPhone6 for instance, returns 4096 for glGetIntegerv(GL_MAX_TEXTURE_SIZE,...). So I'd assume the iPhone6 can load textures as big as required for 4K.

• Fill rate: i.e.: my iPhone6's GPU might be capable of storing 4K Textures, but it might not be fast enough to fill up a 4K frame buffer at an acceptable frame rate. This unfortunately I think the only way to know is by testing. There is also the added overhead of the Cardboard rendering which means 2 passes per frame.

Unity Approach:

Now for a Unity approach, this is what I did:

1. Empty scene with a sphere and cardboard camera at the origin.
2. For the sphere, I created a material based on the Unlit shader with one texture and set the cull face to front.
3. imported the video and assigned it as the material's texture.

My observations from the above test are:

1. When importing images, Unity also allows texture sizes of 4096 so 4K should be possible.
2. Importing a video however, takes a ridiculous amount of time. I believe this is because Unity actually uses Ogg Theora and so it takes forever to transcode to Ogg. As a matter of fact, sometimes I was able to import and sometimes Unity would just stall.
3. When importing a video, I was not given any mmap, filtering or size options as if I was importing an image. So I'm not really sure what Unity is doing to the video. All I get is a quality / compression slider. The end video is still 3840x1920, which was the original resolution.
4. The rest of the experiment works ok, as in, yes, the video playbacks in the sphere. But the framerate is not great.

So at this point my conclusions are:

1. Is it possible to do 4K in Unity? Kind of, In the sense that you can import 4K videos and reproduce them without any external assets.
2. Is it really 4K quality? Probably not. mp4 is lossy and so is ogg, So I assume my video went a second round of lossy compression when it was transcoded to Ogg.
3. Is it practical? Not sure. So far, video importing takes an impractical amount of time. Also, 4K in general might be too much for a mobile device rending in stereo, regardless of Unity or doing it straight in OpenGL.

Answer 1

Short answer:

If you want to playback 4K in Unity you are better off finding some plugin (or coding your own) that uses the native API for video playback that the OS you are targeting uses.

Then you place the video file in either Streaming Assets or PersistentDataPath.

This way you avoid Unity's transcoding.