# 360 degree video of my OpenGL game

I want to make a 360 degree video of my OpenGL game.

Concerning the rendering: Is it enough to render it in OpenGL with a specific projection matrix? If yes, which one? Or can I render it into a cube map, and then encode it? (Which will require much more rendering power and be more complicated, which I want to avoid)

And how do I encode a 360 degree video with FFMPEG?

• Waiting for more savvy people to aswer this definitively, but I'm pretty sure you can't reach or exceed 180° FOV with a transform matrix. You'll have to cubemap it. – Quentin Aug 25 '16 at 9:51

For performance reasons GPUs use some simplified approach to projection and assume that you want to project your geometry on a plane, so it works only for FOV angles < 180 deg.

You could try some tricks in shaders and compute your custom projection there, but then you'll get into another problem - the assumption that polygons edges, being straight lines are rasterized also as straight lines. This assumption breaks for FOVs around 180 and more.

So basically what you can do is to render your scene into 6 separate views, and then join them in post-processing stage into one image.

And in case of encoding the video with FFMPEG - I think it works the same as for regular videos, you just need a different player to crop and pan the correct part of the frame.

• Can't I just render the front and back view each with 180 Degree FOV? (Isn't that what some 360 degree cameras do?) And how has the projection of the image to be before encoding? Let's say I have a 3600 x 1800 pixels resolution. Would there have to be 10 pixels per degree? I then probably could do a postprocessing step and draw the cubemap on a quad fetching the cubemap pixels with angels proportional to the quad coordinates. But this probably won't work with "normal" projection matrices, Is there some sort of "omnidirectional" projection I can use to render it? – woidler Aug 26 '16 at 12:43
• You can set up your camera to FOV 179 and see what you'll get, then compare it with a real camera results. You'll notice that cameras like GoPro bends straight lines, while 3D rendering doesn't - this is the simplification I was talking about. – kolenda Aug 26 '16 at 13:07
• You may also try some custom vertex shader that would compute angle to vertex around vertical axis and then set this vertex X position based on that angle - all of this instead of regular projection. This may help but I didn't try this and couldn't find any description on the web. – kolenda Aug 26 '16 at 13:09

You could also try Dual Paraboloid mapping. The idea is that you render your scene two times with a paraboloid projection - once for front and once for the back paraboloid. If you render the front and back paraboloid to two separate textures, you can combine them later with the inverse of the paraboloid projection to a 360 degree(-ish) image.
This technique is a bit more error prone than cubemapping around the edges where the two paraboloid maps connect, but can give you increased speed compared to rendering six times to cubemap faces.

UPDATE:
I've learned this technique from Kyle Hayward's blog sorry this I will post HLSL code but should be straightforward to convert to glsl):

Transform a vertex from view space to paraboloid space

output.Position = mul(input.Position, WorldView);
float L = length( output.Position.xyz );
output.Position = output.Position / L;
output.Position.z = output.Position.z + 1;
output.Position.x = output.Position.x / output.Position.z;
output.Position.y = output.Position.y / output.Position.z;
output.Position.z = (L - NearPlane) / (FarPlane - NearPlane);
output.Position.w = 1;


Map the dual paraboloid onto a sphere:

// calculate the front paraboloid map texture coordinates
float2 front;
front.x = R.x / (R.z + 1);
front.y = R.y / (R.z + 1);
front.x = .5f * front.x + .5f; //bias and scale to correctly sample a d3d texture
front.y = -.5f * front.y + .5f;
// calculate the back paraboloid map texture coordinates
float2 back;
back.x = R.x / (1 - R.z);
back.y = R.y / (1 - R.z);
back.x = .5f * back.x + .5f; //bias and scale to correctly sample a d3d texture
back.y = -.5f * back.y + .5f;
float4 forward = tex2D( FrontTex, front );    // sample the front paraboloid map
float4 backward = tex2D( BackTex, back );    // sample the back paraboloid map
float4 finalColor = max(forward, backward);

• I thought dual parabaloid mapping is done by rendering to a cubemap first and then putting the result in 2 paraboloid textures. I have no idea how to render one 180 degree paraboloid side with a single projection matrix. – HolyBlackCat Sep 30 '16 at 16:47
• You can do it by manually transforming a position into paraboloid space in the vertex shader. I will post a sample shortly. – János Turánszki Sep 30 '16 at 16:54
• Huh, looks interesting. – HolyBlackCat Sep 30 '16 at 17:28