0
\$\begingroup\$

I'm using the Autopano Giga 4.4 and Autopano Video Pro 2.6 to stitch and create spherical content, but I've recently moved into wanting to create 360° tours via Unity. I know this is partly a developers question, but if anyone out there can help me with the right specs for exporting a spherical image or video from Giga to use to create the image for the inside of the sphere on Unity, I'd appreciate it!

Experimenting with Premiere and FCPX to edit highlight reels of spherical content, I've had seamless experiences created. For some reason in Unity, the images appear to be stretched around the sphere wrong and it's creates issues on the top and bottom of the image:

Example showing the artifacts at the base of the sphere

The image shown is the bottom of the sphere. I'm using the Android platform to hopefully turn these tours into apps when completed. Thank you for any help!

\$\endgroup\$
  • \$\begingroup\$ What material or shader are you using to map the content onto the sphere? \$\endgroup\$ – DMGregory Sep 8 '17 at 17:33
  • \$\begingroup\$ I've been using the Insideout shader for the most part, but I don't remember where I found it at @DMGregory \$\endgroup\$ – Dale Falk Jr. Sep 8 '17 at 18:24
  • \$\begingroup\$ Do you happen to know if your videos are in equirectangular format, or some other style like a cubemap? \$\endgroup\$ – DMGregory Sep 8 '17 at 19:57
  • \$\begingroup\$ They're in equirectangular, I've got them at both 8192x4096 and 3840x1920, but both have the same result of crunching at the north and south poles. That's why I figured it was something on the Unity side with the sphere problems. I'm just a newbie on it as a whole, so knew I needed to reach out @DMGregory \$\endgroup\$ – Dale Falk Jr. Sep 8 '17 at 20:06
3
\$\begingroup\$

This isn't a matter of codecs and resolution, you just need to match the texture mapping onto the sphere geometry to the mapping used in the video format - in this case equirectangular.

Equirectangular mapping (also called latitude/longitude or spherical coordinates) is non-linear. That means normal UV mapping can only approximate it - quite badly at the poles, in fact.

Example of Unity sphere's UV mapping at its lowest point, versus a calculated equirectangular mapping

To fix this, we can calculate our own texture coordinate per fragment using the direction to the fragment being drawn, resulting in a perfect match. Here's an example shader that does this:

Shader "Unlit/PanoramaShader"
{
    Properties
    {
        _MainTex ("Texture", 2D) = "white" {}
    }
    SubShader
    {
        Tags { "RenderType" = "Opaque" }
        LOD 100

        // Render the object inside-out.
        Cull Front

        Pass
        {
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #pragma multi_compile_fog

            #include "UnityCG.cginc"

            struct appdata
            {
                float4 vertex : POSITION;
            };

            struct v2f
            {               
                float4 vertex : SV_POSITION;
                // Pass a view direction instead of a UV coordinate.
                float3 direction : TEXCOORD0;
            };

            sampler2D _MainTex;
            float4 _MainTex_ST;

            v2f vert (appdata v)
            {
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                // Compute worldspace direction from the camera to this vertex.
                o.direction = mul(unity_ObjectToWorld, v.vertex).xyz 
                               - _WorldSpaceCameraPos;
                return o;
            }

            fixed4 frag (v2f i) : SV_Target
            {
                // Convert the direction to the fragment into latitude & longitude.
                float3 pos = normalize(i.direction);
                float2 uv;       
                uv.x = atan2(pos.z, pos.x)* 0.5f;
                uv.y = asin(pos.y);

                // Scale and shift into the 0...1 texture coordinate range.
                uv = uv / 3.141592653589f + 0.5f;

                // Used directly, we'll get a texture filtering seam
                // where the longitude wraps around from 1 to 0.
                // This fixes that (you can skip this if your videos don't mipmap)
                float2 dx = ddx(uv);
                float2 dy = ddy(uv);
                float2 du = float2(dx.x, dy.x);
                du -= (abs(du) > 0.5f) * sign(du);
                dx.x = du.x;
                dy.x = du.y;

                // In case you want to rotate your view using the texture x-offset.
                uv.x += _MainTex_ST.z;     

                // Sample the texture with our calculated UV & seam fixup.
                fixed4 col = tex2Dgrad(_MainTex, uv, dx, dy);

                return col;
            }
            ENDCG
        }
    }
}

As a bonus, since we're just using the direction to the vertex, your object doesn't need to be a sphere - you can render this on a box or even a singe quad parented in front of the camera and it will act as a "magic window" into the 360 scene.

\$\endgroup\$
  • \$\begingroup\$ This works great for Mono videos, but for stereo it does not. Tiling values are not taken into account. Any ideas why or how to fix this? \$\endgroup\$ – JackMini36 Sep 24 '18 at 11:29
  • \$\begingroup\$ Can you describe what you mean by tiling values? \$\endgroup\$ – DMGregory Sep 24 '18 at 11:44
  • \$\begingroup\$ the texture tiling of a material shader: imgur.com/a/XMsDU6t \$\endgroup\$ – JackMini36 Sep 24 '18 at 11:58
  • \$\begingroup\$ Ah, see the line // In case you want to rotate your view using the texture z-offset. uv.x += _MainTex_ST.z; ? That will use the x tiling offset to rotate the view. You could add one for y too if you wanted, but note that changing these does not correctly model parallax observed by two eyes. It would shift the whole video together, rather than shifting nearby objects and leaving distant ones fixed. For stereo you'd need two separate videos or a depth map to use to distort/extrude the image. \$\endgroup\$ – DMGregory Sep 24 '18 at 12:04
  • \$\begingroup\$ the thing is I have a single video file with both projections (in a top bottom format), and I would like to set the y tiling to 0.5 for one eye etc. The above code only seem to take into account the offset instead of tiling though \$\endgroup\$ – JackMini36 Sep 24 '18 at 12:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.