# How can I get shadows to appear correctly when using a custom projection matrix?

EDIT: This has been officially marked as a bug in Unity. If anyone else has this issue, you can vote here for Unity to fix it.

As mentioned in this thread, I'm using a custom projection matrix in Unity. Unfortunately, this breaks the default shadow system in Unity. Thus, I was wondering if anyone knew how to possibly recalculate the shadows with the projection matrix, either through code or a shader?

I'm using this script to modify Unity's projection matrix with my own custom projection matrix:

public Transform[] Corners;
public Transform lookTarget;
public bool drawNearCone, drawFrustum;

Camera theCam;

void Start () {
theCam = camera;
}

void Update () {
Vector3 pa, pb, pc, pd;
pa = Corners[0].position; //Bottom-Left
pb = Corners[1].position; //Bottom-Right
pc = Corners[2].position; //Top-Left
pd = Corners[3].position; //Top-Right

Vector3 pe = theCam.transform.position;// eye position

Vector3 vr = ( pb - pa ).normalized; // right axis of screen
Vector3 vu = ( pc - pa ).normalized; // up axis of screen
Vector3 vn = Vector3.Cross( vr, vu ).normalized; // normal vector of screen

Vector3 va = pa - pe; // from pe to pa
Vector3 vb = pb - pe; // from pe to pb
Vector3 vc = pc - pe; // from pe to pc
Vector3 vd = pd - pe; // from pe to pd

float n = -lookTarget.InverseTransformPoint( theCam.transform.position ).z; // distance to the near clip plane (screen)
float f = theCam.farClipPlane; // distance of far clipping plane
float d = Vector3.Dot( va, vn ); // distance from eye to screen
float l = Vector3.Dot( vr, va ) * n / d; // distance to left screen edge from the 'center'
float r = Vector3.Dot( vr, vb ) * n / d; // distance to right screen edge from 'center'
float b = Vector3.Dot( vu, va ) * n / d; // distance to bottom screen edge from 'center'
float t = Vector3.Dot( vu, vc ) * n / d; // distance to top screen edge from 'center'

Matrix4x4 p = new Matrix4x4(); // Projection matrix
p[0, 0] = 2.0f * n / ( r - l );
p[0, 2] = ( r + l ) / ( r - l );
p[1, 1] = 2.0f * n / ( t - b );
p[1, 2] = ( t + b ) / ( t - b );
p[2, 2] = ( f + n ) / ( n - f );
p[2, 3] = 2.0f * f * n / ( n - f );
p[3, 2] = -1.0f;

theCam.projectionMatrix = p; // Assign matrix to camera

if ( drawNearCone ) { //Draw lines from the camera to the corners f the screen
Debug.DrawRay( theCam.transform.position, va, Color.blue );
Debug.DrawRay( theCam.transform.position, vb, Color.blue );
Debug.DrawRay( theCam.transform.position, vc, Color.blue );
Debug.DrawRay( theCam.transform.position, vd, Color.blue );
}

if ( drawFrustum ) DrawFrustum( theCam ); //Draw actual camera frustum

}

Vector3 ThreePlaneIntersection ( Plane p1, Plane p2, Plane p3 ) { //get the intersection point of 3 planes
return ( ( -p1.distance * Vector3.Cross( p2.normal, p3.normal ) ) +
( -p2.distance * Vector3.Cross( p3.normal, p1.normal ) ) +
( -p3.distance * Vector3.Cross( p1.normal, p2.normal ) ) ) /
( Vector3.Dot( p1.normal, Vector3.Cross( p2.normal, p3.normal ) ) );
}

void DrawFrustum ( Camera cam ) {
Vector3[] nearCorners = new Vector3[4]; //Approx'd nearplane corners
Vector3[] farCorners = new Vector3[4]; //Approx'd farplane corners
Plane[] camPlanes = GeometryUtility.CalculateFrustumPlanes( cam ); //get planes from matrix
Plane temp = camPlanes[1]; camPlanes[1] = camPlanes[2]; camPlanes[2] = temp; //swap [1] and [2] so the order is better for the loop

for ( int i = 0; i < 4; i++ ) {
nearCorners[i] = ThreePlaneIntersection( camPlanes[4], camPlanes[i], camPlanes[( i + 1 ) % 4] ); //near corners on the created projection matrix
farCorners[i] = ThreePlaneIntersection( camPlanes[5], camPlanes[i], camPlanes[( i + 1 ) % 4] ); //far corners on the created projection matrix
}

for ( int i = 0; i < 4; i++ ) {
Debug.DrawLine( nearCorners[i], nearCorners[( i + 1 ) % 4], Color.red, Time.deltaTime, false ); //near corners on the created projection matrix
Debug.DrawLine( farCorners[i], farCorners[( i + 1 ) % 4], Color.red, Time.deltaTime, false ); //far corners on the created projection matrix
Debug.DrawLine( nearCorners[i], farCorners[i], Color.red, Time.deltaTime, false ); //sides of the created projection matrix
}
}


This script skews the view frustum based on the camera's relative position to a look target. It is to simulate a head-tracking program so that the viewing angle changes based on where the person is relative to the scene. As shown in this video.

The issue of Unity's shadows not working when a custom projection matrix is used is very well known among the Unity developer community. The custom projection matrix and what it is used for does not matter. What matters is that shadows either draw incorrectly or not at all in Forward, and deferred rendering in Unity.

Specifically, as mentioned before, I'm interested in if anyone has found a solution for the shadow problem, as there are no well documented solutions. Thus, once again, I was wondering if anyone knew how to possibly recalculate the shadows, or project their own shadows, with the projection matrix, either through code or a shader?

• As a further note, in the recently released Unity 5, shadows are non-existent with a custom projection matrix. – Ben BearFish Mar 5 '15 at 1:18