Solving distortion with non-linear scaling in a shader

Question

I am currently working on implementing an older paper in Unity to show the progression of some graphics technologies, and part of that involves sprites being able to cast shadows. A trivial implementation would be to simply use a sprite of the object we already use, change it's colour, and project it underneath whatever casts that shadow.

Because shadows don't spread out directly the shadow casting object if one were to use a spherical light source, it would be nice to be able to adjust the width of the shadow so that the "top" of the shadow can be wider than the "base" (similar to a trapezium).

My approach has been to use a shader that can modify the vertices with a transformation matrix, as seen in the relevant code snippet below.

v2f vert(appdata_t IN) {
v2f OUT;
OUT.texcoord = IN.texcoord;
OUT.color = IN.color;
float sX = _ScaleX;
float sY = _ScaleY;
float scaleX = sX * OUT.texcoord.y + 1.0;

float4x4 transformMatrix = float4x4 (
    scaleX,0,0,0,
    0,sY,0,0,
    0,0,1,0,
    0,0,0,1
);

float4 skewedVertex = mul(transformMatrix, IN.vertex);
OUT.vertex = UnityObjectToClipPos(skewedVertex);

#ifdef PIXELSNAP_ON
OUT.vertex = UnityPixelSnap (OUT.vertex);
#endif

return OUT;

This unfortunately produces these strange results, which I can't seem to figure out the cause of, but my suspicions are that the UV-coordinates having an origin in one corner of the sprite might have at least something to do with it. Changing the pivot of the sprite from bottom to bottom left doesn't solve it, and instead produces different distortions. The shape displayed in the preview panel of the inspector looks right as well, so I am quite confused.

---

Thanks to a reply from DMGregory the problem likely arises from applying a non-affine transformation. In order to remedy this, one could use projective coordinates. What I am reasonably certain I've done is add an extra dimension to the input texcoord. This doesn't change anything when I scale everything with a set constant (like 2.0 in this case, picked very arbitrarily) as one would expect, since we would just be multiplying and dividing by the same value, but how does one go from this to proper projected coordinates described in the answer to another question, also by DMGregory.

v2f vert(appdata_t IN) {
    v2f OUT;
    OUT.texcoord = float3(IN.texcoord.xy * 2.0, 2.0);
    OUT.color = IN.color;
    float sX = _ScaleX;
    float sY = _ScaleY;
    float scaleX = sX * OUT.texcoord.y/2.0 + 1.0;

    float4x4 transformMatrix = float4x4 (
        scaleX,0,0,0,
        0,sY,0,0,
        0,0,1,0,
        0,0,0,1
    );
    
    float4 skewedVertex = mul(transformMatrix, IN.vertex);
    OUT.vertex = UnityObjectToClipPos(skewedVertex);
    
    #ifdef PIXELSNAP_ON
    OUT.vertex = UnityPixelSnap (OUT.vertex);
    #endif

    return OUT;
}

fixed4 frag(v2f IN) : SV_Target {
    float2 uv = IN.texcoord.xy / IN.texcoord.z;
    fixed4 c = tex2D(_MainTex, uv) * IN.color;
    c.rgb *= c.a;
    return c;
}

Answer 1

You missed a step:

Given a scale factor at each point (say, equal to the breadth of your strip at that spot), you can construct the 3D projective uvw coordinate from your regular 2D uv coordinate this way:

Vector3 uv3 = ((Vector3)uv2) * scale;
uv3.z = scale;

In your case that's:

float scaleX = sX * IN.texcoord.y + 1.0;
OUT.texcoord = float3(IN.texcoord.xy, 1.0f) * scaleX;

The the texture coordinates need to know about scaleX if you want them to be able to adapt to the distortion it introduces.