Draw onto an object's texture based on a raycast hit position in world space

Question

I am trying to get the world position of a pixel inside a fragmented shader.

Let me explain. I have followed a tutorial for a fragmented shader that let's me paint on objects. Right now it works through texture coordinates but I want it to work through pixel's world position. So when I click on a 3D model to be able to compare the Vector3 position (where the click happended) to the pixel Vector3 position and if the distance is small enough to lerp the color.

This is the setup I have. I created a new 3D project just for making the shader with the intent to export it later into my main project. In the scene I have the default main camera, directional light, a object with a script that shows me the fps and a default 3D cube with a mesh collider. I created a new material and a new Standard Surface Shader and added it to the cube. After that I assigned the next C# script to the cube with the shader and a camera reference.

If you want you can download the sample project here

Update: The problem right now is that the blit doesn't work as expected. If you change the shader script as how DMGregory said, remove the blit from the c# script and change the shader from the 3D model material to be the Draw shader, it will work as expected, but without any lighting. For my purposes I had to change distance(_Mouse.xyz, i.worldPos.xyz); to distance(_Mouse.xz, i.worldPos.xz); so it will paint a all the way through the other side. I also had to change a line in Gregory's shader from *o.vertex = float4(v.uv * 2.0f - 1.0f, 1, 1);* to o.vertex = UnityObjectToClipPos(v.vertex); because the blit was completely black. But before changing the o.vertex the uv was correctly unwrapped on the screen and the paint was applied as it should have been. So maybe there is a way save that on c#. For debugging I created a RenderTexture and every frame I am using Blit to update the texture and see what is going on. The render texture does not hold the right position as the object is colored. The 3D model I have has lot of geometry and as the paint goes to the other side it should be all over the place on the render texture...but right now it is just on line from the top to the bottom of the texture. Also I try to paint on the bottom half of the object and the render texture doesn't show anything. Only when I paint on the top half I can see red lines (the default painting color).

This is the code I am using.

Draw.shader

Shader "Unlit/Draw"
{
    Properties
    {
        _MainTex ("Texture", 2D) = "white" {}
        _Coordinate("Coordinate",Vector)=(0,0,0,0)
        _Color("Paint Color",Color)=(1,1,1,1)
    }
    SubShader
    {
        Tags { "RenderType"="Opaque" }
        LOD 100

        Pass
        {
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            #include "UnityCG.cginc"

            struct appdata
            {
                float4 vertex : POSITION;
                float2 uv : TEXCOORD0;
            };

            struct v2f
            {
                float2 uv : TEXCOORD0;
                float4 vertex : SV_POSITION;
            };

            sampler2D _MainTex;
            float4 _MainTex_ST;
            fixed4 _Coordinate,_Color;

            v2f vert (appdata v)
            {
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                o.uv = TRANSFORM_TEX(v.uv, _MainTex);
                return o;
            }

            fixed4 frag (v2f i) : SV_Target
            {
                // sample the texture
                fixed4 col = tex2D(_MainTex, i.uv);
                float draw =pow(saturate(1-distance(i.uv,_Coordinate.xy)),100);
                fixed4 drawcol = _Color * (draw * 1);
                return saturate(col + drawcol);
            }
            ENDCG
        }
    }
}

Draw.cs

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Draw : MonoBehaviour
{
    public Camera cam;
    public Shader paintShader;

    RenderTexture splatMap;
    Material snowMaterial,drawMaterial;

    RaycastHit hit;

    private void Awake()
    {
        Application.targetFrameRate = 200;
    }
    void Start()
    {
        drawMaterial = new Material(paintShader);
        drawMaterial.SetVector("_Color", Color.red);

        snowMaterial = GetComponent<MeshRenderer>().material;
        splatMap = new RenderTexture(1024, 1024, 0, RenderTextureFormat.ARGBFloat);
        snowMaterial.mainTexture = splatMap;
    }
    void Update()
    {
        if (Input.GetMouseButton(0))
        {
            if(Physics.Raycast(cam.ScreenPointToRay(Input.mousePosition),out hit))
            {
                drawMaterial.SetVector("_Coordinate", new Vector4(hit.textureCoord.x, hit.textureCoord.y, 0, 0));
                RenderTexture temp = RenderTexture.GetTemporary(splatMap.width, splatMap.height, 0, RenderTextureFormat.ARGBFloat);
                Graphics.Blit(splatMap, temp);
                Graphics.Blit(temp, splatMap, drawMaterial);
                RenderTexture.ReleaseTemporary(temp);
            }
        }
    }
}

Thank you!

Answer 1

First, if we want to get the object's world position, we need its world matrix. That's passed in automatically as unity_ObjectToWorld when rendering normally, but since we're drawing with Graphics.Blit we need to provide it ourselves.

Add a uniform like this above your vertex shader, where you're defining your _Coordinate uniform (which should be a float4, not a fixed4, or you're likely to have too little precision for your needs):

float4x4 _WorldMatrix;

Then you'll need to populate it in your Update script:

if(Physics.Raycast(cam.ScreenPointToRay(Input.mousePosition),out hit))
{
    // Pass the world position, not the texture coordinate.
    drawMaterial.SetVector("_Coordinate", hit.point);
    drawMaterial.SetMatrix("_WorldMatrix", transform.localToWorldMatrix);

    // You don't need the temporary or the first blit if you just draw transparently 
    // onto your texture, keeping what's there and adding instead of replacing.
    RenderTexture temp = RenderTexture.GetTemporary(splatMap.width, splatMap.height, 0, RenderTextureFormat.ARGBFloat);
    Graphics.Blit(splatMap, temp);
    Graphics.Blit(temp, splatMap, drawMaterial);
    RenderTexture.ReleaseTemporary(temp);
}

Now we need a member to pass our world position from the vertex shader to the fragment shader:

struct v2f
{
    float2 uv : TEXCOORD0;
    float4 vertex : SV_POSITION;
    float3 worldPos : TEXCOORD1;
};

Our vertex shader should spit out vertices in texture space, not clip space, and save the world position to pass down to the fragment shader.

v2f vert (appdata v)
{
    v2f o;
    o.worldPos = mul(_WorldMatrix, v.vertex);

    // Convert 0...1 UV space to -1...1 clip space.
    // Depending on the platform you might need to flip the y direction,
    // or flip your culling winding to get this to draw over your UV net correctly.
    o.vertex = float4(v.uv * 2.0f - 1.0f, 1, 1);

    o.uv = v.uv;
    return o;
}

Lastly, you want to compute your distance in world space:

fixed4 frag (v2f i) : SV_Target
{
    // sample the texture
    fixed4 col = tex2D(_MainTex, i.uv);
    float draw =pow(saturate(1-distance(i.worldPos,_Coordinate.xy)),100);
    fixed4 drawcol = _Color * (draw * 1);
    return saturate(col + drawcol);
}

Note that since you're doing this every frame, col + drawcol will increase very quickly, and saturate toward 1. But I'll trust that you're aware of that behaviour and it's intentional.