Techniques to make 3D models to look like oil paintings

Question

I intend to develop a fantasy 3D game focused on warships, with a look similar to oil paintings.

The reasoning behind this decision is that the art style would allow for a sense of wonder and curiosity.

Currently, the most viable technique I could think of was to make the 3D models into "onions" with multiple layers one after another with multiple types of strokes. But this seems very labor intensive to me, and would probably make the models too heavy to run efficiently (I haven't tested it yet, I'm looking for references before trying).

The main problem is that I couldn't find anything concrete and free to use that could match an oil painting.

The most pertinent example I come across is Arcane, where they use Digital Painting to texture (I think it's texturing and not a machine learning program that paints frame by frame) the characters.

I also found a PS4 game which is a game creation tool called "Dreams", however I don't own a PS4 and I wouldn't be able to extract the models created there for Blender either.

I also found some programs that use machine learning to turn videos into animated oil paintings, but it only works with videos and I don't have the knowledge to try to turn this into an engine (assuming it would be possible).

(This one program in specific is called "EbSynth")

Or, at the end of the day, am I just having too unrealistic expectations, just like anyone excited with a new project?

Answer 1

A kuwahara shader would be a good start, with bright colors as an input. First hit resulted in this:

Shader "Unlit/Oil Painting"
{
    Properties
    {
        _MainTex("Texture", 2D) = "white" {}
        _Radius ("Radius", Range(0, 10)) = 0
    }
    SubShader
    {
        Blend SrcAlpha OneMinusSrcAlpha
        Pass
        {
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #pragma target 3.0
            #include "UnityCG.cginc"
 
            struct v2f {
                float4 pos : SV_POSITION;
                half2 uv : TEXCOORD0;
            };
 
            sampler2D _MainTex;
            float4 _MainTex_ST;
 
            v2f vert(appdata_base v) {
                v2f o;
                o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
                o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
                return o;
            }
 
            int _Radius;
            float4 _MainTex_TexelSize;
 
            float4 frag (v2f i) : SV_Target
            {
                half2 uv = i.uv;
 
                float3 mean[4] = {
                    {0, 0, 0},
                    {0, 0, 0},
                    {0, 0, 0},
                    {0, 0, 0}
                };
 
                float3 sigma[4] = {
                    {0, 0, 0},
                    {0, 0, 0},
                    {0, 0, 0},
                    {0, 0, 0}
                };
 
                float2 start[4] = {{-_Radius, -_Radius}, {-_Radius, 0}, {0, -_Radius}, {0, 0}};
 
                float2 pos;
                float3 col;
                for (int k = 0; k < 4; k++) {
                    for(int i = 0; i <= _Radius; i++) {
                        for(int j = 0; j <= _Radius; j++) {
                            pos = float2(i, j) + start[k];
                            col = tex2Dlod(_MainTex, float4(uv + float2(pos.x * _MainTex_TexelSize.x, pos.y * _MainTex_TexelSize.y), 0., 0.)).rgb;
                            mean[k] += col;
                            sigma[k] += col * col;
                        }
                    }
                }
 
                float sigma2;
 
                float n = pow(_Radius + 1, 2);
                float4 color = tex2D(_MainTex, uv);
                float min = 1;
 
                for (int l = 0; l < 4; l++) {
                    mean[l] /= n;
                    sigma[l] = abs(sigma[l] / n - mean[l] * mean[l]);
                    sigma2 = sigma[l].r + sigma[l].g + sigma[l].b;
 
                    if (sigma2 < min) {
                        min = sigma2;
                        color.rgb = mean[l].rgb;
                    }
                }
                return color;
            }
            ENDCG
        }
    }
}

(Accoding to his youtube channel, My shaders are free-to-use. No need any credit. Source here: Oil Shader)

A fast way would be to apply it to the camera and render everything with it.

• Create a new material

• Assign the shader to the material

• Create a mini script for using it

  [ExecuteInEditMode] public class CameraWithShader: MonoBehaviour {

   public Material material;
   void OnRenderImage (RenderTexture source, RenderTexture destination) {
       Graphics.Blit(source, destination, material);
   }
  

  }

• Place the script on your camera

• Assign the material to the script