How can I draw outlines around 3D models?

Question

How can I draw outlines around 3D models? I'm referring to something like the effects in a recent Pokemon game, which appear to have a single-pixel outline around them:

Answer 1

I don't think any of the other answers here will achieve the effect in Pokémon X/Y. I can't know exactly how it's done, but I figured out a way which seems like pretty much what they do in the game.

In Pokémon X/Y, outlines are drawn both around the silhouette edges and on other non-silhouette edges (like where Raichu's ears meet his head in the following screenshot).

Looking at Raichu's mesh in Blender, you can see the ear (highlighted in orange above) is just a separate, disconnected object which intersects the head, creating an abrupt change in the surface normals.

Based on that, I tried generating the outline based on the normals, which requires rendering in two passes:

First pass: Render the model (textured and cel-shaded) without the outlines, and render the camera-space normals to a second render target.

Second pass: Do a full-screen edge detection filter over the normals from the first pass.

The first two images below show the outputs of the first pass. The third is the outline by itself, and the last is the final combined result.

Here's the OpenGL fragment shader I used for edge detection in the second pass. It's the best I was able to come up with, but there might be a better way. It's probably not very well optimized either.

// first render target from the first pass
uniform sampler2D uTexColor;
// second render target from the first pass
uniform sampler2D uTexNormals;

uniform vec2 uResolution;

in vec2 fsInUV;

out vec4 fsOut0;

void main(void)
{
  float dx = 1.0 / uResolution.x;
  float dy = 1.0 / uResolution.y;

  vec3 center = sampleNrm( uTexNormals, vec2(0.0, 0.0) );

  // sampling just these 3 neighboring fragments keeps the outline thin.
  vec3 top = sampleNrm( uTexNormals, vec2(0.0, dy) );
  vec3 topRight = sampleNrm( uTexNormals, vec2(dx, dy) );
  vec3 right = sampleNrm( uTexNormals, vec2(dx, 0.0) );

  // the rest is pretty arbitrary, but seemed to give me the
  // best-looking results for whatever reason.

  vec3 t = center - top;
  vec3 r = center - right;
  vec3 tr = center - topRight;

  t = abs( t );
  r = abs( r );
  tr = abs( tr );

  float n;
  n = max( n, t.x );
  n = max( n, t.y );
  n = max( n, t.z );
  n = max( n, r.x );
  n = max( n, r.y );
  n = max( n, r.z );
  n = max( n, tr.x );
  n = max( n, tr.y );
  n = max( n, tr.z );

  // threshold and scale.
  n = 1.0 - clamp( clamp((n * 2.0) - 0.8, 0.0, 1.0) * 1.5, 0.0, 1.0 );

  fsOut0.rgb = texture(uTexColor, fsInUV).rgb * (0.1 + 0.9*n);
}

And before rendering the first pass I clear the normals' render target to a vector facing away from the camera:

glDrawBuffer( GL_COLOR_ATTACHMENT1 );
Vec3f clearVec( 0.0, 0.0, -1.0f );
// from normalized vector to rgb color; from [-1,1] to [0,1]
clearVec = (clearVec + Vec3f(1.0f, 1.0f, 1.0f)) * 0.5f;
glClearColor( clearVec.x, clearVec.y, clearVec.z, 0.0f );
glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );

I read somewhere (I'll put a link in the comments) that the Nintendo 3DS uses a fixed-function pipeline instead of shaders, so I guess this can't be the way it's done in the game exactly, but for now I'm convinced my method is close enough.

Answer 2

This effect is particularly common in games that make use of cel shading effects, but is actually something that can be applied independently of the cel shading style.

What you are describing is called "feature edge rendering," and is in general process of highlighting the various contours and outlines of a model. There are many techniques available and many papers on the subject.

A simple technique is to render only the silhouette edge, the outmost outline. This can be done as simply as rendering the original model with a stencil write, and then rendering it again in thick wireframe mode, only where there was no stencil value. See here for an example implementation.

That will not highlight the interior contour and crease edges, though (as shown in your pictures). Generally, to do that effectively, you need to extract information about the mesh's edges (based on discontinuities in the face normals on either side of the edge, and build up a data structure representing each edge.

You can then write shaders to extrude or otherwise render those edges as regular geometry overtop your base model (or in conjunction with it). The position of an edge, and normals of the adjacent faces relative to the view vector, are used to determine if a specific edge can be drawn.

You can find further discussion, details and papers with various examples on the internet. For example:

• http://www.marctenbosch.com/npr_edges/
• http://www.j3l7h.de/publications/halo_web.pdf
• http://graphics.cs.williams.edu/papers/EdgesNPAR04/index.html

Answer 3

For smooth models (very important), this effect is fairly simple. In your fragment/pixel shader you will need the normal of the fragment being shaded. If it is very close to perpendicular (dot(surface_normal,view_vector) <= .01 - you might need to play with that threshold) then color the fragment black instead of its usual color.

This approach "consumes" a little bit of the model to do the outline. This may or may not be what you want. It's very difficult to tell from the Pokemon picture if this is what is being done. It depends on if you expect the outline to be included in any silhouette of the character or if you'd rather have the outline enclose the silhouette (which requires a different technique).

The highlight will be on any part of the surface where it transitions from front-facing to back-facing, including "inner edges" (like the legs on the green Pokemon, or its head - some other techniques would not add any outline to those).

Objects that have hard, non-smooth edges (like a cube) will not receive a highlight in the desired locations with this approach. That means this approach is not an option at all in some cases; I have no idea if Pokemon models are all smooth or not.

Answer 4

The simplest way to do this, common on older hardware before pixel/fragment shaders, and still used on mobile, is to duplicate the model, reverse the vertex winding order so that the model displays inside out (or if you want to, you can do this in your 3D asset creation tool, say Blender, by flipping surface normals -- same thing), then expand the entire duplicate slightly around it's centre, and finally colour/texture this duplicate completely black. This results in outlines around your original model, if it's a simple model such as a cube. For more complex models with concave forms (such as that in the image below), it is necessary to manually tweak the duplicate model to be somewhat "fatter" than its original counterpart, like a Minkowski Sum in 3D. You could start by pushing each vertex out a bit along its normal to form the outline mesh, as Blender's Shrink/Fatten transform does.

Screen space / pixel shader approaches tend to be slower and harder to implement well, but OTOH don't double the number of vertices in your world. So if you're doing high poly work, best opt for that approach. Given modern console and desktop capacity for processing geometry, I'd not worry about a factor of 2 at all. Cartoon-style = low poly for sure, thus duplicating geometry is easiest.

You can test the effect for yourself in e.g. Blender without touching any code. Outlines should look like the image below, note how some are internal, e.g. under arm. More detail here.

.

Answer 5

One of the ways to make an outline is to use our models normal vectors. Normal vectors are vectors that are perpendicular to their surface (pointing away from the surface). The trick here is to split your character model into two parts. The vertices that are facing the camera and the vertices that are facing away from the camera. We will call them FRONT and BACK respectively.

For the outline we take our BACK vertices and move them slightly in the direction of their normal vectors. Think about it like making the part of our character that is facing away from the camera a little fatter. After we have that done, we assign them a color of our choice and we have a nice outline.

Shader "Custom/OutlineShader" {
    Properties {
        _MainTex ("Base (RGB)", 2D) = "white" {}
        _Outline("Outline Thickness", Range(0.0, 0.3)) = 0.002
        _OutlineColor("Outline Color", Color) = (0,0,0,1)
    }

    CGINCLUDE
    #include "UnityCG.cginc"

    sampler2D _MainTex;
    half4 _MainTex_ST;

    half _Outline;
    half4 _OutlineColor;

    struct appdata {
        half4 vertex : POSITION;
        half4 uv : TEXCOORD0;
        half3 normal : NORMAL;
        fixed4 color : COLOR;
    };

    struct v2f {
        half4 pos : POSITION;
        half2 uv : TEXCOORD0;
        fixed4 color : COLOR;
    };
    ENDCG

    SubShader 
    {
        Tags {
            "RenderType"="Opaque"
            "Queue" = "Transparent"
        }
        
        Pass{
            Name "OUTLINE"

            Cull Front

            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            v2f vert(appdata v)
            {
                v2f o;
                o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
                half3 norm = mul((half3x3)UNITY_MATRIX_IT_MV, v.normal);
                half2 offset = TransformViewToProjection(norm.xy);
                o.pos.xy += offset * o.pos.z * _Outline;
                o.color = _OutlineColor;
                return o;
            }

            fixed4 frag(v2f i) : COLOR
            {
                fixed4 o;
                o = i.color;
                return o;
            }
            ENDCG
        }

        Pass 
        {
            Name "TEXTURE"

            Cull Back
            ZWrite On
            ZTest LEqual

            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            v2f vert(appdata v)
            {
                v2f o;
                o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
                o.uv = TRANSFORM_TEX(v.uv, _MainTex);
                o.color = v.color;
                return o;
            }

            fixed4 frag(v2f i) : COLOR 
            {
                fixed4 o;
                o = tex2D(_MainTex, i.uv.xy);
                return o;
            }
            ENDCG
        }
    } 
}

Line 41: The “Cull Front” setting tells the shader to perform a culling on the front facing vertices. It means that we will ignore all the front facing vertices in this pass. We are left with the BACK side that we want to manipulate a little.

Lines 51-53: The math of moving vertices along their normal vectors.

Line 54: Setting the vertex color to our color of choice defined in the shaders properties.

Useful Link: http://wiki.unity3d.com/index.php/Silhouette-Outlined_Diffuse

---

Update

another example

   Shader "Custom/CustomOutline" {
            Properties {
                _Color ("Color", Color) = (1,1,1,1)
                _Outline ("Outline Color", Color) = (0,0,0,1)
                _MainTex ("Albedo (RGB)", 2D) = "white" {}
                _Glossiness ("Smoothness", Range(0,1)) = 0.5
                _Size ("Outline Thickness", Float) = 1.5
            }
            SubShader {
                Tags { "RenderType"="Opaque" }
                LOD 200
 
                // render outline
 
                Pass {
                Stencil {
                    Ref 1
                    Comp NotEqual
                }
           
                Cull Off
                ZWrite Off
       
                    CGPROGRAM
                    #pragma vertex vert
                    #pragma fragment frag
                    #include "UnityCG.cginc"
                    half _Size;
                    fixed4 _Outline;
                    struct v2f {
                        float4 pos : SV_POSITION;
                    };
                    v2f vert (appdata_base v) {
                        v2f o;
                        v.vertex.xyz += v.normal * _Size;
                        o.pos = UnityObjectToClipPos (v.vertex);
                        return o;
                    }
                    half4 frag (v2f i) : SV_Target
                    {
                        return _Outline;
                    }
                    ENDCG
                }
           
                Tags { "RenderType"="Opaque" }
                LOD 200
   
                // render model
       
                Stencil {
                    Ref 1
                    Comp always
                    Pass replace
                }
 
 
                CGPROGRAM
                // Physically based Standard lighting model, and enable shadows on all light types
                #pragma surface surf Standard fullforwardshadows
                // Use shader model 3.0 target, to get nicer looking lighting
                #pragma target 3.0
                sampler2D _MainTex;
                struct Input {
                    float2 uv_MainTex;
                };
                half _Glossiness;
                fixed4 _Color;
                void surf (Input IN, inout SurfaceOutputStandard o) {
                    // Albedo comes from a texture tinted by color
                    fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
                    o.Albedo = c.rgb;
                    // Metallic and smoothness come from slider variables
                    o.Smoothness = _Glossiness;
                    o.Alpha = c.a;
                }
                ENDCG
            }
            FallBack "Diffuse"
        }
 

Answer 6

The most common way I've seen this done is via a second render pass on your model. Essentially, duplicate it and flip the normals, and shove that into a vertex shader. In the shader, scale each vertex along its normal. In the pixel/fragment shader, draw black. That'll give you both external and internal outlines, like around lips, eyes, etc. This is actually a fairly cheap draw call, if nothing else it's generally cheaper than post processing the line, depending on the number of models and their complexity. Guilty Gear Xrd uses this method because it's easy to control the thickness of the line via vertex color.

The second way of doing inner lines I learned from the same game. In your UV map, align your texture along the u or v axis, particularly in areas where you want an inner line. Draw a black line along either axis, and move your UV coordinates into or out of that line to create the inner line.

See the video from GDC for a better explanation: https://www.youtube.com/watch?v=yhGjCzxJV3E

Answer 7

One of a great ways of doing it is to render your scene on a Framebuffer texture, to then render that texture while doing a Sobel Filtering on every pixel, which is an easy technique for edge detection. This way you can not only make the scene pixelated (setting a low resolution to the Framebuffer texture), but also have access to every pixel values to make Sobel work.