# How does 3D mesh morphing work?

In the Sims 4, you can drag to reshape the face and body when you create a sim. You can see a similar system implemented in this Body Visualizer.

How can I implement similar geometry morphing in my game?

In the Sims 4, you can drag to reshape the face when you create a sim. How is the geometry morphing implemented?

In general, how do you code a system that morphs different parts of a mesh?

I don't know if there is any commonly agreed way of doing this, but basically what you are asking is not as complicated as it might seem. You create a basic unmodified mesh. Then you add functions with specific parameters and boundary conditions at local areas of your mesh. To give you a simple 2d example, let's consider the function in the following picture is your base mesh:

You can modify the shape by adding arbitrary y values at any x position. If you want to have a smooth curve you need a mathematical function, that describes your modification or morphing curve. However, this function needs to fulfill the condition that it's gradient and function value is 0 at its interval boundaries, otherwise, you will see some kinks or jumps. One function that fulfills this requirement is the cosine function minus 1 in the interval 0 to 2Pi:

Now all you have to do is to scale the functions intervals and function values to your liking and to the ranges of the mesh area you want to be affected. After some shifting and scaling you might get a morphing function like this:

Finding the right parameters and their limits for such a function is the job of the artist that creates the mesh. Of course, artists are usually no mathematicians and you can't expect them to have a solid knowledge of suitable functions for this task, especially not in 3d. Luckily, there is a set of universal functions for this task: Splines (search for B-Splines or NURBs). These special functions can easily be "bend" to any shape and artists usually already use them to generate the base meshes.

I have to state here, that I am not an industry professional and not an artist. Just a passionate hobby programmer that works on his own game engine and reads a lot of related literature. But I might not be up to date anymore. So there might be other (smarter) approaches to this. For example, you can probably get more realistic results by using a model based on muscles, bones, etc. and calculate the corresponding skin surface from their sizes and shapes. However, this would just move the problem to another area (shape of muscles and bones).

If you want to understand a little bit better, how the plots above are generated, the Python code I used to generate them is attached below. Modify the 3 parameters that control the position, width, and height to see how they affect the result.

import numpy as np
import matplotlib.pyplot as plt

# parameters
morph_height = 0.2
morph_width = 80
morph_shift = 25

# original shape
x = np.arange(-1, 1.001, 0.01)
y_base = -0.5 * x * x + 1

# generate morph function
y_morph = np.zeros(len(x))
for i in range(morph_width + 1):
# scale function interval
p = 2 * np.pi * i / morph_width
# scale height and write to affected area
y_morph[i + morph_shift] = morph_height * (np.cos(p) - 1) / 2

y_total = y_base + y_morph

plt.plot(x, y_base)
plt.ylim(0, 2)
plt.xlabel("x")
plt.ylabel("y")
plt.title("unmodified shape")

plt.figure()
plt.plot(x, y_morph)
plt.ylim(-1, 1)
plt.xlabel("x")
plt.ylabel("y")
plt.title("morph function")

plt.figure()
plt.plot(x, y_total)
plt.ylim(0, 2)
plt.xlabel("x")
plt.ylabel("y")
plt.title("result")

plt.show()


We do not know the details of how they did it… However, there is a modding community for The Sims and they manage.

The idea is as follows: There is one base model for body type. There are series of sliders that can modify the base model. Each one will have a set of vertex they can modify, and a second target model. Then moving the slider is interpolating the vertex between the base model and the target model.

It gets complicated when multiple slider can control the same vertex. I don't know how they do it in the game, probably only the developers know…

What follows is how I would tackle the problem. For each vertex, let us work in a coordinate system with origin at the position of the vertex on the base model. In other words, we will work with the offset by which we are going to move the vertex.

Now, each slider will try to move the vertex in a particular direction. That is, each slider, would independently apply a particular offset. We can compute the offset for each slider, and then we just add them.

Why add? Because that way all sliders that are at 0 have no effect.

I have not implemented this. What I'm saying that I would start there.

My rational for the above approach is to imagine a space of posible inputs. This space would have a dimension for each slider. And we know the offset we want to get at some particular points, which happen to be along the axis. For the rest of the space we want the output to be smooth and continuous. Doing a linear combination of them makes sense. And that is just a weighted sum of the difference between the target model of each slider and the base model, where the value of the slider is the weight.

I hope that makes sense.

If we wanted to that in GPU, we would do it in a vertex shader… which means we need to upload to the GPU all the coordinates for all the target models too (here is an idea: use textures)… and this computation would be happening all the time. I think that is overkill. Do this computation once, in CPU or GPU, and then just upload the result to the GPU.

There are a several ways to do it, but the general principle is the same:

You can morph between more than 2 meshes, but for this example, let's assume 2; the logic remains exactly the same for more of them.

At the root, each vertex is essentially two vertices (one from the source mesh, and one from the destination mesh) and you interpolate between the two, based on where you are in the morph. So, on one side you have the position of Mesh1, on the other side, the position of Mesh2 and then, anything in between.

This can be trivially done on the GPU.

The main difficulty is how to match vertices from mesh1 to mesh2. If you don't match them properly, your mesh will completely deform and collapse as it undergoes morphing. For example, interpolating between a vertex that is part of the nose and one part of the ear is not going to look very good.

There are multiple ways this can be done and none of them are ideal because you will always encounter edge cases. When there is no trivial solution for those, usually vertices end up manually tagged together by the artist.

If you have a mesh that is mostly convex, it is a lot easier: you can determine the center of both meshes and then, for each vertex compute a line going through the vertex and that center, for both meshes. For each vertex on mesh1, you can match a vertex on mesh2 by comparing the angles between the lines and taking the one with the smallest one. This is how it used to be done a very long time ago.

An improvement to this was eventually developed and instead of looking at vertices, you'd start considering triangles, and the center of each triangle. Using this, you'd match triangles and from there match vertices.

And, many more techniques were developed, but since you're starting with this, these two will give you some quick results you can implement easily and improve upon; there are more advanced methods, but they're not trivial to implement like these and may require work in the tool pipeline.

Now we're been looking only at morphing the meshes, but many effects you see are actually interpolating between multiple bone setups. Morphing the mesh will go from Face A to Face B for example (and you can cross fade textures for example), but body types, facial expressions, etc use exactly the same system but instead of interpolating between vertices, you interpolate between the different bone positions. So each vertex's position is weighted to several bones in your rig and you can interpolate those positions directly and make someone fat, tall, etc. So, keep in mind that many effects are really not the mesh itself morphing. One face with multiple bone setups will give you a very large number of faces for example.

Now, there are plug-ins made for this in 3DSMax and Maya that will allow you to edit the vertex groups so the matching can be done visually. This is the simplest way to do it, but if you begin in this, I wouldn't recommend you spend too much time on the tool side because this can be extremely time consuming and you'll spend a year coding tools and exporters instead of a game (I've seen that happen a lot). To get something going quick, use relatively similar meshes and automatic vertex matching because you'll be able to code that very fast and it may work well enough for your application.