In Unity, how do I dynamically manipulate parts of a mesh?

Question

I'm trying to achieve mesh manipulation of any mesh based on user input (mostly sliders). Here's an example of what I want. I am not sure whether to call this "morphing", as I am trying to change one part of the mesh without modifying the other parts.

How does this work?

Answer 1

Unity allows you to easily edit the vertices in a mesh.

The problem would be the editing only some parts of the mesh. You could edit vertices based on how close they are to an origin, like Blender's "proportional edit".

Answer 2

I haven't done this myself, but I know where you might want to look or start.

You would need to come up with some type of algorithm, I would look at the latest Game Programming Gems or GPU Gems to see if there is some insight. I do not think EA will part with their code. I have seen this sort of stuff you want to do work at a basic level in Spore. The earlier form of the technology might be publicly available.

I would find a method in Unity3D to get the local vertices of a control point on the mesh. Then you could apply some scaling function, after the scaling pass, you might need a clean up pass. You would need to clean up vertices, and shape fragments, especially if the geometry you wish to scale extends into the geometry you didn't want to change.

Also, if you're growing an animating object on moving parts, you may run into some issues with the rig.

You may need to define a scaling curve depending on the type of geometry you're optimizing.

Some sort of tessellation would allow you to work with less vertices and optimize your algorithm.

You might be able to define your own bones in Unity3D, and use that for some type of scaling.

Lastly, you could request an update to the Unity3D engine with this sort of feature.

Answer 3

You can use Blend Shapes for this, which would allow you to account for possibly animation problems in your modeling tool rather than further down the workflow, but if you want

Finer Control

Identify the regions you want to morph and assign weights to each vertex within those regions so that your modifications can taper off. That part is no different from regular vertex weighting for animation.

Next, you'll define the transformation matrix that will be applied to each vertex, where the minimum would be the identity matrix.

class BlendTarget
{
    public Vector3 Translation;
    public Quaternion Rotation;
    public Vector3 Scale;

    // The "center" of the transformation - used for rotation/scaling
    public Vector3 Origin;

    // Vertex indices to modify
    public int[] Indices;
    // Amount to modify the vertex - from 0 to 1
    public float[] Weights;

    public Matrix4x4 Matrix
    {
        get { return Matrix4x4.TRS(Translation, Rotation, Scale); }
    }

Such that, for each vertex in your predefined region:

    public void Apply(Mesh baseMesh, Mesh meshResult, float amount)
    {
        if (baseMesh == null)
            throw new ArgumentNullException("baseMesh");
        if (meshResult == null)
            throw new ArgumentNullException("meshResult");
        var identity = Matrix4x4.identity;

        var morphTarget = Matrix;
        // Matrix applied overall, weighted by "amount"
        morphTarget = MatrixLerp(ref identity, ref morphTarget, amount);
        for (var i = 0; i < Indices.Length; i++)
        {
            // Matrix applied to this vertex by its respective weight
            var vertexMatrix = MatrixLerp(ref identity,
                                          ref morphTarget,
                                          Weights[i]);
            var index = Indices[i];
            // Applying the transformation from the base mesh vertex
            // Additional calculations would be necessary if some regions overlap
            meshResult.vertices[index]
                = vertexMatrix.MultiplyPoint3x4(baseMesh.vertices[index] - Origin) + Origin;
        }
    }

    public static Matrix4x4 MatrixLerp(ref Matrix4x4 from, ref Matrix4x4 to, float s)
    {
        var ret = new Matrix4x4();
        for (var i = 0; i < 16; i++)
            ret[i] = Mathf.Lerp(from[i], to[i], s);
        return ret;
    }
}

What you do beyond that is up to your imagination!