How do I apply skeletal animation from a .x (Direct X) file?

Question

Using the .x format to export a model from Blender, I can load a mesh, armature and animation. I have no problems generating the mesh and viewing models in game. Additionally, I have animations and the armature properly loaded into appropriate data structures.

My problem is properly applying the animation to the models. I have the framework for applying the models and the code for selecting animations and stepping through frames.

From what I understand, the AnimationKeys inside the AnimationSet supplies the transformations to transform the bind pose to the pose in the animated frame. As small example:

Animation {
  {Armature_001_Bone}
  AnimationKey { 
    2; //Position
    121; //number of frames
    0;3;     0.000000, 0.000000, 0.000000;;,
    1;3;     0.000000, 0.000000, 0.005524;;,
    2;3;     0.000000, 0.000000, 0.022217;;,
    ...
  }
  AnimationKey { 
    0; //Quaternion Rotation 
    121;
    0;4;   -0.707107, 0.707107, 0.000000, 0.000000;;,
    1;4;   -0.697332, 0.697332, 0.015710, 0.015710;;,
    2;4;   -0.684805, 0.684805, 0.035442, 0.035442;;,
    ...
  }
  AnimationKey { 
    1; //Scale
    121;
    0;3;     1.000000, 1.000000, 1.000000;;,
    1;3;     1.000000, 1.000000, 1.000000;;,
    2;3;     1.000000, 1.000000, 1.000000;;,
    ...
  }
}

So, to apply frame 2, I would take the position, rotation and scale from frame 2, create a transformation matrix (call it Transform_A) from them and apply that matrix the vertices controlled by Armature_001_Bone at their weights. So I'd stuff TransformA into my shader and transform the vertex. Something like:

vertexPos = vertexPos * bones[ int(bfs_BoneIndices.x) ] * bfs_BoneWeights.x;

Where bfs_BoneIndices and bfs_BoneWeights are values specific to the current vertex.

When loading in the mesh vertices, I transform them by the rootTransform and the meshTransform. This ensures they're oriented and scaled correctly for viewing the bind pose.

The problem is when I create that transformation matrix (using the position, rotation and scale from the animation), it doesn't properly transform the vertex. There's likely more to it than just using the animation data. I also tried applying the bone transform hierarchies, still no dice. Basically I end up with some twisted models. It should also be noted that I'm working in openGL, so any matrix transposes that might need to be applied should be considered.

What data do I need and how do I combine it for applying .x animations to models?

I've made some mock ups of what this looks like, in case that's useful.

First I wanted to just test the translation, this is a bobbing head, what it looks like in Blender:

https://i.stack.imgur.com/NAc4B.gif

And what it looks like in game (don't mind the colors):

https://i.stack.imgur.com/nj2du.gif

Then for just rotation, the animation was the head rotating 360 degrees around the vertical axis. This is what that looks like in game:

https://i.stack.imgur.com/gVyUW.gif

Note, there should be no tilt, only rotation like a merry-go-round.

Update

I have the translation part of the animation working. But it feels kind of hacky and I don't see how to apply it to the rotation.

The translation works by taking these steps:

1. Take the position from the animation frame and swap the y and z values
2. Translate the transformation matrix by the altered position
3. Transpose the transformation matrix
4. Apply the transformation matrix to the vertices

So that's how it can work, but how is it supposed to work generally for position, scale and rotation?

Answer 1

It should also be noted that I'm working in openGL, so any matrix transposes that might need to be applied should be considered.

Maybe that's the key, have you tried negating some Z-coordinates around to convert from DirectX left-handed coordinates to OpenGL right-handed ones?

You could negate all Zs before applying your transformations, and re-negate when done, see if the output gets any better.

EDIT

Swapping the Y and Z-coordinates is another way to change handedness. So you do have to adapt your transformations, check this paper for details.

The fact that you have to transpose you matrix also means you going from row-major to column-major or the other way around. This should be less complicated to handle, you'll probably just have to find the right "place" to do it in your pipeline. I'm just guessing, but doing it on quaternions could simply means changing them to matrices, transpose, and go back quaternions, if your Math lib offers your this option.

Answer 2

OK, I put some more time toward this and got it working for multiple bones, and considerably less hack-like (though still a little). The system I had in place originally was almost correct. However, the main problems were:

1. Blender saves Quaternions in W, X, Y, Z format. I was loading them X, Y, Z, W.

2. My code for converting a Quaternion to a matrix was incorrect. I'm actually still not sure why this is incorrect, but I know the following code works, as part of my Quaternion class:

   public Matrix4f toMatrix() {
       Matrix4f tmpMatrix = new Matrix4f();
       if (this.length() > 0)
           this.normalise();
   
       float xx = this.x * this.x;
       float xy = this.x * this.y;
       float xz = this.x * this.z;
       float wx = this.x * this.w;
   
       float yy = this.y * this.y;
       float yz = this.y * this.z;
       float wy = this.y * this.w;
   
       float zz = this.z * this.z;
       float wz = this.z * this.w;
       //note the "* -1"s
       //I needed those to make the same matrices Blender was expecting
       tmpMatrix.m00 = (1.0f - 2.0f * (yy + zz)) * -1;
       tmpMatrix.m01 = (       2.0f * (xy - wz)) * -1;
       tmpMatrix.m02 =        2.0f * (xz + wy);
       tmpMatrix.m03 = 0;
   
       tmpMatrix.m10 =        2.0f * (xy + wz);
       tmpMatrix.m11 = 1.0f - 2.0f * (xx + zz);
       tmpMatrix.m12 = (       2.0f * (yz - wx)) * -1;
       tmpMatrix.m13 = 0;
   
       tmpMatrix.m20 =        2.0f * (xz - wy);
       tmpMatrix.m21 =        2.0f * (yz + wx);
       tmpMatrix.m22 = (1.0f - 2.0f * (xx + yy)) * -1;
       tmpMatrix.m23 = 0;
       return tmpMatrix;
   }
   

Other than that, it's just:

finalMatrix = boneMatrix * skinMatrix * invertedBindMatrix;

Note that the boneMatrix should be using the animated matrices for each bone instead of the bind matrices. It should also get all it's parent matrices like so:

private Matrix4f getBoneMatrix() {
        if (parent == null) {
            return Matrix4f.mul(Matrix4f.mul(parentArmature.getBindMatrix(), parentArmature.getArmatureRootMatrix(), null), boneMatrix, null);
        } else {
            return Matrix4f.mul(parent.getBoneMatrix(), boneMatrix, null);
        }
    }

And it works. Leave a comment if you need clarification.