GPU skinning algorithm incorrectly distorts mesh

Question

I've been trying to implement a GPU-based matrix palette skinning algorithm with WebGL, but the rendering appears incorrect even though I can't find evident conceptual problems in the underlying algorithm. Left image is the rigged model in Blender, right image is the same mesh rendered with my algorithm.

I wrote a custom JSON exporter which exports the exact same data as the THREE.js exporter - position and rotation quaternions for bindposes and keyframes, weights and indices (checked manually). With THREE.js the animation renders correctly, so the issue must lie somewhere in my matrix manipulation process.

My process is the following:

1. Calculate the world matrices for each bone in the bindpose. This is obtained by first calculating localMatrix with mat4.fromRotationTranslation(localMatrix, bone.rot, bone.pos), then by copying localMatrix in the worldMatrix if the current bone is the root, or by multiplying localMatrix with the parent's worldMatrix if it is not; finally, the inverse bindpose matrix is stored. (* note that, per exporter invariant, each parent precedes all of its children in the array, so each parent's worldMatrix will have been calculated by the time it is requested; see this article).

     for(var i = 0; i < this.geometry.bones.length; i++) {
       var bone = this.geometry.bones[i], localMatrix = mat4.create();
   
       mat4.fromRotationTranslation(localMatrix, bone.rot, bone.pos);
   
       bone.worldMatrix = mat4.create();
       bone.inverseBindpose = mat4.create();
   
       if(bone.parent == -1) {
         mat4.copy(bone.worldMatrix, localMatrix);
       } else {
         // *
         mat4.multiply(bone.worldMatrix, this.geometry.bones[bone.parent].worldMatrix, localMatrix);
       }
       mat4.invert(bone.inverseBindpose, bone.worldMatrix);
     }
   

2. For each keyframe, recalculate the bone hierarchy with the same algorithm but with the degrees of freedom specified by the keyframe, then for each keyframe-bone calculate a matrix offsetting from the bindpose by multiplying its world matrix with the inverseBindpose calculated in the first step.

     var kf = this.geometry.keyframes;
     for(var i = 0; i < kf.length; i++) {
   
       var flat = [];
   
       for(var j = 0; j < kf[i].length; j++) {
         var bone = kf[i][j],
             parent = this.geometry.bones[j].parent,
             localMatrix = mat4.create();
   
         mat4.fromRotationTranslation(localMatrix, bone.rot, bone.pos);
   
         bone.worldMatrix = mat4.create();
   
         if(parent == -1) {
           mat4.copy(bone.worldMatrix, localMatrix);
         } else {
           mat4.multiply(bone.worldMatrix, kf[i][parent].worldMatrix, localMatrix);
         }
   
         var offsetMatrix = mat4.create();
         mat4.multiply(offsetMatrix, bone.worldMatrix, this.geometry.bones[j].inverseBindpose);
         bone.offsetMatrix = offsetMatrix;
   
         flat.push.apply(flat, offsetMatrix);
       }
       this.keyframes[i] = new Float32Array(flat);
   
     }
   

3. Plug everything in the buffers and invoke the vertex shader. I tried deforming a mesh in the same way on the CPU and the results are exactly the same, so I think this rules it out and the problem lies in the matrices, but here's the shader for the sake of completeness.

     uniform mat4 uP, uV, uM;
     uniform mat4 uBonesFrame[8];
     uniform mat3 uN;
     uniform bool uSkin;
   
     attribute vec3 aVertex, aNormal;
     attribute vec2 aTexCoord;
   
     attribute highp vec2 aSWeights;
     attribute highp vec2 aSIndices;
   
     varying vec3 vVertex, vNormal;
   
     mat4 boneTransform() {
   
       mat4 ret;
       float normfac = 1.0 / (aSWeights.x + aSWeights.y);
   
       ret = normfac * aSWeights.y * uBonesFrame[int(aSIndices.y)] +
             normfac * aSWeights.x * uBonesFrame[int(aSIndices.x)];
   
       return ret;
     }
   
     void main() {
   
       mat4 bt = uSkin ?
         boneTransform()
         :
         mat4(
             1., 0., 0., 0.,
             0., 1., 0., 0.,
             0., 0., 1., 0.,
             0., 0., 0., 1.
         );
   
       gl_Position = uP * uV * uM * bt * vec4(aVertex, 1.0);
       vVertex = (bt * vec4(aVertex, 1.0)).xyz;
       vNormal = (bt * vec4(aNormal, 0.0)).xyz;
     }
   

In pseudo code, it goes like this:

For each bone
  Calculate localMatrix from quaternion and translation
  If (is root bone) worldMatrix = localMatrix
  Else worldMatrix = parent's worldMatrix * localMatrix
  inverseBindpose = invert(worldMatrix)

For each keyframe
  For each bone
    Calculate localMatrixKF from quaternion and translation
    If (is root bone) worldMatrixKF = localMatrixKF
    Else worldMatrixKF = parent's worldMatrixKF * localMatrixKF
    offsetMatrixKF = worldMatrixKF * inverseBindpose
    Use offsetMatrixKF to deform the vertex

I've also tried deforming (0.0, 0.0, 0.0) with the bones' matrices to obtain the joint positions and I found (0.0, 0.0, 0.0), (-.87, .50, 0.00), (-.87, 2.50, 0.0) and (1.73, 4.00, 0.00) with the offset matrices and (0.0, 0.0, 0.0), (-.87, 1.00, 0.00), (-.87, 1.50, 0.0) and (1.73, 1.00, 0.00) with the world matrices at keyframe 2, where it bends past the XZ plane. The world coordinates look ok to me, I'm not really sure about the offset matrices, the 2.50 and 4.00 values look a little large to me.

Am I doing something incredibly wrong?

If necessary, I'll upload the full code somewhere.

Thank you in advance for your patience, I'm losing my mind over this.

---

Answer 1

I solved my own problem. I drew the wireframe of the bones and then I understood. I didn't apply the transformations in Blender so I was deforming a mesh bigger twice than it should have been. See the picture:

The armature is clearly smaller than the mesh, this is the reason why the deformation came out incorrect. By applying rotation, location and scale in Blender (Ctrl+A on the 3D view) the problem went away. The algorithm as I described in the question turns out to be fundamentally correct. I hope that it might be of use to someone at least! :)