# Implementing skeletal animation and GPU skinning (DOOM MD5)

DISCLAIMER
This question has been completely rewritten to narrow the scope of the question in light of previous suggestions and answers, but the same problem still persists.

Currently I'm trying to implement skeletal animation (GPU skinning) in my project.

So far I have achieved single joint translation and rotation, and multi-jointed translation. The problem arises when I try to rotate a multi-jointed skeleton.

At the moment I'm following quite a few sources in trying to implement skeletal animation. The MD5 spec, a blog post with source code and I've also seen many other tutorials.

Currently when I try and load the popular "Bob Lamp" model, I get this result. • My maths is ordered with "vector to the left"
• The vectors and rotations of animation frames are being converted into matrices
• I pre-multiply the inverse bind pose and pose matrices

#version 330 core

smooth out vec2 vVaryingTexCoords;
smooth out vec3 vVaryingNormals;
smooth out vec4 vWeightColor;

uniform mat4 MV;
uniform mat4 MVP;
uniform mat4 Pallete;

layout(location = 0) in vec3 vPos;
layout(location = 1) in vec2 vTexCoords;
layout(location = 2) in vec3 vNormals;
layout(location = 3) in int vSkeleton;
layout(location = 4) in vec3 vWeight;

void main()
{
vec4 wpos2 = vec4(0.0);
vec4 norm2 = vec4(0.0);
vec4 wpos = vec4(vPos, 1.0);
vec4 norm = vec4(vNormals, 0.0);
vec4 weight = vec4(vWeight, (1.0f-(vWeight + vWeight + vWeight)));

mat4 BoneTransform = mat4(0.0);
for(int i = 0; i < 4; i++) {
float fW = weight[i];

if(fW > 0) {
int jI = vSkeleton[i];
vec4 tmpPos = wpos * (Pallete[jI]);
wpos2 += fW * tmpPos;
}
}

wpos2 /= wpos2.w;

vWeightColor = weight;
vVaryingTexCoords = vTexCoords;
vVaryingNormals = normalize(vec3(vec4(vNormals, 0.0) * MV));
gl_Position = wpos2 * MVP;
}


Another peculiar thing to mention is that multi-jointed animation works when the vertices are inline with the bones. As soon as the vertices move away from the bones, they deform incorrectly.

UPDATE
In response to @MickLH

My shader code now has wpos2 /= wpos2.w after the for loop. This is the result. • I've had similar problems when implementing a .ms3d loader. Try simplifying things as much as you can, for example you could remove vertex weights from the equation, also I would be experimenting with a 3D object as it might give you a better understanding of whats going wrong. Also I believe your line BoneTransform += ... should be BoneTransform *= .... – akaltar Oct 18 '13 at 19:15
• I checked my old source codes, and found out that my recommendation is still not good and it should be written as BoneTransform = (...) * BoneTransform; – akaltar Oct 18 '13 at 19:28
• @akaltar I've already tried simplifying things, the current code now can do single bone rotation and translation plus multijointed translation, I've just been stuck on getting the multijointed rotation correctly. I'll try the code you suggested. – Soapy Oct 18 '13 at 20:25
• @akaltar could you possibly show me your're code please? – Soapy Oct 18 '13 at 20:44
• Please do not massively re-write your question after answer have been posted, it invalidates the posted answers. It's better to post a new question in that case (this is a Q&A site, not a discussion forum). – Josh Nov 26 '13 at 17:05

Your layout is illegal, vSkeleton needs 4 indices since it is a 4-part array.

layout(location = 3) in int vSkeleton;
layout(location = 4) in vec3 vWeight;


should be:

layout(location = 3) in int vSkeleton;
layout(location = 7) in vec3 vWeight;


or better yet you should use glGetUniformLocation and let the GLSL compiler assign them for your:

in int vSkeleton;
in vec3 vWeight;


I'm not sure if this will solve your problem, but at least I can point some lines in your code, that I don't belive are correct.

First of all, you need to decide how you're going to multiply your matrices and vectors, i.e. are you going to use the left-side or right-side convention. In your shader code you have this line:

wpos = BoneTransform * wpos;


which suggests "vectors on the right", but then you have this:

gl_Position = wpos * MVP;


so it simply does not match. So, you either need to change the first on or the second one. But does it matter which one? Since you're calling your matrix MVP I will assume that this is equal to

ModelViewMatrix * ProjectionMatrix


and in that case you should probably use the "vectors on the left" convention. It's not very common but that's totally fine to me. So it may look like the only thing you need to fix is wpos = wpos * BoneTransform. Indeed, the formula

BoneTransform += (invBindPose[vSkeleton[i]] * Pallete[vSkeleton[i]]) * weight[i];


is good enough for left-side vector multiplication, at least if the Pallete matrices are computed the right way, but are they?

In fact, this is determined by the MD5mech file format itself. I've look throught their documentation but I'm not entirely sure which convention they're actually using. BTW, because of this ambiguity, it's always better to use 4x4 matrices than quaternion + vector pair. In your situation this is a matter of checking wheter you should use pqi instead of pq in the formula

Math3::quat::RotateVector3(rpos, pq, jv);


or not. Also another line

Math3::quat nq = pq * jq;


could be nq = jq * pq as well, so in fact you have at most 4 cases to check. Probably the easiest way to go is to expermient, but I would encourage you to look in the source code of the Math3 library to determine if their RotateVector3 routine behaves as described in the MD5mesh docs.

Also have in mind that there are at least two conventions to store quaternions in memory, which differ in the order of quaternion's coordinates. The natural one is w,x,y,z but I've observed that some engines/libraries tends to place the w coordinate at the end of the sequence. That is usually fine, unless you're forced to convert from vec4 to quaternion or vice-versa. From my experiance it can be a real pain to look for errors of this type, so be carefull with that.

@akaltar Please note, that *= is used for composing transformations. Here we only want to compute a baricentric combination of some affine transformations, so in the formula for BoneTransform the use of += is totally fine.

• I have updated my question to respond to your points. – Soapy Oct 22 '13 at 11:35
• Please try using (in shader code) the transposition of the matrices returned by GetMatrix routine, while keeping the order of multiplication wpos = wpos * BoneTransform. – Tomasz Lenarcik Oct 22 '13 at 13:46
• Okay so trying this I changed BoneTransform += (invBindPose[vSkeleton[i]] * Pallete[vSkeleton[i]]) * weight[i]; to BoneTransform += (transpose(invBindPose[vSkeleton[i]]) * transpose(Pallete[vSkeleton[i]])) * weight[i];. The result is a deformation which looks like the central image in the first set of images I posted in this question. – Soapy Oct 22 '13 at 16:50
• @Soapy I saw that you're using Parent.Pos and Parent.Translation in your code. What's the difference between these tow guys? – Tomasz Lenarcik Oct 23 '13 at 11:31
• There is none, they both the same thing. Sorry for the confusion. – Soapy Oct 23 '13 at 11:51

1. normalize(weight); is wrong as it does not make the components add up to one, it makes the length of the vector one which is not what you want. But as you do not use it's return value this line doesn't do anything. This line is probably not required from what I understand of your setup.

2. BoneTransform += makes no sense. Adding matrixes together does not concatenate the transforms, you need to multiply them. But this isn't correct for another reason:

You may not apply bone weights to the transformations, you have to apply them to the transformed vertex:

for(int i = 0; i < 4; i++) {
if(vSkeleton[i] != -1) {
if(i == 0) {
BoneTransform = ((invBindPose[vSkeleton[i]] * Pallete[vSkeleton[i]]) * weight[i]);
// BoneTransform = ((Pallete[vSkeleton[i]] * invBindPose[vSkeleton[i]]) * weight[i]);
} else {
BoneTransform += ((invBindPose[vSkeleton[i]] * Pallete[vSkeleton[i]]) * weight[i]);
// BoneTransform += ((Pallete[vSkeleton[i]] * invBindPose[vSkeleton[i]]) * weight[i]);
}
}
}

wpos = BoneTransform * wpos;


Has to look more like this:

wpos = 0.0;
for(int i = 0; i < 4; i++)
{
if(vSkeleton[i] != -1)
{
BoneTransform = invBindPose[vSkeleton[i]] * Pallete[vSkeleton[i]];
wpos += ( BoneTransform * vec4(vPos, 1.0) ) * weight[i];
}
}


This is one reason why hlsl's mul function is better thant glsl's * operator. You may not switch around the operands when doing (matrix * vector) * scalar.

I'm not sure if I caught all the problems. There could still be problems on the cpu side and how your pass parameters. Let me know how it goes

3. PERFORMANCE: You should remove the binding pose from the bone transformation on the cpu. This has to be done for each vertex on the gpu and only once for each bone when done on the cpu.

• I have updated my question to respond to your points. – Soapy Oct 22 '13 at 11:38
• I'm pretty sure += is wrong here as it is applied to the transformation not to the transformed vertex as I did it in my proposal. The transformations itself can not simply get scaled and summed together, think about what happens when scaling a rotation matrix. Summing the weighted resulting vertices using += is a barycentric combination that results in the final vertex. – Archy Oct 22 '13 at 11:57
• I suggest you write a little test where you transform the bone positions and rotations on the cpu and render helper objects to figure out where the problem lies. – Archy Oct 22 '13 at 12:01