Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

I'm trying to write code to transfer animations that were designed for one skeleton to look correct on another skeleton. The source animations consist only of rotations except for translations on the root (they're the mocap animations from the CMU motion capture database). Many 3D applications (eg Maya) have this facility built-in, but I'm trying to write a (very simple) version of it for my game.

I've done some work on bone mapping, and because the skeletons are hierarchically similar (bipeds), I can do 1:1 bone mapping for everything but the spine (can work on that later). The problem, however, is that the base skeleton/bind poses are different, and the bones are different scales (shorter/longer), so if I just copy the rotation straight over it looks very strange:

http://www.youtube.com/watch?v=H6Qq37TM4Pg

The model there is from Titan Quest, but I will eventually be retargeting it to my own skeletons. I've locked everything but the arms in place to show the problem (the legs look even weirder). As you can see, the left arm is doing OK, but the right arm is sort of inverted. (BTW, the shadows don't actually look that bad, blame CamStudio/Microsoft Video 1).

I've attempted a number of things similar to lorancou's solution below to no avail (ie multiplying each frame in the animation by a bone-specific multiplier). If anyone has any resources on stuff like this (papers, source code, etc), that would be really helpful.

share|improve this question
    
How do you expect me to ignore the tail and the thing between the legs? :P –  kaoD Apr 9 '12 at 13:47
2  
@kaoD If you have to ask, the skeleton is rooted at (0,0) so there's a fake bone there. As for the tail... everyone knows life is better if you have a tail. I've always thought it would be efficient for things like carrying coffee cups and balancing on tree limbs. –  OverMachoGrande Apr 9 '12 at 23:04
    
I have seen a real time demo of this where a kinect was used to animate a model displayed in xna. Think the code was on an open source site. Will search... –  George Duckett Apr 11 '12 at 19:29
1  
    
I suspect that your problem is more with the distinct bind poses than with the bones scaling, you could try isolating that. For instance start from the original skeleton, scale a few bones on it to create a new skeleton, and see if your algorithm breaks with this one. If it doesn't, then restart from the original skeleton but this time don't scale bones, just rotate them and see if your algorithm breaks. If it does, then yeah there's probably an additional transform to perform somewhere. –  Laurent Couvidou Apr 14 '12 at 12:51

3 Answers 3

up vote 5 down vote accepted

The problem was one of numerical stability. Approx 30 hours of work on this over the course of 2 months, only to figure out I was doing it right from the very start. When I ortho-normalized the rotation matrices before plugging them into the retarget code, the simple solution of multiplying source * inverse(target) worked perfectly. Of course, there's a lot more to retargeting than that (in particular, taking into account the different shapes of the skeleton, ie shoulder width, etc). Here's the code I'm using for the simple, naieve approach, if anyone is curious:

    public static SkeletalAnimation retarget(SkeletalAnimation animation, Skeleton target, string boneMapFilePath)
    {
        if(animation == null) throw new ArgumentNullException("animation");
        if(target == null) throw new ArgumentNullException("target");

        Skeleton source = animation.skeleton;
        if(source == target) return animation;

        int nSourceBones = source.count;
        int nTargetBones = target.count;
        int nFrames = animation.nFrames; 
        AnimationData[] sourceData = animation.data;
        Matrix[] sourceTransforms = new Matrix[nSourceBones];
        Matrix[] targetTransforms = new Matrix[nTargetBones];
        AnimationData[] temp = new AnimationData[nSourceBones];
        AnimationData[] targetData = new AnimationData[nTargetBones * nFrames];

        // Get a map where map[iTargetBone] = iSourceBone or -1 if no such bone
        int[] map = parseBoneMap(source, target, boneMapFilePath);

        for(int iFrame = 0; iFrame < nFrames; iFrame++)
        {
            int sourceBase = iFrame * nSourceBones;
            int targetBase = iFrame * nTargetBones;

            // Copy the root translation and rotation directly over
            AnimationData rootData = targetData[targetBase] = sourceData[sourceBase];

            // Get the source pose for this frame
            Array.Copy(sourceData, sourceBase, temp, 0, nSourceBones);
            source.getAbsoluteTransforms(temp, sourceTransforms);

            // Rotate target bones to face that direction
            Matrix m;
            AnimationData.toMatrix(ref rootData, out m);
            Matrix.Multiply(ref m, ref target.relatives[0], out targetTransforms[0]);
            for(int iTargetBone = 1; iTargetBone < nTargetBones; iTargetBone++)
            {
                int targetIndex = targetBase + iTargetBone;
                int iTargetParent = target.hierarchy[iTargetBone];
                int iSourceBone = map[iTargetBone];
                if(iSourceBone <= 0)
                {
                    targetData[targetIndex].rotation = Quaternion.Identity;
                    Matrix.Multiply(ref target.relatives[iTargetBone], ref targetTransforms[iTargetParent], out targetTransforms[iTargetBone]);
                }
                else
                {
                    Matrix currentTransform, inverseCurrent, sourceTransform, final, m2;
                    Quaternion rot;

                    // Get the "current" transformation (transform that would be applied if rot is Quaternion.Identity)
                    Matrix.Multiply(ref target.relatives[iTargetBone], ref targetTransforms[iTargetParent], out currentTransform);
                    Math2.orthoNormalize(ref currentTransform);
                    Matrix.Invert(ref currentTransform, out inverseCurrent);
                    Math2.orthoNormalize(ref inverseCurrent);

                    // Get the final rotation
                    Math2.orthoNormalize(ref sourceTransforms[iSourceBone], out sourceTransform);
                    Matrix.Multiply(ref sourceTransform, ref inverseCurrent, out final);
                    Math2.orthoNormalize(ref final);
                    Quaternion.RotationMatrix(ref final, out rot);

                    // Calculate this bone's absolute position to use as next bone's parent
                    targetData[targetIndex].rotation = rot;
                    Matrix.RotationQuaternion(ref rot, out m);
                    Matrix.Multiply(ref m, ref target.relatives[iTargetBone], out m2);
                    Matrix.Multiply(ref m2, ref targetTransforms[iTargetParent], out targetTransforms[iTargetBone]);
                }
            }
        }

        return new SkeletalAnimation(target, targetData, animation.fps, nFrames);
    }
share|improve this answer

I believe that your easiest option is simply to match the original bind pose with your new skeleton if you've got the possibility (if your new skeleton isn't skinned yet).

If you can't do that, here's something you could try. This is just intuition, I'm probably overlooking lots of things, but it might help you to find the light. For each bone:

  • In your "old" bind pose, you've got one quaternion that describes the relative rotation of this bone compared to its parent bone. Here's a hint for how to find it. Let's call it q_old.

  • Ibid. for your "new" bind pose, let's call it q_new.

  • You can find the relative rotation from the "new" bind pose to the "old" bin pose, as described here. That's q_new_to_old = inverse(q_new) * q_old.

  • Then in one animation key, you've got your one quaternion that transforms that bone from the "old" bind pose to an animated pose. Let's call this one q_anim.

Instead of using q_anim directly, try using q_new_to_old * q_anim. This should "cancel" the orientation differences between the bind poses, before applying the animation.

It might do the trick.

EDIT

Your code above seems to follow the logic I'm describing here, but something's inverted. Instead of doing this:

multipliers[iSourceBone] = Quaternion.Invert(sourceBoneRot) * targetBoneRot;

You could try that:

multipliers[iSourceBone] = Quaternion.Invert(targetBoneRot) * sourceBoneRot;

I think that you need to transform from your target to your source before applying the source animation, to get the same final orientation.

share|improve this answer
    
The bind poses of both the sources and the targets are going to vary, which is why I'm implementing this :-) . Indeed, multiplying by the inverse of the target rotation was the first thing I tried. I tried recalculating the bone rotations, as per your suggestion, but the result was the same. Here's a video of what's going wrong: youtube.com/watch?v=H6Qq37TM4Pg –  OverMachoGrande Apr 16 '12 at 5:09
    
Are you sure you're always expressing your rotations relatively to a parent bone? Seeing your video, it looks like you're using an absolute/world rotation somewhere, where you should use a relative-to-parent rotation instead. –  Laurent Couvidou Apr 16 '12 at 15:57
    
Yes, I'm fairly sure I'm using the relative transformations here (I've tried with absolutes, it looks much stranger).I updated the OP with the code I was using for this video. Rather than trying to debug it this way, I'd rather see some source code or tutorials where it was done successfully, then I can figure out what I'm doing wrong. –  OverMachoGrande Apr 16 '12 at 16:31
    
Sure, but maybe there's no tutorial to do exactly that :) I think you inverted something in your code above, I'll edit my answer. –  Laurent Couvidou Apr 16 '12 at 21:29
    
I've tried many ways, and none of the worked. I'm going to try actually calculating the global rotations on a per-frame basis to see what's going wrong. Thanks for your help, though; I'll give you the 100 rep. –  OverMachoGrande Apr 18 '12 at 23:37

See this link where they take a Kinect skeleton and map it to custom models to display in XNA. It's open-source. See also this part of the documentation in particular regarding skeleton mapping.

share|improve this answer
    
I found that via Google... it looks good, but the interesting part (the BoneMapper itself) is part of their proprietary closed-source library. The project just shows you how to use their API to do it. –  OverMachoGrande Apr 12 '12 at 22:24
    
Ahh, sorry. Didn't check in that much detail. –  George Duckett Apr 12 '12 at 23:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.