I'm building a custom engine and I have a confusing asset pipeline. I have skeletal animations in Blender and voxel mesh data in Qubicle. My engine loads the skeletal data and applies joint transformations to the voxel mesh vertex data. I'm doing this flow to preserve voxel data for use in the engine.
I'm struggling with trying to align the skeletal space to the mesh space. There are a few reasons for this. Blender is centered on the 0,0,0 axis. Quibicle uses a discrete voxel grid for alignment and you can't align to a center origin. You have to snap to the grid.
You can see that Blender uses a right handed coordinate system with Z axis being the vertical axis. The skeleton has a center origin on the Z axis. Additionally, the skeleton is facing the opposite 'front' direction.
Quibicle has Y as the vertical axis and the mesh facing positively on the Z axis. You can see that the quibicle mesh is offset from the 0,0,0 origin point because it is snapped to the voxel grid.
I'm working with WebGL, writing raw glsl, and combining the transformations in a shader like:
vec4 worldSpacePosition = model * //this is actually local * parent entity transformation matrices joints[jointId] * //Blender space combination of joint matrix * inverse bindPose vec4(position.xyz, 1); // Vertex in quibicle space gl_Position = projection * view * worldSpacePosition;
This gives me spaghetti animations:
You can see in the gif that the mesh is offset from the local axis helper like is in Qubicle. The Blender transformations are then action on on the 0,0,0 local origin where the mesh is not.
I have tried many different tweaks to the above shader, specifically alterations to
joints[jointId] matrix. I assumed that I could just add an offset to the Blender joint matrix or inverse bindPose matrix to make it match the Qubicle offset.
The only thing I've found so far that works is if I add another transformation, a "centerOffset" before the joint matrix. I can get the verts to align up with the 'blender space' and make the animation transforms accurate.
vec4 worldSpacePosition = model * //this is actually local * parent entity transformation matrices joints[jointId] * //Blender space combination of joint matrix * inverse bindPose centerOffset * // Center offset calculated from the AABB of the voxel mesh vec4(position.xyz, 1); // Vertex in Quibicle space gl_Position = projection * view * worldSpacePosition;
centerOffset is a mat4 translation that shifts the mesh by half its X and Z which will center it at local 0,0,0
const centerOffset = mat4.fromTranslation( [ -Math.floor((mesh.localAABB / 2)), 0, //don't recenter on the Y axis -Math.floor((mesh.localAABB / 2)) ] );
The glsl matrices deserve some further explanation.
My engine has an entity component system with child-parent transformation.
model is a scale * rotation * translation matrix and it is the local * parent transformation.
My animation has 1-1 bone-to-mesh relationship. Vert attributes have a
joints is an uniform
The joint transformation comes from blender it is actually created from some pre processing sub transformations.
const zReflection = [ -1, 0, 0, 0, 0, 1, 0, 0, 0, 0, -1, 0, 0, 0, 0, 1 ] let joint = mat4.multiply(blenderInverseBindePose, blenderJointMatrix) joint = mat4.multiply(joint, zReflection); joint = mat4.multiply(zReflection, joint);
I'll be honest this
zReflection is pretty gnarly and I'm not 100% sure how it works.
zReflection was my attempt to do a 'reflect' to get the animation to face the same direction as the mesh. Would appreciate a spot check here.
Though my "
centerOffset" fix works I would like to keep the fixes attached to the animation data in Blender space. This means offsetting the the blender skeleton data, the inverse bindpose, or joint matrices. I want apply it to the skeleton because the skeleton will always be relative to the meshes they control. When I'm offsetting the mesh data with
centerOffset this gets messy when you have parent child relationships and the mesh is offset with a local transformation from its parents.
My real question is how to adjust the joint matrix and inverse bindpose matrices from Blender space so that they align with the vert data from Quibicle space?