An exchange of two axes is a mirror operation. So you'll need to choose an axis on which to reflect the object using its scale triplet, then change the rotation to compensate.
Let's say we mirror the object on the x axis: newScale = (-1, 1, 1) * scale
. Now a unit vector that had been pointing toward +x points toward -x, and one that had been pointing toward +z or +y still point to those directions.
We next want to rotate this object so the formerly +x, now -x vector points to +x, the +z vector points to +x, and the +y vector points to +z. (to get the expected outcome of our x/z swap). That's a 180 degree rotation about the line y = z, or as a quaternion: newRot = (0, 1.0f/sqrt(2), 1.0f/sqrt(2), 0) * rot
.
(Here using XYZW component order according to the glTF spec)
But we could just as easily choose to treat this as a mirror in the y axis instead. Then we'd get: newScale = (1, -1, 1) * scale
, and unit vectors in the +x and +z directions would stay put, while a unit vector in the +y direction gets mapped to -y. To turn these around, we'd want a +90 degree rotation around the -x axis: newRot = (-1.0f/sqrt(2), 0, 0, 1.0f/sqrt(2)) * rot
.
Or, to round things out, we could treat it as a mirror in the z axis: newScale = (1, 1, -1) * scale
. +x and +y are preserved, and +z maps to -z. So then we'd want to rotate +90 degrees around the +x axis to fix this up: newRot = (1.0f/sqrt(2), 0, 0, 1.0f/sqrt(2)) * rot
.
Of course, the trouble here is that mirroring your object flips the winding of all your triangles. So you may need to detect if your new transformation includes an odd number of sign flips or axis exchanges (negative determinant when expressed as a matrix), and if so, swizzle your indices as you load them to get back to the desired winding.
In general, if you can decompose your target transformation into a translation t
, rotation r
, and scale s
, then you can apply...
newScale = scale * (inverse(rot) * s)
newRot = r * rot;
newTrans = trans + t;
But as noted in the comments, not every affine transformation can be represented this way. Those that include non-uniform scaling along axes diagonal to the root node's coordinate axes, or shearing transformations, can't be exactly represented as a sequence of axis -aligned scale, rotation, and translation (in that order).