How to calculate rotations for an object with scaling transformations?

Question

I am working on implementing scaling transformations into a game engine (C++), which previously assumed only translations and rotations. Visually, the scaling works, but a lot of the game logic started to break.

For example: I have a turret, which is a composite object (turret base --> weapon). I need to change the turret's yaw and pitch so it faces the target when it fires. To do this, I first get the target position in object space by inverting the object transformation matrices (taking into account that the gun may be offset from the turret base), and I use the resulting point to calculate the necessary yaw and pitch.

void Rotate_Turret(Turret t, Vec4 targetPos)
{
    // Get the TF matrices
    Mat4 turretBaseTf = Get_Tf(t.TurretBase);
    Mat4 turretGunTf = Get_Tf(t.TurretGun);

    // Calculate target in the turret base space (for yaw)
    // "inv" transposes the rotation part and inverts the translation
    Vec4 baseSpaceTarget = inv(turretBaseTf) * targetPos;

    // Calculate yaw
    float relativeYaw = atan2(baseSpaceTarget.Y, baseSpaceTarget.X);

    // Get the offset of the gun from the base
    Vec4 gunOffset = inv(turretBaseTf) * turretGunTf.Translation();

    // Get the XY plane component of the base space target vector
    float targetDistanceXYBase = sqrt((baseSpaceTarget.X * baseSpaceTarget.X) + (baseSpaceTarget.Y * baseSpaceTarget.Y));

    // Get the vector components for the gun, taking the offsets into account
    float targetDistanceZGun = baseSpaceTarget.Z - gunOffset.Z;
    float targetDistanceXYGun = targetDistanceXYBase - gunOffset.X;
    float relativeTilt = atan2(targetDistanceZGun, targetDistanceXYGun);

    // Set the rotations
    t.Rotate(relativeYaw, relativeTilt);
}

This works fine at 1:1 scale, but as soon as I upscale the turret at the root level, it keeps shooting consistently above the target. I tried inverting the scale in the matrices in the above process, but it doesn't seem to work. What am I doing wrong?

Thanks in advance for the help!

Answer 1

I double-checked every bit of the math by calculating the angle directly between the weapon and the target position, and I compared that to the results of the code described above. I'm pretty sure I'm getting the correct results, so I don't get why it doesn't tilt the weapon correctly (the Z angle is correct, i.e it turns exactly toward me, but doesn't tilt the gun low enough).

The only thing I can think of is that somehow the resulting tilt angle itself needs to change based on scale, since a scaled matrix will have another rotation matrix applied to it afterward (i.e when I apply the rotation to the turret weapon to make it aim at the target)