I've kinda solved this myself by adapting the Camera::setDirection code from OGRE to work for my need:
Now I have the following code:
Vector3 boneDir = getBoneWorldOrientation(mEntity, mHips) * Vector3::UNIT_Z; //mDirection; Vector3 dir = mCameraTrans->_getDerivedDirection(); Vector3 zAdjustVec = dir; zAdjustVec.normalise(); Quaternion targetWorldOrientation; Vector3 xVec = Vector3::UNIT_Y.crossProduct(zAdjustVec); xVec.normalise(); Vector3 yVec = zAdjustVec.crossProduct(xVec); yVec.normalise(); targetWorldOrientation.FromAxes(xVec, yVec, zAdjustVec);
mTransform is basically the SceneNode of the character, I'm multiplying the target orientation with it's inverse orientation to get it to parent/character-space.
mHips->_setDerivedOrientation(mTransform->_getDerivedOrientation().Inverse() * targetWorldOrientation);
The above code was taken from OgreCamera.cpp, the setDirection function, the part with the fixed yaw axis.
Now, that works... Sorta, I now get this problem at near-90 degree pitch, the orientation kind of flips to the other side, I've made another video to showcase this:
Any ideas on how to fix this ?
I've asked on both the OGRE3D and GameDev.net forums, I'll link to there just in case.