You seem to be stacking matrix calculations. The "node original transform sources" should be kept pristine while calculating "node in the graph stuff".
Assuming your original code looks a bit like this:
bool haveParent;
Node* parent;
Mat4 rotation; // or Quat
Mat4& getRotation() const { return rotation; }
Vec3 position;
Vec3& getPosition() const { return position; }
Mat4 Node::getTransform() {
return getTranslation() * getRotation();
}
const Vec3&
void Node::update() {
Mat4 parentTrans = (this->haveParent ? this->parent->getTransform() : Mat4::Identity());
Mat4 thisTrans = this->getTransform();
Mat4 translate = Mat4::Translation(this->getPosition());
Mat4 rotate(this->getRotation());
Mat4 result = parentTrans * thisTrans * translate * rotate;
this->setPosition(result.getTranslation());
this->setRotation(result.getRotation());
}
You would probably need something like this:
// Assume the attributes are private and the methods are public.
Node* mParent; // nullptr on initialization
Mat4 mThisNodeRotation; // or Quat
Mat4& getThisNodeRotation() const { return mThisNodeRotation; }
Vec3 mThisNodePosition;
Vec3& getThisNodePosition() const { return mThisNodePosition; }
Mat4 mWorldRotation; // Rotation computed with the parent's ancestries
Mat4 mWorldPosition; // Translation computed with the parent's ancestries
const Mat4& getWorldRotation() const { return mWorldRotation; }
void setWorldRotation( const Mat4& aValue ) { mWorldRotation = aValue; }
const Vec3& getWorldPosition() const { return mWorldPosition; }
void setWorldPosition( const Vec3& aValue ) { mWorldPosition = aValue; }
const Mat4& getWorldTransform() const { return mWorldPosition * mWorldRotation; }
Mat4 Node::getThisNodeTransform() {
return getTranslation() * getThisNodeRotation();
}
void Node::update() {
Mat4 parentTrans = (this->parent ? this->parent->getWorldTransform() : Mat4::Identity());
Mat4 worldResult = parentTrans * getThisNodePosition() * getThisNodeRotation();
this->setWorldPosition(result.getTranslation());
this->setWorldRotation(result.getRotation());
}
I have not tested the code, but it's been done with nearly the best of my knowledge.
When computing the final position of a node, you really have to keep a wall between the node's data, and what's being applied to the scene graph: when computing the position in the scene graph, it must not change the node's original settings (of course, it's the position w.r.t. the parent, but this does not need to be modified during the update and the calculation of the final location).
My coding standards have been applied, please disregard as I'm becoming an old monkey.