Why would you have to re-calculate it? The moment of inertia is a calculate-once feature of the object and persistent as long as you do not modify the object. Adding a force (or impulse) does not affect it's moment of inertia. Even if it rotates around "whatever" axis (of whatever reason), the object's moment of inertia is still always the same, there will just be additional forces in the equation.
Consider a car wheel that isn't attached at it's centerpoint P0 but at a point P1 at it's periphery. And never mind the inconvenient bumpyness when you drive :-). Spinning up that wheel from that silly attachpoint P1 requires a torque that consists of two parts:
- Rotating a mass around P1, the mass is seen (from P1:s point of view) as a single m(ass) located at P0.
- Spinning up the wheel. It wheel also rotates. Proof: Consider the attach point is just 1 mm off. It spins around the well placed wheel axis. Then consider it's 10 mm off - spins. The consider it's at the periphery - spins. No matter where P1 is, even if it's 1 km away, the wheel spins. So the 2nd thing to do is spinning up the wheel, and here the moment of inertia is what you work against.
From the wheels' point of view, it has no idea what it rotates around. Those mini-guys that live on the wheel only see their home rotating (eg. by using a compass) and of course there are some odd, inconvenient extra stuff, the centrifugal force around P1, but that has nothing to do with the wheel's moment of inertia.
A torque is always independent of attach point, or "view" point. The only thing that matters is the direction of the rotational axis. It doesn't matter (from the objects point of view) where the axis hits the object, that only introduces calculation for an external system, if the object is part of one such.
Another thing to know: If you lift up your car wheel and (try to) untighten a bolt, the wheel rotates (women do that :-)) and there is a torque "in the wheel". Even if you move the bolt 1 km from the wheel center, and rotate it, the torque will be the same. Or, if the move the wheel axis 1 km away - doesn't matter provided that both you and the wheel axis are free to move. If you are not, then you are an external system and the wheel becomes part of that.
If you only look at the object but nothing external, it's moment of inertia is persistent and thus a calc-once feature. Your pre-calc will do for the duration of the object un-modofied to it's shape or mass.