I've implemented simple 3D game physics engine. I already have decent collision detection in place, now I'm trying to figure out the collision response part. I'm using impulse based method to calculate the post-collision velocities. This works pretty well, however, it does not completely prevent the bodies to keep interpenetrating. So I have additional piece of code to resolve the penetration. Currently, I just move the bodies along the contact normal by half the penetration depth - first body in the direction of the contact normal, second body in the opposite direction.
This is ok most of the time, but there are some undesirable effects. For example, imagine a narrow corridor and an object moving through it. If the object hits one of the walls of the corridor, the penetration resolution moves it into the opposite wall, then in the next frame back into the first wall and so on. The effect is the object is kind of vibrating really fast between the walls which is not pretty.
So my question is whether there is a better way to resolve the penetration? Perhaps not move the bodies, just somehow adjust their velocities (in addition to the impulse computation) so they stop moving toward each other and the penetration resolves itself in the next couple of frames. I'm just guessing here. Any ideas?