I use the following approach (similar to the mass splitting algorithm of Tonge http://www.richardtonge.com/):
- detect all colliding pairs in your scene/context. Let (A,B) be such a pair. Apply a ghost/mass splitting idea: if A is in contact with M bodies and B is in contact with N other bodies, then temporarily set the mass of A to
m_A/M and that of B to m_B/N
- compute reaction/restitution force contributions for each pair (A,B) and store these contributions in A and B's own accumulators
- compute restitution velocities from impulses (as you stated) and store them in the same fashion (as deltaV velocity residues in their own accumulators for each (A,B) pair)
- compute penalty displacements (again, accumulate displacements, do not apply them instantly!)
- reset the masses of all bodies previously designated as parties in collision pairs (
m_A = m_A * M and m_B = m_B * N)
This approach is similar to how the Jacobi iterative algorithm works with linear simultaneous systems of equations. And it's not guaranteed to converge, but in my simulator it does the job quite smoothly.. in 3D (yes, an extra dimension adds twice the difficulty!).
Caveat: correct positions and velocities only after your collision detection/handling phase is over! That way you simultaneously update your colliding actors. Also, the restitution forces must be taken into account next time when you integrate for positions and velocities.
EDIT: Well, I guess you're using the already abused Verlet integration method (this one's become a household name within gamedev enthusiasts). In this specter of collision handling and integration, you might want to take a look here.
UPDATE: Some of the information on how to approach collision (and self collision for that matter of fact) can be found in these papers:
The approach I proposed is not by a long shot an original contribution, many games use it with plausible results and it was best employed by Jakobsen in his Hitman game engine.
From a somewhat practical experience, penalty forces (similar to linear or exponential springs getting their input from the penetration distance) do not properly solve penetrations when other forces from the bodies that collide manage to be greater than them. That's why I chose to combine three (almost redundant) approaches: Newtonian reaction forces (you push the wall, the wall pushes back), impulse derived velocities (snooker balls colliding) and a non-natural "move the bodies away from each other geometrically" solution. Together they seem to provide everything: get rid of most ugly interpenetration artifacts, colliding bodies tend to interact with each other on the long run (due to restitution velocities and forces - at least the forces that tended to drag the bodies in a collision scenario are cancelled out and the bodies bounce away from each other). Lastly, for further understanding of these simple but common concepts, I suggest analysing these slides.
My "abused method" epithet describing the Verlet integration steps is targeted at a popular culture belief that this is the Holy Grail of integration methods. It is just marginally better than its Symplectic Euler (also called by some semi-implicit Euler) cousin. Way more complicated integration methods exist (and all bear the implicit name in them). Powerful game engines make use of them, but indie developers don't have the time to experiment with those since Verlet, when tuned to a specific scenario, really does wonders. Also, there is absolutely no integration method that can deal with stiff constraints without a little cheating being involved (can't find the link, but the paper I'm referring to should be called "X.Provot - "Deformation Constraints in a Mass-spring Model to Describe Rigid Cloth Behaviour" ".
