collisions: How to handle contact constraints for a rigidbody constrained to a line segment

I'm trying to come up with a strategy to handle a situation in 2D, where a rigidbody moves along a straight line segment, and can collide with other similar bodies and also static tiles.

A naive approach was to handle collisions by projecting the displacement vectors onto the line segment (worked ok for aabb), and then at the end snap the position of the rigidbody back to the line, which had lots of conflicts.

I'm trying to implement impulse based physics, so far only handling contacts, and unsure how to both constrain the rigidbody to a line segment and handle contacts at the same time. Or is it better to do in two separate phases. It's probably easier to allow some displacement from the line and then gradually move the body back to it. Any suggestions?

The solution I went with was to treat this like a contact constraint, if the center of the object went beyond either end of the line segment or if it was displaced laterally from the line then I just set a new contact with a normal pointing perpendicularly towards the line or pointing (anti-)parallel to the line. Then solve it together with the other contacts.

setDistanceToPath(platform: Platform, manifold: ContactManifold) {
let p = platform.rb.position;
let ap = Vec2.subtract(p, platform.path.start);
let d = platform.path.direction;
let ap_d = Vec2.dot(ap, d);

let right = Vec2.perRight(d);
let dp = Vec2.dot(right, ap);

const setLateralContact = (dp: number) => {
if (dp < 0) {
manifold.setContact(platform.rb.position, -dp, right);
}
else if (dp > 0) {
manifold.setContact(platform.rb.position, dp, Vec2.perLeft(d));
}
else {
manifold.clearContacts();
}
};

if (ap_d < 0) {
if (Math.abs(dp) > -ap_d) {
setLateralContact(dp);
}
else {
manifold.setContact(platform.rb.position, -ap_d, d);
}
}
else {
let bp = Vec2.subtract(p, platform.path.end);
let inv_d = Vec2.inverse(d);
let bp_d = Vec2.dot(bp, inv_d);

if (bp_d < 0) {
if (Math.abs(dp) > -bp_d) {
setLateralContact(dp);
}
else {
manifold.setContact(platform.rb.position, -bp_d, inv_d);
}
}
else {
setLateralContact(dp);
}
}
};