why we can't just interpolate our z values to test if the point is behind or in front of the near plane?
Because a vertex that's behind the near plane might be behind the camera entirely. That means the triangle crosses through a singularity in our projection, and part of it will be projected to the opposite quadrant of the screen from where we expect to see it, and closer to the middle of the screen than it should be (as it recedes into depth behind us).
Even points that aren't quite behind the camera could get divided by an extremely small
w component when we do our perspective division, rocketing off to absurdly distant locations in screen space where we have very little numerical precision left, causing the triangle attached to them to swim and vibrate from small changes in the camera transform.
You may have noticed artifacts in very old 3D games where polygons would appear to jitter and distort or the textures slosh along them when they passed very close to the camera — that's this effect in action.
To see why we can't just switch to higher precision numbers to fix this, imagine a long skinny triangle laying on the ground, point toward you. The lines of the left and right edges converge toward the bottom of your field of view. Now move so that the point is just behind you, and lean down so your head is very close to the ground. From this perspective, the lines of the left and right edges of the triangle can appear parallel or even diverging toward the bottom of your field of view, because your perspective magnifies the space between them faster than the space between them is shrinking.
There is no place (no finite coordinates) where we can put that third triangle point in the screen space to achieve this effect. Our only choice to project this properly with triangle rasterization is to split it into more than one triangle whose points are at finite screen coordinates, and rasterize each one.
As for algorithms to perform this clipping, the basic idea is to check: for each edge of the triangle, are its two vertices on opposite sides of the near plane?
If no, for all edges, then you can draw or reject the triangle as a whole.
If so for at least one edge, we need to clip.
Generate a new vertex at the intersection of the near plane with each crossing edge, interpolating its vertex attributes accordingly.
After repeating this for each edge, you should have 3-4 (not necessarily distinct) points on or in front of the near plane, including at least one of the triangle's original points.
If you have 3, that's your new (possibly degenerate) triangle to rasterize.
If you have 4 distinct points, then you have a quadrilateral, and you can split it along one diagonal or the other to rasterize it as two triangles.
For the exact implementation, I'll defer to others who have written this up in far more detail. As a consumer of the graphics pipeline, rather than a GPU engineer myself, I've mercifully never needed to implement clipping by hand. 😉 I can just let the GPU handle that for me automatically, handling numerical precision issues and edge cases I've never even thought of.