For the most part, depth sorting does not occur at the triangle level. For opaque geometry - which is the majority of geometry drawn in a typical 3D application - depth sorting is usually done per pixel, using a depth buffer, also called a Z-buffer. It's a block of video memory that keeps track of the depth (distance into the screen) of each drawn pixel. When a new triangle is drawn, the GPU calculates the depth of each pixel in the triangle and compares it to the depth already stored in the depth buffer for that pixel. Only if the new pixel is closer does it get drawn (and the depth buffer updated). Thus, geometry that's behind other geometry gets culled away.
This image-space approach allows for interpenetrating geometry to be rendered correctly, and is a lot easier to parallelize (i.e. to make it fast) than an approach based on depth sorting at the triangle level.
Now, for transparent geometry it's another story. For transparent stuff the depth buffer doesn't help you, because you can't just cull away a transparent pixel that's behind another transparent pixel. In this case, the usual approach is to sort objects or individual triangles from back to front on the CPU, then the GPU will draw them in the correct order. The GPU doesn't have any built-in sorting capabilities that help with this case. Sorting by the depth of each triangle's centroid is a common and generally reasonable approximation, although it does not work in all cases.
Lastly, there are advanced techniques known collectively as "order-independent transparency" that allow you to do per-pixel depth sorting on the GPU for transparent geometry. These techniques can generate a visually correct image in cases where triangle sorting fails, but are very expensive in performance and memory.