This really is just the way things are with collision. Not just 3D, but also 2D. Take the following example:
The green and red AABBs are colliding, and the contact manifold is the blue area. The contact points will be in the blue area somewhere (exactly where can vary with algorithm, but the corners where the blue/red/green meets are ideal-ish).
What surface normal should be returned? The top edge of the red AABB or the left edge? If the green box is falling down, maybe we can reasonably guess the top edge. if it's moving right, maybe we can reasonably guess the left edge. What if it was moving down and right? Do we take the axis of least penetration? The velocity axis of greatest speed? Some heuristic of both? What if the boxes would have collided exactly on the corners?
Extend this to a complex 3D surface potentially made up of 100s of tris/faces. You're still only going to have a small number of ideal-ish contact points. What surface normal should be returned? The average surface normal over the whole tri-mesh (which doesn't make sense for most objects)? The points directly "under" the colliding box's corners (which aren't well defined for most other shapes)? Do you try to find the closest face to the contact points generated (which would require a second pass, since contact points are not calculated directly from any mesh faces)? If you do find the closest face, do you take the face's normal or do you interpolate the face's vertices at the point of contact to get the correct normal for "smooth" objects?
Really, the primary problem is that the contact points aren't all the contact points. In many cases, that would be an infinite set of points, after all. They're just a few points that are distributed well that make it reasonably feasible to approximate physical reaction by applying forces at said points to push the colliding objects around in a reaslitic-ish manner. The specific points/locations of actual object contact are abstracted away behind a simplified mathematical model. Hence, the idea of a specific surface normal of contact simply doesn't make much sense in the general case.
Of course, with more specific constraints and limitations on your objects, world, and motion, you can create alternative collision algorithms that can tell you about surface normal. In the 2D case above, if we assume that the boxes never rotate and that we know the relative velocity and last position of each, would could use continuous collision detection to find out exactly when they would collide and which features collided, supplying us with the exact feature where collision occured, which could then be used as a contact/collision/surface normal. Platformer games are built entirely on such assumptions and special tricks (Which is why using a general physics library like Box2D or Havok or the light never produces the kind of tight, precise control you find in classic platformers like Mario or Sonic; I wouldn't want to say it's impossible, but... it's impossible).
General Newtonian physics libraries, like that used in Unity3D, can't make those kinds of simplifications and assumptions. Therefor, you don't get collision surface normals, you get a contact manifold, generally simplified down to contact points, and that's it.