You are correct that a combined axis-angle representation like the one you describe has a stronger expressive power than many other systems because it can more conveniently store a rotation speed.
However, in practice, people actually use quaternions and 3×3 matrices to manipulate rotations a lot more than just represent them.
One typical operation is the combination of two rotations:
- with quaternions, this is usually 16 multiplications.
- with 3×3 matrices, this is usually 27 multiplications.
- with an axis-angle representation, you don’t have much choice but to convert the transformation to something else, often a quaternion, requiring at least 16 multiplications plus a square root and three divisions (for normalisation) and at least two sine and two cosine operations.
Another operation is the transformation of a vector by a rotation:
- with quaternions, this is usually 15 multiplications
- with 3×3 matrices, this is usually 9 multiplications
- with an axis-angle representation, this is at least 20 multiplications, a square root, three divisions, and two sine/cosine operations.
As you can see, the storage efficiency of an axis/angle representation is usually not worth it when you actually need to do things with the rotation. This article on Wikipedia has some more details.