I was wondering if it is a improvement to use OpenGL for matrix calculations instead of using the CPU. And if it is a improvement, is it worth it to change the math class to use OpenGL?
It depends what you mean. The answer is generally no.
If you're talking about using OpenGL to leverage the GPU as a general-purpose parallel computation device, it can potentially be reasonable to offload matrix operations to the GPU if you're doing a ton of them. The GPU is good at lots of parallel math operations, especially math operations regarding matrices and vectors.
However it's not so great at getting the results of those operations back over to the CPU. There is overhead to read that data back, so generally you want to be doing thousands of parallel math operations at once to amortize the cost of that readback. Or you want to be doing those math operations in a context where you don't actually need to read the results back, but can continue to process them only on the GPU (as you would if you were simply rendering things or doing the matrix multiplies for hardware skinning, et cetea).
If you're just talking about multiplying one or two matrices at a time in a context where you also need those results on the CPU, it's definitely not worth it at that scale.
This is a misunderstanding on your part. glTranslate, glRotate, glScale, etc, do not happen on the GPU. Just because they're part of the OpenGL API you should not think that means they happen on the GPU.
What happens instead is that the matrix operations are performed on the CPU and the final calculated matrix is then uploaded to the GPU - in other words, very much the same as if you had used a matrix library instead of the OpenGL functions.