Well, if you stick to something very simple then you can do it using only basic linear algebra. I would list the absolute necessities as understanding a) orthogonal coordinate systems; b) matrix multiplication and c) how and why the two are related.
And if you want to actually construct some arbitrary rotation matrices then you'll at least need to understand the basic trigonometric functions and their relation to the circle.
The wikipedia page for Transformation matrix summarizes pretty much everything, and the links from that page has even more information (see 3D projection in particular).
There's also some amount of non-mathematical understanding required of course. If you want to put something together on your own then figuring out how and why to do something might require quite a lot of logical thinking, even when the math in question would turn out to be trivial (which might turn out to be most of the time).
However, I'd say that the most important skill isn't math but the ability to search for information. There is a lot of material in books and on the internet. If there is something you don't understand when reading then you can be quite certain that there are articles and books (or Stack Exchange for that matter) available to explain.
Right... It seems I've misunderstood the question, and overshot terribly. See above for a better aimed answer :)