This may be a long shot, but is there a small framework out there anywhere for testing custom bounding classes and other collision objects? (for example between rays, spheres and AABB's)
I don't think you'll find any testing frameworks specifically geared towards testing bounding volumes and other geometric primitives or operations, but you can accomplish what you want with unit testing frameworks.
Note that you don't need to create strict unit tests to actually employ a unit testing framework; it's perfect acceptable to use them to simply drive some quick automated tests. I would peruse the above, plus whatever else you might find on Google, pick a framework you like working with, and write your tests in that.
When building tests, I find it beneficial to create one test for all classifications of input. This actually tends to be fairly easy when writing tests for mathematical operations -- for example, were I writing a set of tests for the sign() function I would have at least three: one for zero, one for a positive input, and one for a negative input.
It's also helpful to work through the expected test results by hand prior to writing the test. This will both help you reinforce the math yourself and help ensure you use the same conventions used in your actual math routines.