I've got to write, what can be summed up as, a complete 3D game from scratch this semester. Up until now I have only programmed 2D games in my spare time, the transition doesn't seem tough, the game's simple. The only issue I have is collision detection. The only thing I could find was AABB, bounding spheres or recommendations of various physics engines. I have to program a submarine that's going to be moving freely inside of a cave system, AFAIK I can't use physics libraries, so none of the above solves my problem.
Up until now I was using SAT for my collision detection. Are there any similar, great algorithms, but crafted for 3D collision? I'm not talking about octrees, or other optimizations, I'm talking about direct collision detection of one set of 3D polygons with another set of 3D polygons. I thought about using SAT twice, project the mesh from the top and the side, but then it seems so hard to even divide 3D space into convex shapes. Also that seems like far too much computation even with octrees.
How do professionals do it? Could somebody shed some light.