Technique to find a mesh intersecting with a primitive solid without using colliders?

For what it's worth I'm using Unity3d, but I believe the problem is more general.

There are hundreds if not thousands of meshes with hundreds of vertices each. There is one sphere that can be intersecting any of those meshes and is constantly moving.

Are AABB colliders approximating the mesh's volume the best way to go about this? Or is there a more specific technique that can be used here?

(A limitation I have is that my colliders need to be rotated, so they're each stored in a game object, so the number of game objects becomes absurdly large for my meshes(tens of thousands) which is slowing down my game.)

I have a couple of properties unique to my situation I believe might allow me to ditch colliders:

• I don't care about the meshes colliding with each other. Just the one sphere.
• I only need to know that the intersection exists, I don't care about the point of the intersection or any other metadata. Just a boolean of the intersection occuring and what mesh it happened with.
• I don't need to know the exact moment/frame when the intersection occurs. If I can find out an intersection occurred within 100-200ms/20-30 frames of it happening it'll be fine.

As a general solution, a sphere collider can be imitated with this a simple 3D distance formula equation: Distance = sqrt((x_2 - x_1)**2 + (y_2 - y_1)**2 + (z_2 - z_1)**2) where x_1, y_1, and z_1 are the x, y, and z coordinates of center of the sphere and x_2, y_2, and z_2 are the x, y, and z coordinates of a given vertex.

Using this idea, you could simply write something like (and this is pseudocode):

for(each Mesh)
{
for(each Vertex in Mesh)
{
Distance = sqrt((Sphere.x - Vertex.x)**2 +
(Sphere.y - Vertex.y)**2 + (Sphere.z - Vertex.z)**2);