You typically do this with some kind of spatial partitioning. You only check objects against other nearby objects, and you don't need to update any objects that are far away from or behind the camera in many cases.
A relatively easy spatial partitioning scheme to get you started is an octree. Make a box around the world. All objects in it. When the box gets too full based on some pre-determined threshold, break the box into 8 sub-boxes and move each object into the appropriate sub-box (each box is a node in a tree, and only leafs contain objects). Do this recursively down to some reasonable minimum size. Objects can only collide with other objects in the same box or objects in adjacent boxes. When there are enough objects in your world, there will also be a number of divisions in your octree, keeping far-apart objects in separate, non-adjacent boxes.
You can also relatively easily check if any box collides with your view frustrum, and recurse through the tree, marking which boxes are visible, near visible, and completley invisible. You can use this to stop physics processing for areas far away from the camera. You can also use this same information to despawn enemies and items or disable AI updates. Instead of stopping physics/AI, you can reduce them so that they still udpate but take less time.
This information is also very useful to graphics to help you avoid rendering scenery and objects that can't be seen.
There are other, more efficient spatial structures, but which one is best depends on lot on your game. A very different set of techniques is useful for an indoor FPS vs a top-down RTS vs a wide-open terrain FPS, which is why many engines today support more than one, and evne allow mixing them together (e.g. for smooth transitions from the outdoors to indoors, where the algorithms of choice differ). For learning, stick to something simple like an octree. You have far, far more complicated problems to solve in getting 3D physics to work well.