There are a number of reasons why rendering and physics pipelines have traditionally been kept discrete. Bear in mind as I list these, that this is not just about games. Your question touches on any application that uses a 3D rendering technology like OpenGL, it's competitors, or it's forerunners.
- Not every application that use 3D, needs physics. Remember that OpenGL wasn't just built for games, its usage covers everything from medical simulations to modelling sales statistics, air traffic simulations to complex chemical reactions, and so on and so forth.
Any system which serves too many purposes becomes diluted in terms of efficiency. A graphic pipeline has to be incredibly fast. The moment you start introducing points where that pipeline has to interface with other subsystems such as the main system memory (where your code resides), you are going to experience order-of-magnitude reductions in efficiency. The hardware on your graphics card is very specifically narrowed down and ultra-optimised for pushing pixels, at great cost and many years of highly competitive research.
The nature of the mathematics and data structures surrounding physics operations (particularly collision detection and resolution, without which there really is no physics) is largely different from the mathematics required for rendering.
- There are solutions that deal with physics in hardware. But unlike the way in which we render, the way in which we perform physics in any given situation differs widely. The most fundamental indication of this is that Newtonian physics is not one-size fits all; there are other ways of modelling physics mathematically that are more appropriate in other situations, such as Hamiltonian mechanics. And in games particularly, every individual game may choose to model its physics differently, whether in 2D or 3D. These need not even reflect real-world physics! -- because a game is a product of the imagination. In other words, physics are ultimately part of your game dynamic, and that may change from game to game -- let alone all of the other solutions to which technologies like OpenGL are applied.
Accurately simulating physics on a vertex-by-vertex level is, for the most part, not presently a viable option. Given the high poly counts of models in most games, and the inherent difficulty of collision detection involving concave polyhedra, it is not as simple as just calculating physics off the model provided. For many if not most 3D games, bounding volumes in the form of cylinders or boxes are used to simplify collision detection, where that sort of level of collision detection is even required. Given current technology, the level of processing involved wouldn't leave much room for the rest of your game logic to run. Even Nvidia's PhysX requires complex, concave polyhedra to be decomposed down into simpler convex polyhedra, for physics simulation.
Your graphics card is producing perspective with the transformations it performs. That is different from the transformations performed in physics, which have nothing to do with perspective as such -- it is simply calculating basic positions and orientations in your world. If you know about MVC, you will understand that there is a distinct difference between the data you hold in your application, and how your present that data.
The computer technology industry is driven by need, and while visualisation is an almost universal need, physics simulations are by no means universal either as a requirement or in terms of their respective implementations.
So my advice is, stop worrying about going against the grain and start focusing on how to do the two things you need to do: rendering and physics. You aren't going to get at the data in your graphics card's processing pipeline (CUDA/OpenCl are exceptions): you push in triangles and material data, it pumps out your 3D world as moving image.
As a final aside, your question is not without sense. The desire for a combined basis for physics and rendering, and the fact that floating-point math can be much slower than fixed point, shows us why voxel technologies are seeing a huge resurgence of interest: they simplify the entire world down to grid-positioned, axis-aligned convex polyhedra. This vastly improves performance, by reducing the amount of floating-point vector math needed for physical operations, as you are now working primarily in a spatially-subdividable, instanceable, integer-indexed grid. This same grid can be used for both rendering and physics, particularly as you may use different grid resolutions for these two distinct subsystems (when using an octree-based solution such as SVOs).
