A common approach is to write separate renderers or renderers with "feature gates."
That is, you can make separate files for each API (GraphicsGL2.cpp, GraphicsGL4.cpp, etc.). You can then make some kind of
IRenderer interface each implements or you can simply have one header and conditionally compile the renderer .cpp files for each platform (compile-time static polymorphism of a kind).
Alternatively, if you're sticking with one API family (GL), then you would have code like so in your renderer:
You could use function pointers as well as set them up during initialization.
Yes, this can be messy. That is one of the many "joys" of using OpenGL. After having to select the right version, find extensions, and work around the myriad of vendor bugs you end up with a hodge-podge of code.
Using shared libraries offers little benefit over just embedding the different renderers, assuming you again are sticking to one API family.
If you end up using completely different APIs (Direct3D vs OpenGL vs GL|ES etc.) then your best bet is the compilation method I first mentioned. There's zero reason to use OpenGL on Windows if you already have D3D, you need a very specific implementation of GL for OSX, mobile platforms need GL|ES, and Linux needs to be kept at an old low-level version of GL if you want to support the standard FOSS drivers.
If you're looking to maximize portability with the least amount of code then write everything with GL|ES (which is compatible with sufficiently new versions of desktop GL for the most part, given the
ARB_es3_compatibility) and don't worry too much about fallbacks to old versions (and hence limit your conditional code to the narrowest cases).