On a modern OS, where the desktop window compositing system is backed by a hardware-accelerated API like OpenGL or D3D, using the OS's drawing API will ultimately involve some calls being made into that underlying hardware-accelerated API. However:
- It is not always the case that the OS is using GPU-based compositing (particularly on older versions).
- You are paying a penalty in the form of the additional abstraction layers between you and the GPU.
In other words, on older versions of Windows, making calls into the GDI API can very well result in completely CPU-side software processing and rendering of the primitives you are drawing.
Even if the above is not the case, you're still calling into GDI, which does a lot of work to transform your input into internal formats, copy data around to buffers, et cetera. Eventually that data is translated to something D3D could comprehend. This usually means the GDI buffers are blit to textures on the CPU (exactly as they would have been in the old days) which is then given to the GPU to render.
In other words, you could do a lot better in terms of efficiency of rendering 2D elements by directly accessing D3D or OpenGL.
Futhermore, if you want to use 3D, the OS level APIs generally provide no facility for that (since they were originally designed to do 2D composition of GUI elements), so you'll do all that 3D geometry transformation on the CPU side in order to transform it to a form you can submit to GDI. You won't be utilizing the GPU hardware effectively at all.