GDI is a software rendering API; all the work it does to accomplish the drawing of lines, shapes, fonts, and images is done in software. This is because GDI needs to provide a very high level abstraction over target hardware, which is useful for the context in which GDI is intended to be used (drawing application graphics on the huge variety of hardware Windows supports).
This differs from D3D and OpenGL, which ultimately result in a large portion of the heavy lifting to transform and rasterize 3D geometry being executed on the specialized hardware of the GPU.
On modern machines, the renderings produced by GDI are ultimately handed off to the GPU for final output to the screen. You can envision this conceptually as handing D3D a series of textured quads, where each texture is some GDI image (in practice, it's more involved). The key point is that GDI is still doing all its work CPU-side.
OpenGL and DirectX, if we are talking about graphics, are somehow
built into a graphics card and you can access them using appropriate
environment and draw some graphics.
This isn't strictly correct; OpenGL and D3D are not built in to the graphics card. OpenGL and D3D are primarily implemented by drivers for the card. The drivers are software, not hardware, although each hardware vendor writes their own according to the internal, often-proprietary specifications of the cards themselves.