From what I understand, the main difference between software rendering and hardware rendering is that in software rendering you use the CPU to determine what to color each pixel, and in hardware rendering you let the GPU take care of calculations and determine how to draw each pixel.
Basically, yes. The difference is that you have complete control over a software rendering pipeline, because you're building it on generalized computational hardware. A modern GPU is a much more specialized machine, and even though they are reasonably programmable these days, there are still aspects of them that are not, and they are still purpose-built to accomplish their task in a very particular fashion.
However, in the end, isn't an API such as OpenGL still required to do
the actual drawing no matter which method you pick?
Yes, some API that ultimately deals with communicating with the graphics hardware is required. While modern versions of operating systems generally composite their windows with OpenGL or D3D, this is only relatively recently the case. Older operating systems would have talked to different drivers to get pixel data from their drawing APIs (such as GDI, for Windows) pushed across to whatever graphics hardware was attached; typically direct access to the video memory was exposed via some mechanism.
I saw a basic software rendering engine that someone built, and the person said he still used OpenGL to do the actual drawing after using the CPU to do the rendering.
In a modern machine, you can no longer get direct access to video memory, so modern software rendering pipelines tend to do everything on the CPU until they have a final, rasterized 2D image. Then they "blit" that image to the screen by rendering it as a fullscreen quad with some hardware graphics API like OpenGL.
And if so, is it true that a computer must at least have an integrated
GPU or dedicated GPU to display anything at all?
Yes. All computers need some kind of hardware to convert visual information to an output device. The specific term "GPU" was pushed in the late 90s, mainly by Nvida, so prior to that we tended to call these chips different things, but ultimately some kind of hardware was always involved.
