I understand mathematically how transformation matrices work, in that a transformation matrix multiplied by a vector results in a new vector, and we use the matrices for rotation, translation, etc. I also understand that in 3D, the transformation matrices are applied to every vertex of each model in order to transform them, although that's basically the extent of my understanding vis-a-vis 3D transforms, and maybe even that understanding is a little wrong/simplistic.
However, many game engines/graphics rending libraries are designed with the intention of rendering 2D graphics (ex: Flash, XNA, SDL, Allegro), although I know at least some of them are really 3D under the hood. These engines allow the programmer to use transformation matrices to manipulate 2D entities, to do something like implementing a camera, for example. But I don't understand what the transformation matrices I'm plugging in to the engine are being multiplied by in order to perform the transformation. For a while I assumed it was applying the transformation matrix to each pixel of the original image, but that doesn't seem right to me.
So what vectors are the transformation matrices being multiplied by in the 2D transformations? Was I right, and it's really a per-pixel operation? Or does it do some kind of 3D magic under the hood?