You can use a model. Often, HUD elements, sometimes including weapons, are rendered seperately from the environment. In the simplest example you'd set up the animation so that the ironsights line up with the zero-axis, and align the model's zero axis with the camera.
Using Unity as an example, you'd put the weapon in the HUD layer so that it's drawn with a camera that only draws the HUD layer.
Alternatively, you can set up the player avatar so that it lines up the same way, using the same camera. This has the added benefit of displaying to other players what you're doing without having to make a separate model, but re-positioning the gun with respect to the camera can be fiddly.
Of course, it's possible to use an image as well. If memory serves, TF2 uses a combination of these - with the sniper rifle animation leading into a sprite overlay with the sights dead center of the screen.