LateUpdate is meant to be called after all the other updates are done. You'll want to utilize this method because you want to ensure the player position has been updated before you update the position of your camera. (Actually the Unity documentation mentions something very similar.)
Now that we know where to put it, there are two ways I can think of to make the camera follow the player:
Use any kind of interpolation: A simple example might be linear interpolation, expressed in parameterized vector notation, vectors are in bold
NewPos = CamPos + t * normalize(PlayerPos - CamPos);
Lerp is also a build in function that's available to you.
Using this with LateUpdate will give lag similar effect.
Elastic based camera: With the spring formula
F = k x
Fcan be factored into F=m* a where
m is mass(constant) and a is acceleration so you can modify the camera position using the calculated values.
In general this will give you a spring like behavior, but you need to multiply with some damping factor so it won't "bounce" forever.