Yes, you are right, this 2D water effect can be simulated using math sine function :
wave = sin(phase + t * frequency) * amplitude
phase is a constant, put whatever you want.
t to horizontal position of pixel/vertex you are processing :
t = x;
amplitude over the time (that will make the waves moving up and down) :
amplitude = sin(t * wave_speed) * wave_height
combine several waves to get a more relastic effect :
wave_final = wave0 + wave1 + ... .
For each wave, change some parameters a little bit (eg: phase, frequency,...).
Here is a quick example i made, using only two waves :
http://glslsandbox.com/e#4988.0 (require recent browser and WebGL enabled)
NOTE : this is a shader, but doing this using 2D primitives is the same approach .
EDIT : you dont specify any framework or 2d rendering system, but here is how it could be rendered using polygons / triangle strips :