Local minima and maxima
Randomize a set of float numbers from 0 to 1.0 (exclusive), denoting position along the x-axis. Do this once, and call these minima (they sit below the baseline). Do this again, and call these maxima (they sit above the baseline). Push each minimum down by a random amount. Push each maximum up by a random amount. Now merge the two sets into a single set, and sort by increasing x. Then join the points in order of increasing x. You could use lines, or some curve function if you want it more organic. There's your random terrain.
Sine-wave mashup
This alternative which might (a) be simpler to implement than Perlin noise, if somewhat more brute-force in nature, and (b) might simultaneously open your eyes as to how simple the 1D Perlin noise solution Nathan's explaining, is.
Consider a sine wave. It roughly approximates what you have in your diagram, except it's too smooth and regular.
You can generate a bunch of random sine waves. By random I mean, with random amplitudes, random frequencies, and random offsets from the origin. Then merge them. Then take only the uppermost (or lowermost) part of the combined wave functions. Then smooth the result. Now you should have some sort of an organic but irregular surface. TBH this is something of an oversimplification, and there are mathematically far better ways to merge/smooth, but you get the general idea.
This is exactly what 1D Perlin noise is. All you're doing is taking a cross-section through the noise plane that Perlin produces. Perlin noise is just a set of overlaid/merged waves which have random frequencies, amplitudes, and offsets (all of which are within user-specified ranges, being the parameters you pass to the Perlin noise function), and which operates in 2 or 3 dimensions.
So you can take either approach, and you'll get something like what you want. I'd still suggest using Perlin though, as there are no question marks there. It's a given that it'll work the way you want.