One issue you'll run into is that the mesh is camera-relative, so you'll get a "swimming" effect as you move and pan the camera around, as the topology of the mesh sweeps over the underlying heightfield data.  This can be a distracting visual artifact.