I think I solved the problem. I was confronted with the same problem:
I used a SurfaceFormat.Single-Texture to make some displacement in my HeightmapShader and everything works until I change the resolution (switched to Full-screen or back). During this switch my SkyboxDrawer complains about exact the same thing. But my Skybox-texture was in SurfaceFormat.Color, so the exception doesn't make sense at all.
So I start googling and found out, that Sharwn Hargreaves (a programmer of XNA with a great blog on XNA and other stuff) mentioned that there is a bug in XNA4.0 connected to the correct sync of settings of the sampler states with their C# counterparts.
My sampler looks like the following, so I thought all that should be done, was done.
sampler TextureSampler = sampler_state
Texture = <Texture>;
MinFilter = POINT;
MagFilter = POINT;
MipFilter = POINT;
AddressU = Wrap;
AddressV = Wrap;
But what really solved the problem was setting the VertexTextureSampler before drawing my Heightmap. This made the given exception in my SkyboxDrawer disappear.
GraphicsDevice.VertexSamplerStates = SamplerState.PointClamp;
For completeness: My SkyboxDrawer uses the BasicShader whereas my HeightmapDrawer uses a custom HeightmapShader. So the only connection between them are the states in the grafics pipeline.
But there is a unsolved "problem". As far as I know a vertex textures sampler in XNA only can use floating point based formats like SurfaceFormat.Single, SurfaceFormat.Vector2, SurfaceFormat.HalfSingle and so on. These formats require POINT sampling. So why the hell am I able to change the GraficsDevice.VertexTextureSampler[*] to something else?! Did I miss something?