I didn't know what the exactly name for this effect, so I just paste the link of the video here, I want to know what is the knowledge background needed if I want to create such a effect?
The twisting/distorting effect can be done by making a triangle mesh that is initially a rectangle covering the screen, but with a fair amount of subdivision. Then you let the user alter the shape by clicking and dragging. One way to do that is by soft selection: when they click, you find all the verts within a certain distance of the pointer and compute weights that vary smoothly from 1.0 at the pointer to zero at the outside of the radius; then you adjust the positions of the points by weight * the drag vector. Another way to do it might be with a soft-body physics model, although that's a lot trickier.
You apply an image to it by simply putting a texture on the mesh. How they get the Windows desktop to be the texture I don't know. It's pretty easy to get a static screenshot of the desktop as your program starts up, and use that as a texture, but that wouldn't let you interact with it and get an updated render. In Vista/Win7 with the DWM it might be possible to get it to let you look at the composited desktop render target. However in the video it looks like Windows XP. I suspect there might actually be two computers running, one "slave" that displays the desktop normally, and one "master" that captures the slave's video signal and uses that as a texture in the distortion program. That's just a guess, though.