Why can't a blendShader sample anything but the current coordinate of the background image?

Question

In Flash, you can set a DisplayObject's blendShader property to a pixel shader (flash.shaders.Shader class).

The mechanism is nice, because Flash automatically provides your Shader with two input images, including the background surface and the foreground display object's bitmap.

The problem is that at runtime, the shader doesn't allow you to sample the background anywhere but under the current output coordinate. If you try to sample other coordinates, it just returns the color of the current coordinate instead, ignoring the coordinates you specified. This seems to occur only at runtime, because it works properly in the Pixel Bender toolkit.

This limitation makes it impossible to simulate, for example, the Aero Glass effect in Windows Vista/7, because you cannot sample the background properly for blurring.

I must mention that it is possible to create the effect in Flash through manual composition techniques, but it's hard to determine when it actually needs updated, because Flash does not provide information about when a particular area of the screen or a particular display object needs re-rendered. For example, you may have a fixed glass surface with objects moving underneath it that don't dispatch events when they move. The only alternative is to re-render the glass bar every frame, which is inefficient, which is why I am trying to do it through a blendShader so Flash determines when it needs rendered automatically.

Is there a technical reason for this limitation, or is it an oversight of some sort? Does anyone know of a workaround, or a way I could provide my manual composition implementation with information about when it needs re-rendered?

The limitation is mentioned with no explanation in the last note in this page: http://help.adobe.com/en_US/as3/dev/WSB19E965E-CCD2-4174-8077-8E5D0141A4A8.html It says:

"Note: When a Pixel Bender shader program is run as a blend in Flash Player or AIR, the sampling and outCoord() functions behave differently than in other contexts.In a blend, a sampling function will always return the current pixel being evaluated by the shader. You cannot, for example, use add an offset to outCoord() in order to sample a neighboring pixel. Likewise, if you use the outCoord() function outside a sampling function, its coordinates always evaluate to 0. You cannot, for example, use the position of a pixel to influence how the blended images are combined."

Answer 1

It's because "blend shaders" are things that don't actually exist.

Imagine if someone decided to implement "vertex shaders" in say, the GeForce 256. There's no hardware support for them, so it would either be implemented on the CPU, or the language would have to have a lot of limitations. Vertex positions would have to pass through one or two matrix multiples; you couldn't do any other math to them. Texture coordinates could get exactly one matrix multiply. Or you could take texture coordinates by multiplying the vertex position by a matrix. Or you could use a special function to generate it. But you couldn't use regular math.

And so on.

There are no "blend shaders" in real hardware. Therefore, a blend shader is bound by the limitations of the actual hardware, which doesn't allow for programmable blending. Blending hardware operates on a specific sample(s) within a specific pixel. It is given a value from the fragment/pixel shader, and it is given a value from the framebuffer. Any apparent additional functionality from blend shaders is carefully created by modifying the actual fragment/pixel shader used by the hardware.

Even so, you're still stuck with the basic limitations of blending.