I've been the graphics performance guy for a few big-name 3D iOS titles, so I completely feel your pain on this one.
My experience has been that transparent pixels are the most expensive thing you can render on modern iOS devices. Your conjecture is right; the greater number of pixels on the iPhone4 really kills its performance in general, and transparent pixels makes it doubly-bad. You'll find that you have the same problem on the iPad 1, as well.
The particularly slow point is the point where the drawing of a pixel needs to check what the color behind it was. This occurs for transparencies, for alpha masking ("discard" statements in shaders), etc.
In order to deal with this, we took a few steps:
- Wherever possible, we made more draw calls in order to draw more opaque textures. For example, we made tree foliage consist of two parts; the opaque bits in the middle, and the transparent bits around the outside. This meant we drew far more triangles and had to perform more draw calls overall, but it reduced the number of transparent pixels on the screen. Similarly, when we wanted a "vignette" effect which applied a circular darkening around the outside of the screen, we did that by drawing small rectangles with the darkness texture at the edges of the screen, instead of by naively drawing a single full-screen quad that covered the whole screen.
- Wherever possible, we used OpenGLES shaders to render the transparency as part of the fragment shader. So inside the fragment shader it would do normal shader activities, but would also check the fragment's position on the screen, and figure out any overlay modifiers which needed to be applied.
- We disabled many shaders and transparency effects on iPhone4 and iPad1, judging that they were too expensive for the hardware.
We also tested (but didn't actually ship with):
- Render into a render target that's only 3/4 of the full screen resolution, and scale it up for display. This made a huge improvement to performance, but we decided the quality loss was unacceptable for our game.
So there are definitely savings which can be made. But I want to stress that you're going to have performance problems on the iPhone 4 and the iPad 1 regardless of what you do; their graphic chips simply aren't fast enough to power their number of pixels, and neither have good shader performance (my tests indicated that their baseline performance running with fixed-function-alike shaders had them running 5-10% slower than they did under the fixed function pipeline). And this was a full-time job for me; if you're in a smaller team, you may not want to invest your man-months in that way.
As a general rule of thumb, your best bet is to limit overdraw as much as you can, and to try to minimise the number of pixels affected by transparency as much as you can (either by making less stuff transparent, or by moving the transparent stuff further away from the camera, so it's not covering as many pixels).
And then you have to decide how low you're willing to go in terms of frame rate on the troublesome iPhone 4 and iPad 1, and start cutting effects in order to meet that target frame rate.