Is deferred rendering/shading possible with OpenGL ES 2.0 ?

Question

I asked this on StackOverflow, but it might make more sense here:

Has anyone implemented deferred rendering/shading under OpenGL ES 2.0? It doesn't support MRTs, so with only one color buffer, it's not something that can be implemented in the "usual" manner.

Specifically, I'm exploring on iPad, iPhone4 (maaaybe iPhone 3gs), and Android. On the GLESView app on iPad/iPhone4/iPhone3gs, the GL_OES_RGB8_RGBA8 extension is present, and I haven't looked too deeply yet, but with 8bits/channel, this idea is interesting: http://www.gamedev.net/topic/562138-opengl-es-20-and-deferred-shading/

Any other ideas? Is it even worth doing, performance-wise?

Answer 1

Yes, it is possible. However, it isn't particularly worthwhile.

First, unless you have access to the NV_draw_buffers extension (as the name implies, it is NVIDIA-only. So unless you're running on Tegra, you don't have it), framebuffer objects under ES 2.0 can only render to one image at a time. So to generate your G-buffers, you will need to render your scene multiple times, thus taking away one of the advantages of deferred rendering.

Second, bandwidth on mobile platforms is not the same as you would get even on mid-grade GPUs. And bandwidth is critical to making deferred rendering (for many lights) worthwhile. Without that bandwidth, the light passes are really going to hurt, performance-wise.

Third, PowerVR hardware is really not designed for this kind of thing. It optimizes rendering with its tile-based rendering hardware. So deferred rendering on top of that would be less helpful than in a traditional scan-conversion architecture.

Answer 2

The main problem is Fillrate. On mobile GPUs, your fill rate is low that you can't do Deferred shading in realtime at native resolution.

On iPhone 4 & iPad 1, fillrate is just ridiculous. The only device IOS with good fillrate is iPad 2, but i doubt there is enough... On android, only Tegra devices have the GL_NV_draw_buffers to use MRT but fillrate is very low too. The Mali 400 seems to have the best fillrate. If you want cry, just try to fill a color rectangle at fullscreen resolution 4 times... Many devices can't do it 60 fps.

On desktop GPUs, you have 10 times (or more) fillrate as mobile gpus. Don't forget that mobile GPUs use the same memory as CPU and you don't have dedicated memory.

There are some examples in WebGL (same API) so that isn't a limitation of the API.

Answer 3

Really you have to consider what is the absolute minimum you need for a deferred renderer. If you fall back to deferred lighting it reduces the amount of data that needs to be stored in the GBuffer, and really it's a helluva lot cheaper than rendering half the scene 3 time or more to support a low amount of lights.

I would go for the following GBuffer format:

• Reuse the depth buffer for the lighting pass, not sure how widely this is supported on mobile devices, but it's a free depth texture.
• A single GBuffer texture, inside it I would store: Normal U, Normal V, Param 0, Param 1. Lambert-Azimuthal encoding looks really nice for normals and compresses them down into just two components, relatively cheap to encode and decode as well.
• Two parameters for lighting variables is a lot, could use one as an enumeration for multiple lighting functions if the hardware does well with dynamic branching.

Deferred lighting is similar to deferred rendering, except you render the scene twice:

1. Render Geometry Depth, Normals, and Lighting Parameters into the GBuffer.
2. Render Lights into the accumulation buffer.
3. Render Geometry with material shaders, composite your lighting here as well. If your good with working out reverse operators of lighting equations you can do a LOT of really cool things with this step.
4. Do any post-processing you can afford, be sure to abuse the depth and normal textures to death here for efficiency sake.

About storing the lighting results. I've become fond of storing diffuse color and specular luminance so that the accumulation buffer only needs to be a standard 32-bit color texture. You can estimate the specular color by calculating chroma of diffuse color and combining that with specular luminance.

The most important part however is going to be using the depth-stencil buffer to it's fullest, ensure that your not rendering any of the lighting code where it's not needed. I'd even go so far as to add some discard statements into the fragment shaders on terms that will drop light visibility below the device's displayable range (1e-3 is usually a safe cutoff).

Answer 4

Consider deferred lighting. In a nutshell, deferred lighting is a technique that uses a reduced version of deferred shading to calculate a screenspace lightmap. In a second pass, the geometry is rendered again using the screenspace lightmap as lighting information.

This technique is used to reduce the size of the G-Buffer, since fewer attributes are needed. It also buys you the advantage, that the G-Buffer and the screenspace lightmap can be of lower resolution than the screen.

I had implemented a strict GLES 2.0 based renderer (although an experimental one), and I managed to boil the G-Buffer down to one RGBA texture (yes, I used a texture2D instead of a renderbuffer). It contained the screen space normal map + the depth buffer in the alpha channel (which was compressed using a logarithm, as far as I remember).

The position attributes (called world here) can be calculated during the lighting pass using the fact, that in a perspectivic projection, .xy is devided by .z, so that:

$$xy_{frustum}=xy_{world}/z_{world}$$

I approximated the position attribute's xy by doing:

$$xy_{world}=xy_{frustum}*z_{world}$$

Note: I had to do further adjustments depending on the projection matrix' settings.

Also worth noting is, that I was capable to omit the .z component of the normal vectors, since i could reconstruct .z from .xy since the normal vector is normalized so that:

$$ \sqrt{x^2+y^2+z^2}=1\\ x^2+y^2+z^2=1\\ z^2=1-(x^2+y^2)\\ z=\sqrt{1-(x^2+y^2)} $$

Using this technique, i was capable to free up another channel in my RGBA G-Buffer and used this to store the screen space specular-map (or glossiness, if you will).

Answer 5

Yes, it is absolutely possible. Fill rate is not such a problem because mobile graphics chips are designed to deal with very high resolution screens. In fact, deferred rendering helps with this because your lighting calculation is independent from scene complexity, so overdraw doesn't cause a slowdown. Here is my implementation on a fourth generation iPad: http://www.youtube.com/watch?v=K4X1oF6b4V8

If you want four times the performance, just half the resolution of the texture you render to. I can't really see any different with 3D graphics on a retina screen, anyways.

Mobile graphics chips are extremely good at deferred rendering because of the way they handle render-to-texture. Unlike PC graphics cards, which typically incur a huge performance hit when you start rendering to a texture instead of a window context, mobile graphics are designed to do this with no performance hit. So you get the scalability of a deferred renderer, without the initial performance penalty you experience on a desktop graphics card.

At the time of my implementation, OpenGLES was missing render to multiple targets, so I had to draw the screen color and normal in separate passes. This may be fixed in more recent versions of OpenGLES, but don't know if the solutions are available yet in consumer mobile hardware.