In this thread Rendering a model with transparent or translucent uv map applied doesn't work , I asked about what was wrong with my model that the transparency wasn't working. I've got an answer that it was the sorting order that was wrong. I need to use back-to-front when, I assume, I was using front-to-back. Could somebody elaborate on that? I don't really understand what is the sorting order and how can I change it.
Transparency with OpenGL is typically achieved using alpha blending, as you are attempting to do with
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA). In order for you to get the desired effect, though, it helps to understand what the blending function means.
Let's consider the following scenario: you first draw a triangle, and then you want to draw a semi-transparent square on top of it. So first you draw a blue triangle, with the RGBA colours (0.0, 0.0, 1.0, 1.0). The alpha channel of 1.0 means it's completely opaque. Then you draw the green square on top, (0.0, 1.0, 0.0, 0.4). The resulting colour of where they overlap will be:
(0.0, 1.0, 0.0) * 0.4 + (0.0, 0.0, 1.0) * (1.0 - 0.4)
In other words:
square_colour * square_alpha + triangle_colour * (1.0 - square_alpha)
It's a very simple blending function. How it's actually achieved: when drawing a fragment (in this case a fragment of the square), the already existing fragment at that location gets sampled (which turns out to be green, since we drew a green triangle there). Then, using the old colour value at that location, and the colour of the fragment to be drawn, it blends the two for the resulting fragment. If you draw another semi-transparent object on top of that, it will sample the new colour, and again perform blending.
Now what if you were to change the order you drew the triangle and the square? Well, you'd draw the square, which is partially transparent, then you'd draw the opaque triangle, and... you guessed it, the result would be an opaque triangle, unblended.
So when you are rendering a 3D model with transparency, the order you draw them in matters! Not to mention, the problem gets worse with depth testing:
Let's say the square is closer to the camera than the triangle. If you draw the square, then the triangle, the square should still be on top. Except... the triangle won't get drawn at all? The reason is simple: with depth testing, before drawing the fragment, that location gets sampled once again, this time for depth. If the old fragment is closer to the camera than the new fragment, the new fragment gets discarded! So in the case of the square and the triangle, the triangle gets discarded, despite the square being "transparent".
Without depth testing, if you draw the square first, then the triangle, you get what you'd expect, the opaque triangle on top (not what you want).
Solution? Render the stuff furthest away from the camera first, then get closer (i.e. draw it back-to-front).
If you only have one layer of transparency, then what you want to do is: first draw everything that is opaque, then everything that is transparent. If you have multiple layers of transparency, then things get more complicated. Google depth sorting, and you'll find various different methods for achieving the desired effect.
The most robust method to ensure there are no transparency artifacts is to:
- Disable writes to the depth buffer (although you may want to have a separate depth buffer for effects, I'll explain later)
- Decompose your object into convex pieces (unnecessary in this specific case, a box is already convex)
- Sort the parts of all objects from farthest away to closest
- For each part in order, draw the backfaces and then the frontfaces (or just the frontfaces)
Depending on what you want, drawing both backfaces and frontfaces might be overkill, but it is useful for effects like opacity depending on thickness, refraction, and subsurface scattering.
You can safely draw all the fully opaque objects in one pass, because the depth buffer will take care of any overlap. To sort the convex parts, you can use the GJK distance algorithm to calculate the distance from the part to the near camera plane. This is embarrassingly parallel, so you can easily exploit multiple threads to do it quickly.
When attempting to do transparency, you need to be very meticulous in assuring you are drawing the rearmost elements first and you should also make sure that the polygons behind your cube (including parts of the cube itself) are not being culled. Using Opengl's built in depth test culling will not bend them properly if you draw them out of order. Here is the OpenGL spec on Transparency. http://www.opengl.org/archives/resources/faq/technical/transparency.htm