0
\$\begingroup\$

Developer of the game explains how z-buffer works in Dwarf Fortress (loosely quoted):

"If you clear the buffer every single frame, it kills the game. You have sort of a depth of field. You say I need about 20000 integers worth of depth. So your first frame is 0 to 20000. And the next frame is 20000 to 40000, you treat everything lower as garbage. So you go 20 to 40, 40 to 60, 60 to 80. You have to clean it if you dealing with 4 byte integer something like every 100000 times. So the problem goes away."

Source: https://youtu.be/A9g1XMr2TTA?t=1628

His description is not completely clear for me. It seems he is using depth range from lowest z-level to highest and updating only one layer per frame?

\$\endgroup\$
  • 2
    \$\begingroup\$ He is talking about a software rasterizer I think. Is that what you're going with? He also states that modern hardware/software takes care of the depth buffer all you have to do is make use of it. I must say him rocking his body is extremely distracting \$\endgroup\$ – Sidar Mar 28 '18 at 7:06
  • \$\begingroup\$ I've listened to him multiple times and I can not understand what he is getting at. You say I need about 20000 integers worth of depth. So your first frame is 0 to 20000. I feel like both the 20000 here speak of different things? First part seems to talk about the amount of values he needs, second part sounds like he is talking about some depth range? \$\endgroup\$ – Sidar Mar 28 '18 at 7:26
  • \$\begingroup\$ @Sidar he means that he first uses values in the range of 0 gto 20000 and don't touch anything else. Then he goes one step further and uses the values between 20000 and 40000, this way the old values can still remain, but they won't interfere with the new ones. Kind of like having a very long corridor and dividing it into rooms and filling them with different kinds of liquids \$\endgroup\$ – Bálint Mar 28 '18 at 8:24
  • \$\begingroup\$ Right but his phrasing is strange \$\endgroup\$ – Sidar Mar 28 '18 at 8:58
3
\$\begingroup\$

What is the Depth Buffer?

The depth buffer stores how far away from the "camera" is what you have rendered. This allows the graphic system (the GPU does this in modern hardware) to check if what you are drawing is in front or behind, for every pixel, and based on that decide if it is drawn (and update the depth buffer) or not. We call that, the depth test.

I want to note that it is possible to invert the way the depth buffer works (regardless if you are doing it in on CPU or in GPU - as you should). To do that, you can flip your projection matrix along the depth axis and configure the depth test function appropiately.

I would like to recommend the Computerphile series Triangles and Pixels. The video "The Visibility Problem - Computerphile" (third in the series) explains the motivation of the depth buffer and how it works.

How was he using the Depth Buffer?

And so, if you can't clean the buffer, there is a trick I learned back when I was trying to do 3D graphics back in the 90's, before we had (???) cards, and things that would do for us, and OpenGL to do it for us.

You had to, actually, go draw pixel by pixel, via triangle, and you had the texture coordinate and you would be updating them as you went.

And you'd have the kind of the depth of it. You have to update that as you went along the triangle. And then to avoid triangles being printed over each other when they shouldn't be, that depth should be stored in the buffer.

(...)

And so, the big problem with the z-buffer is that you got a screen. (...) You have to clear it every single time, if you were thinking naively about it. And that just kills the game.

He is saying that in on old systems, they had to clear the depth buffer in CPU (drawing pixel by pixel). Consequently, cleaning the depth buffer was slow. That is what he means by it "kills the game" (makes it too slow to be playable).

So, what you do is you have your sort of depth field, or whatever, and you'd say "Oh, I need about 20000 integers worth of depth here" and then you start your first frame 0 to 20000 and then your next frame you say my values are 20000 to 40000 and you treat anything lower like garbage.

Because of that, for early games, they did a trick: They avoid clearing the depth buffer, but instead they make sure to that everything they are drawing is in front of what was previously there. That means higher depth values in this case.

The idea is that the first frame will have objects in the depth range of 0 to 20000 (because that is how much depth rage he needs), and the next frame will have depth values in the range of 20000 to 40000 it will always draw on top of the previous frame.

Note: He seems to be working with an inverted depth buffer (what has a higher value is ontop of what has a lower value). If he is handling the depth buffer in CPU - which seems to be the case - it is not a problem implementing it like that. To be clear: The default depth test function is to overwrite what has a higher value (That is GL_LESS in OpenGL). Instead he overwrites what has a lower value.

You only have to clean it if you dealing with 4 byte integer, you would clean it once like every 100000 times. So, the problem goes away.

Eventually you run out of possible depth values (a 4 bytes signed integer let you count up to 2147483647, then you use 20000 values each frame, meaning that each 107374 frames - roughly each 100000 frames - you have to clean the depth buffer).

Does Dwarf Fortress render like that?

And so the path-finding buffer would work like that in Dwarf Fortress. I know what the longest path could be, and then that's the value we work with, and the I had (???) reset the floor. So that we were working from 0 to 200000, and 200 to 400, and 400 to 600...

And I almost never have to clean the path map now; it cleans like every 100000 paths that are made through a section of it.

It appears he got the figures wrong this time. The steps would be 20000 if you want to clean each 100000 iterations, with a 4 bytes integer.

He is saying that Dwarf Fortress uses this for path finding (not for rendering). He allocates a path-finding buffer. I am not sure what he stores there, that is, I do not know how paths are represented. The gist is that the values stored are left behind and he moves to use a higher range instead of clearing the buffer. By doing that he can minimize the number of times he has to clear it.


Is that good advice for rendering a modern game?

He is NOT currently using it for rendering; he did use it in old hardware. If you are developing for modernish OpenGL or Direct X, do not do what he says there.

I want to quote this from the Common Mistakes page of Khronos Group Wiki:

Depth Buffer Precision

When you select a pixelformat for your window, and you ask for a Depth Buffer, the depth buffer is typically stored as a Normalized Integer with a bitdepth of 16, 24, or 32 bits.

Note: You can create images with true floating-point depth formats. But these can only be used with Framebuffer Objects, not the Default Framebuffer.

In OpenGL, all depth values lie in the range [0, 1]. The integer normalization process simply converts this floating-point range into integer values of the appropriate precision. It is the integer value that is stored in the depth buffer.

(...)

Now that the misconception about depth buffers being floating point is resolved, what is wrong with this call?

glReadPixels(0, 0, width, height, GL_DEPTH_COMPONENT, GL_FLOAT, mypixels);

Because the depth format is a normalized integer format, the driver will have to use the CPU to convert the normalized integer data into floating-point values. This is slow.


Swap Buffers

A modern OpenGL program should always use double buffering. A modern 3D OpenGL program should also have a depth buffer.

Render sequence should be like this:

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
RenderScene();
SwapBuffers(hdc);  //For Windows

The buffers should always be cleared. On much older hardware, there was a technique to get away without clearing the scene, but on even semi-recent hardware, this will actually make things slower. So always do the clear.

I want to note that there are scenarios where not clearing the depth buffer makes sense. For example if you are loading a pre-rendered scene each frame, then there is no point to clear it, because you are going to fill it right away with default values. That is a valid optimization.

However, the above quote is the guideline and usually the best practice. So, please start there, and optimize only if you really need it.

I will also link you to the Depth Precision Visualized page from NVIDIA. For an abstract: being the depth a float, it is open to many floating-point precision issues.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.