# Painter's theorem inside and outside test [closed]

I have read from Hearn and Baker computer graphics book. I read the test#2 painter's theorem which image is given below.

This image showing surface S is completely behind the surface S', by checking Substitute the coordinates of all vertices of S(x, y,z) into the plane equation of S' and check for the sign. If all vertices of S are inside S' then S is behind S'. (Fig. 1).

i. e. Ax + By+ Cz + D < 0 ,x, y, z are S vertices.

But I have read from this websites which showing this concepts but in totally opposite manner. See this image: I read from that site, which showing opposite thing on same image. This showing all vertices of S are outside of S'.

i. e. Ax + By+ Cz + D > 0 ,x, y, z are S vertices.

My question is how is it possible Hearn and Baker saying S is completely inside the surface S' but my mentioned website saying S is outside of S' by sign test of plane equation?

• I’m voting to close this question because it is a cross-post. Dec 30 '21 at 12:29
• – D.W.
Dec 30 '21 at 20:01

I'm not familiar with the exact situation being discussed here — it's been a while since I've worked with plane equations — but in general, much of the math in computer graphics has opportunities to change signs or axes and get the same answers. A surface is a 2-dimensional object and when you embed it in a 3-dimensional space you need an arbitrary convention defining which side of it is “inside” and which side is “outside”. In this case,

$$Ax + By + Cz + D > 0$$

is the same inequality as

$$(-A)x + (-B)y + (-C)z + (-D) < 0$$

so if we compute the coefficients of the plane in a way which swaps the signs, then our front-or-behind tests should also be reversed to come up with the same results.

The web site you link mentions

A, B, C, D are from plane equation of P (choose normal away from view plane since define "outside" with respect to the view plane)

If you “choose normal” the opposite way, you will get the opposite set of coefficients.

The math works the same either way, but it's good to pick existing conventions and stick with them (and document which one you're using), because mixing them up is a great way to get confused, especially as two unintended swaps can cancel out and get the right answer by an unintended route, creating a lurking bug that pops up and makes all your graphics disappear when you change one of those two parts.

Another example of arbitrary binary choices:

In GPU triangle mesh rendering, a commonly seen example of an arbitrary sign choice is choosing whether your triangles have their vertices ordered counterclockwise or clockwise, when viewed from the outside/above. Either convention works, and allows the GPU to efficiently skip rendering half of the model of a solid object, by skipping all triangles whose vertices are in the opposite order (as projected on the screen) and therefore must be facing the back-side of the mesh which should never be seen because the front-side is hiding it and the object is supposed to be opaque and solid.

• one thing tell in linked website the coefficient are positive and books coefficient are negative? How could when A, B, C, D are positive or negative if I seeing Negetive z direction? Dec 4 '21 at 5:01
• @Ponting I'm sorry, I am not familiar enough with this particular math to tell you how to do it properly. All I can say is that the < and > versions are not necessarily contradictory — just set up differently. Maybe one of them has the Z axis running the other direction (left-handed vs. right-handed coordinate systems), for example. Dec 4 '21 at 5:49
• when negative z direction A, B, C are negative? Dec 4 '21 at 6:01
• I have one confusion. If I say Ax+By+Cz+D>0 then (x, y, z) is the outside of the plane and then (−A)x+(−B)y+(−C)z+(−D)<0 indicates (x, y, z) inside of the plane ? Dec 4 '21 at 6:07
• As Kevin Reid has told you, the convention of whether we call > 0 "inside" or "outside" is completely arbitrary. There is no one true set of terminology to use here. One site might say > 0 is inside and < 0 is outside, while another equally correct source can say > 0 is outside and < 0 is inside. They're both right, because the universe doesn't define these terms, we do. All that matters is that within a single project we pick one definition and use it consistently for all the math we do in that project, so that the answers we get for each task we do in that project agree with one another. Dec 4 '21 at 11:51