# How do I calculate the normal of a plane defined using homogeneous coordinates?

I can define a plane in 3D space using three 3D points (p0, p1, p2) that all lie in the plane and that form a non-degenerate triangle. Calculating the normal of the plane is as simple as calculating the cross product of any two non-parallel 3D vectors that lie in the plane, for instance (p1 - p0) and (p2 - p0). So far, so good.

But what if I use homogeneous coordinates for my points? How do I find the normal of the plane then? Of course, if all points have w != 0, I could simply normalize them (divide by w) and treat them as regular 3D points. But what if one or more points have w == 0?

The way I see it, there are three possible answers to this question:

1. You do it the same way. If so, please explain how to calculate the cross product of two 3D vectors defined using homogeneous coordinates.
2. You do it some other way. If so, please explain how.
3. You can't, your question makes no sense and you are clearly confused. If so, please explain why.

You do it the same way, basically. You just need to get rid of the divisions by zero.

First of all, there is no solution if w0=w1=w2=0. This degenerate case should be handled as if all points were aligned (because in a sense, they are, at infinity).

Let us assume, without loss of generality, that w0!=0. If not, just swap p0 and p1 for instance.

Here is how you would do it when no w is zero: divide the 3-dimensional points by their respective w value, then take a cross-product of the vectors you mention:

     |x0|         |x1|         |x2|
p0 = |y0|    p1 = |y1|    p2 = |y2|
|z0|         |z1|         |z2|

q0 = p0/w0   q1 = p1/w1   q2 = p2/w2

v = (q2 - q0) × (q1 - q0)
= (p2/w2 - p0/w0) × (p1/w1 - p0/w0)
= 1/(w2*w1) (p2 - p0 w2/w0) × (p1 - p0 w1/w0)


But wait, if this vector v is normal to the plane, then obviously v multiplied by w1*w2 is still normal to the plane! So here is another vector that works:

v' = (p2 - w2/w0 p0) × (p1 - w1/w0 p0)


This one involves no division by zero and is therefore safe to take to the w1=0, w2=0 or w1=w2=0 limits.

• Thank you for writing out all the steps in the derivation of v', it really makes it easier to understand. – sarnesjo Apr 18 '12 at 5:25
• However, I don't understand why there is no solution when w0 == w1 == w2 == 0. I understand why the math doesn't check out, because of the division by zero, but I thought that a plane defined by the points (1,0,0,0), (0,1,0,0), (0,0,1,0) (using order (x,y,z,w)) would have the same normal as one defined by (1,0,0,1), (0,1,0,1), (0,0,1,1). Perhaps my intuition about a point at infinity is incorrect? Can you help me understand? – sarnesjo Apr 18 '12 at 5:32
• @sarnesjo it is true that if you take points (1,0,0,w), (0,1,0,w), (0,0,1,w) with w>0 then the normal vector is (1,1,1) and it doesn't appear to be affected by the value of w so one may think pushing w to the limit 0 will yield a normal vector of (1,1,1). However it is illegal to calculate the limit of a three-variable function by fixing a relationship between the variables. – sam hocevar Apr 18 '12 at 6:35
• @sarnesjo take for instance points (1,0,0,w), (0,1,0,w), (0,0,1,2w). At w=0 they become the same points as before. However, their normal vector is now (1/2,1/2,1). You can see there is an inconsistency here! It's because the limit of a multivariate function is only valid if it is not affected by the “path” the different variables take. There is another example on Wikipedia. – sam hocevar Apr 18 '12 at 6:46