How do I calculate the normal of a plane defined using homogeneous coordinates?

Question

I can define a plane in 3D space using three 3D points (p0, p1, p2) that all lie in the plane and that form a non-degenerate triangle. Calculating the normal of the plane is as simple as calculating the cross product of any two non-parallel 3D vectors that lie in the plane, for instance (p1 - p0) and (p2 - p0). So far, so good.

But what if I use homogeneous coordinates for my points? How do I find the normal of the plane then? Of course, if all points have w != 0, I could simply normalize them (divide by w) and treat them as regular 3D points. But what if one or more points have w == 0?

The way I see it, there are three possible answers to this question:

1. You do it the same way. If so, please explain how to calculate the cross product of two 3D vectors defined using homogeneous coordinates.
2. You do it some other way. If so, please explain how.
3. You can't, your question makes no sense and you are clearly confused. If so, please explain why.

Answer 1

You do it the same way, basically. You just need to get rid of the divisions by zero.

First of all, there is no solution if w0=w1=w2=0. This degenerate case should be handled as if all points were aligned (because in a sense, they are, at infinity).

Let us assume, without loss of generality, that w0!=0. If not, just swap p0 and p1 for instance.

Here is how you would do it when no w is zero: divide the 3-dimensional points by their respective w value, then take a cross-product of the vectors you mention:

     |x0|         |x1|         |x2|
p0 = |y0|    p1 = |y1|    p2 = |y2|
     |z0|         |z1|         |z2|

q0 = p0/w0   q1 = p1/w1   q2 = p2/w2

v = (q2 - q0) × (q1 - q0)
  = (p2/w2 - p0/w0) × (p1/w1 - p0/w0)
  = 1/(w2*w1) (p2 - p0 w2/w0) × (p1 - p0 w1/w0)

But wait, if this vector v is normal to the plane, then obviously v multiplied by w1*w2 is still normal to the plane! So here is another vector that works:

v' = (p2 - w2/w0 p0) × (p1 - w1/w0 p0)

This one involves no division by zero and is therefore safe to take to the w1=0, w2=0 or w1=w2=0 limits.