Geometric Primitives - 3D Math Primer for Graphics and Game Development - page 314

We will construct two vectors according to the clockwise ordering. The
notation e stands for “edge” vector, since these equations commonly arise
when computing the plane equation for a triangle. (The indexing may
seem weird, but bear with us—this is the indexing we'll use in Section 9.6.1
where triangles are discussed in more detail.) The cross product of these two
vectors yields the perpendicular vector n , but this vector is not necessarily
of unit length. As mentioned earlier, we usually normalize n . All of this is
summarized succinctly by

e 3 × e 1
e 3

The normal of a plane
containing three points

e 3 = p 2 − p 1 , e 1 = p 3 − p 2 , n =

.

(9.12)

× e 1

Notice that if the points are collinear, then e 3 and e 1 will be parallel, and
thus the cross product will be 0 , which cannot be normalized. This math-
ematical singularity coincides with the physical singularity that collinear
points do not unambiguously define a plane.
Now that we know n , all that is left to do is compute d. This is easily
done by taking the dot product of one of the points and n .

9.5.3 “Best Fit” Plane for More than Three Points
Occasionally, we may wish to compute the plane equation for a set of more
than three points. The most common example of such a set of points
is the vertices of a polygon. In this case, the vertices are assumed to
be enumerated in a clockwise fashion around the polygon. (The ordering
matters because it is how we decide which side is the front and which is the
back, which in turn determines which direction our normal will point.)
One naıve solution is to arbitrarily select three consecutive points and
compute the plane equation from those three points. However, the three
points we choose might be collinear, or nearly collinear, which is almost
as bad because it is numerically inaccurate. Or perhaps the polygon is
concave and the three points we have chosen are a point of concavity and
therefore form a counterclockwise turn (which would result in a normal
that points in the wrong direction). Or the vertices of the polygon may not
be coplanar, which can happen due to numeric imprecision or the method
used to generate the polygon. What we really want is a way to compute
the “best fit” plane for a set of points that takes into account all of the
points. Given n points,

p 1 = x 1 y 1 z 1 ,

p 2 = x 2 y 2 z 2 ,

.