Mathematical Prerequisites - Introduction to 3D Game Programming with DirectX 9.0 - page 29

Thus, we have the following formula to normalize the normal vec-
tor of the plane ( n , d ):

1

n d

, ,

n d

n n n

We can use the following D3DX function to normalize a plane's normal
vector:
D3DXPLANE *D3DXPlaneNormalize(
D3DXPLANE *pOut, // Resulting normalized plane.

CONST D3DXPLANE *pP // Input plane.

);

Transforming a Plane
Lengyel shows in Mathematics for 3D Game Programming & Computer
Graphics that we can transform a plane ( n , d ) by treating it as a 4D vec-
tor and multiplying it by the inverse-transpose of the desired transfor-
mation matrix. Note that the plane's normal vector must be normalized
first.
We use the following D3DX function to do this:
D3DXPLANE *D3DXPlaneTransform(
D3DXPLANE *pOut,

// Result

CONST D3DXPLANE *pP, // Input plane.

CONST D3DXMATRIX *pM // Transformation matrix.

);
Sample code:
D3DXMATRIX T(...); // Init. T to a desired transformation.

D3DXMATRIX inverseOfT;
D3DXMATRIX inverseTransposeOfT;

D3DXMatrixInverse( &inverseOfT, 0, &T );
D3DXMatrixTranspose( &inverseTransposeOfT, &inverseOfT );

D3DXPLANE p(...); // Init. Plane.
D3DXPlaneNormalize( &p, &p ); // make sure normal is normalized.

D3DXPlaneTransform( &p, &p, &inverseTransposeOfT );

Nearest Point on a Plane to a Particular Point
Suppose that we have a point p in space and would like to find the point

q on the plane ( n , d ) that is closest to p . Note that the plane's normal

vector is assumed to be of unit length—this simplifies the problem a
bit.