More on Matrices - 3D Math Primer for Graphics and Game Development - page 174

• Each row of the matrix must be a unit vector.

• The rows of the matrix must be mutually perpendicular.

Similar statements can be made regarding the columns of the matrix, since
if M is orthogonal, then M T must be orthogonal as well.
Notice that these criteria are precisely those that we said in Section 3.3.3
were satisfied by an orthonormal set of basis vectors. In that section, we also
noted that an orthonormal basis was particularly useful because we could
perform, by using the dot product, the “opposite” coordinate transform
from the one that is always available. When we say that the transpose of
an orthogonal matrix is equal to its inverse, we are just restating this fact
in the formal language of linear algebra.
Also notice that three of the orthogonality equations are duplicates, be-
cause the dot product is commutative. Thus, these nine equations actually
express only six constraints. In an arbitrary 3 × 3 matrix there are nine
elements and thus nine degrees of freedom, but in an orthogonal matrix, six
degrees of freedom are removed by the constraints, leaving three degrees of
freedom. It is significant that three is also the number of degrees of free-
dom inherent in 3D rotation. (However, rotation matrices cannot contain
a reflection, so there is “slightly more freedom” in the set of orthogonal
matrices than in the set of orientations in 3D.)
When computing a matrix inverse, we will usually only take advantage
of orthogonality if we know a priori that a matrix is orthogonal. If we don't
know in advance, it's probably a waste of time to check. In the best case,
we check for orthogonality and find that the matrix is indeed orthogonal,
and then we transpose the matrix. But this may take almost as much time
as doing the inversion. In the worst case, the matrix is not orthogonal, and
any time we spent checking was definitely wasted. Finally, even matrices
that are orthogonal in the abstract may not be exactly orthogonal when
represented in floating point, and so we must use tolerances, which have to
be tuned.

One important note is needed here on terminology that can be slightly
confusing. In linear algebra, we describe a set of basis vectors as orthogonal
if they are mutually perpendicular. It is not required that they have unit
length. If they do have unit length, they are an orthonormal basis. Thus the
rows and columns of an orthogonal matrix are orthonormal basis vectors.
However, constructing a matrix from a set of orthogonal basis vectors does
not necessarily result in an orthogonal matrix (unless the basis vectors are
also orthonormal).