Stenciling - Introduction to 3D Game Programming with DirectX 9.0 - page 134

A left hand side operand ( LHS = ref&mask ) that is determined by
AND ing an application-defined stencil reference value ( ref ) with an
application-defined masking value ( mask ).
A right hand side ( RHS = value&mask ) that is determined by
AND ing the entry in the stencil buffer for the particular pixel that
we are testing ( value ) with an application-defined masking value
( mask ).
The stencil test then compares the LHS with the RHS, as specified by
the comparison operation . The entire expression evaluates to a Boolean
(true or false) value. We write the pixel to the back buffer if the test
evaluates to true (passes). If the test evaluates to false (fails), we block
the pixel from being written to the back buffer. And of course, if a pixel
isn't written to the back buffer, it isn't written to the depth buffer
either.
8.1.3 Controlling the Stencil Test
To give us flexibility, Direct3D allows us to control the variables used in
the stencil test. In other words, we get to specify the stencil reference
value, the mask value, and even the comparison operation. Though we
do not get to explicitly set the stencil value, we do have some control
over what values get written to the stencil buffer (in addition to clear-
ing the stencil buffer).
8.1.3.1 Stencil Reference Value
The stencil reference value ref is zero by default, but we can change it
with the D3DRS_STENCILREF render state. For example, the following
code sets the stencil reference value to one:
Device->SetRenderState(D3DRS_STENCILREF, 0x1);
Note that we tend to use hexadecimal because it makes it easier to see
the bit alignment of an integer, and this is useful to see when doing
bit-wise operations, such as ANDing.
8.1.3.2 Stencil Mask
The stencil masking value mask is used to mask (hide) bits in both the
ref and value variables. The default mask is 0xffffffff , which
doesn't mask any bits. We can change the mask by setting the
D3DRS_STENCILMASK render state. The following example masks the
16 high bits:
Device->SetRenderState(D3DRS_STENCILMASK, 0x0000ffff);
Team-Fly ®