What is a stencil buffer?
A stencil buffer is similar to a depth buffer. In fact, it uses part of the depth buffer (for this reason, the depth buffer is often called the depth-stencil buffer). The stencil buffer allows the programmer to set a stencil function that will test the “reference” stencil value – a global value – against the value already in the stencil buffer each time a pixel is rendered. The outcome of that stencil test determines if the color value of the pixel is written to the render target, and if the depth value of that pixel is written to the depth buffer. For example, if you render some objects to a scene while the reference stencil is 0, and the stencil buffer has been cleared to 1, the stencil buffer will contain a cut-out pattern of zeros where those objects were rendered. If the reference value was then set to 1, and the StencilFunction was set to CompareFunction.LessEqual, the only pixels that could be rendered to the scene are those that are not in the same location where the stencil va
A stencil buffer is an additional buffer of per-pixel information, much like a z-buffer. In fact, it resides in some of the bits of a z-buffer. Common stencil/z-buffer formats are 15-bit z and 1-bit stencil, or 24-bit z and 8-bit stencil. It is possible to perform simple arithmetic operations on the contents of the stencil buffer on a per-pixel basis as polygons are rendered. For example, the stencil buffer can be incremented or decremented, or the pixel can be rejected if the stencil value fails a simple comparison test. This is useful for effects that involve marking out a region of the frame buffer and then performing rendering only the marked (or unmarked) region. Good examples are volumetric effects like shadow volumes.
