Tessellation Shaders - DirectX Programming Guides

Tessellation Shaders in DirectX

Tessellation shaders provide a powerful mechanism to dynamically subdivide geometric primitives in real-time, enabling significant enhancements in visual fidelity and computational efficiency. This guide explores the core concepts and implementation details of tessellation in DirectX.

Introduction to Tessellation

Tessellation allows you to take a coarse mesh and refine it into a much denser one on the GPU. This is particularly useful for techniques like:

Level of Detail (LOD): Automatically generate more detail for objects closer to the camera.
Terrain Generation: Create complex, detailed terrain from simpler heightmaps.
Subdivision Surfaces: Implement smooth, curved surfaces using low-polygon control meshes.
Displacement Mapping: Add intricate surface detail by displacing vertices based on a texture.

The Tessellation Pipeline Stages

Tessellation introduces two new shader stages between the hull shader and the domain shader:

Tessellator Stage: This fixed-function stage takes the output from the hull shader and generates the tessellation factors. These factors determine how many new vertices are created for each primitive.
Domain Shader: This programmable stage processes the newly generated vertices. It receives the tessellation factors and the interpolated data from the hull shader and outputs the final, refined geometry.

Hull Shader

The hull shader is responsible for preparing data for the tessellator and the domain shader. It:

Outputs tessellation factors, which can be constant or vary across the primitive.
Outputs patch constants, which are constant per patch and passed to the domain shader.
Can take input from the geometry shader or vertex shader.

A basic hull shader might look like this:

// Patch constant function
struct HS_CONTROL_POINT_OUTPUT_CS
{
    float TessFactor[3] : tessfactor; // Edge tessellation factors (U, V, Width)
};

struct HS_PATCH_CONSTANT_OUTPUT_PC
{
    float EdgeTessFactor[3] : tessfactor; // tessfactor(0..2)
    float InsideTessFactor : tessfactor_inner; // tessfactor_inner
    float3      ScreenSpaceNormal : SCREENSPACENORMAL;
};

HS_PATCH_CONSTANT_OUTPUT_PC main_cs(InputPatch patch, uint patchID : SV_PrimitiveID)
{
    HS_PATCH_CONSTANT_OUTPUT_PC output = (HS_PATCH_CONSTANT_OUTPUT_PC)0.0f;

    // Example: Constant tessellation factors
    output.EdgeTessFactor[0] = 4.0f; // Edge 0 (between vertex 0 and 1)
    output.EdgeTessFactor[1] = 4.0f; // Edge 1 (between vertex 1 and 2)
    output.EdgeTessFactor[2] = 4.0f; // Edge 2 (between vertex 2 and 0)
    output.InsideTessFactor = 4.0f; // Inner tessellation factor

    // Compute screen-space normal for potential per-patch manipulation
    // ... (implementation details)

    return output;
}

// Hull shader main function
struct HS_OUTPUT_HS
{
    float4 Position : SV_Position;
    float2 Tex : TEXCOORD;
};

[domain("tri")] // or "quad", "isoline"
[partitioning("fractional_even")] // or "fractional_odd", "integer", "pow2"
[outputtopology("triangle_cw")] // or "triangle_ccw"
[patchconstantfunc("main_cs")]
[outputcontrolpoints(3)] // Number of control points
HS_OUTPUT_HS main_hs(InputPatch patch, uint cpID : SV_ControlPointID)
{
    HS_OUTPUT_HS output = (HS_OUTPUT_HS)0.0f;
    output.Position = patch[cpID].Position;
    output.Tex = patch[cpID].Tex;
    return output;
}

Domain Shader

The domain shader is where the actual vertex generation and manipulation happens. For each generated vertex:

It receives the interpolated tessellation factors (U, V coordinates for triangles/quads) and patch constants.
It outputs the final vertex position and other attributes for the rasterizer.

A basic domain shader might look like this:

struct DS_INPUT
{
    float3 domainOrigin : SV_DomainOrigin; // For tessellation factors (U, V)
    float3 domainEdgeTessFactor : SV_TessFactor; // For edge tessellation factors
    float3 screenSpaceNormal : SCREENSPACENORMAL; // From patch constant function
};

struct DS_OUTPUT
{
    float4 Position : SV_Position;
    float2 Tex : TEXCOORD;
};

[domain("tri")] // Must match hull shader
DS_OUTPUT main_ds(HS_PATCH_CONSTANT_OUTPUT_PC input, float3 uvw : SV_DomainLocation, const OutputPatch patch)
{
    DS_OUTPUT output;

    // Interpolate vertex positions based on barycentric coordinates (uvw)
    float3 pos0 = patch[0].Position.xyz / patch[0].Position.w;
    float3 pos1 = patch[1].Position.xyz / patch[1].Position.w;
    float3 pos2 = patch[2].Position.xyz / patch[2].Position.w;

    float3 interpolatedPos = pos0 * uvw.x + pos1 * uvw.y + pos2 * uvw.z;

    // Example: Simple displacement based on UV coordinates
    // float displacement = tex2D(DisplacementMapSampler, patch[0].Tex).r; // If UVs are consistent
    // interpolatedPos.y += displacement * displacementScale;

    output.Position = float4(interpolatedPos, 1.0f);

    // Interpolate texture coordinates
    output.Tex = patch[0].Tex * uvw.x + patch[1].Tex * uvw.y + patch[2].Tex * uvw.z;

    // Transform to clip space (if needed, often handled by the vertex shader output)
    // output.Position = mul(output.Position, ViewProjectionMatrix);

    return output;
}

Key Concepts and Considerations