Geometry Shader Concepts

The geometry shader is a programmable stage in the DirectX graphics pipeline that sits between the vertex shader and the rasterizer. Unlike the vertex shader, which processes one vertex at a time, the geometry shader can process entire primitives (points, lines, triangles) and can generate new primitives or discard existing ones. This flexibility allows for advanced graphical effects such as tessellation, procedural geometry generation, and billboarding.

Purpose and Capabilities

The primary purpose of the geometry shader is to take one or more vertices of a primitive as input and output zero or more primitives. It operates on primitives one at a time, meaning it receives all vertices belonging to a single primitive (e.g., three vertices for a triangle) and can output any number of primitives of any type. This makes it distinct from the vertex shader, which operates on a per-vertex basis and outputs a single vertex for each input vertex.

Key Capabilities:

Primitive Generation: The geometry shader can create new primitives, such as generating two triangles from a single line segment to form a quadrilateral for billboarding effects.
Primitive Decomposition: It can break down larger primitives into smaller ones, for example, splitting a triangle into three smaller triangles.
Primitive Modification: Vertices can be transformed, and new vertices can be added or removed.
Primitive Culling: Primitives can be discarded entirely, preventing them from being processed by subsequent stages.
Instancing: While not its primary purpose, geometry shaders can be used to implement hardware instancing-like effects by generating multiple copies of a primitive.

Input and Output

The geometry shader receives input primitives from the rasterizer-discard (RD) stage (typically after the vertex shader and tessellation stages). The input primitive type can be a point, line, triangle, or even larger primitives depending on the API version and shader model.

The geometry shader outputs a stream of vertices, which are then assembled into primitives by the Primitive Assembler stage. The output primitive type must be explicitly declared.

Common Primitive Types:

Input: point, line, triangle, lineadj (lines with adjacency information), triangleadj (triangles with adjacency information).
Output: point, line, triangle, linestrip, tristrip.

Adjacency information (lineadj and triangleadj) provides neighboring vertices, which can be useful for algorithms like edge-detection or mesh simplification.

Shader Model Support

The geometry shader was introduced with Shader Model 4.0. Later versions, such as Shader Model 5.0 and beyond, have introduced improvements and new features.

Example: Generating a Quad from a Point

A common use case is to generate a billboard quad from a single point. The geometry shader receives the point primitive and outputs a triangle strip that forms a quad oriented towards the camera.

#include <d3d11_4.h>

// Input structure matching the vertex shader output
struct GS_INPUT {
    float4 pos : SV_POSITION;
    float2 tex : TEXCOORD;
};

// Output structure for the geometry shader
struct GS_OUTPUT {
    float4 pos : SV_POSITION;
    float2 tex : TEXCOORD;
};

GS_OUTPUT output_vertex(GS_INPUT input) {
    GS_OUTPUT output;
    output.pos = input.pos;
    output.tex = input.tex;
    return output;
}

[maxvertexcount(4)] // Output a quadrilateral (4 vertices)
void GeometryShaderMain(triangle GS_INPUT input[3] : SV_GeometryShader) {
    // In a real scenario, you'd calculate billboard vertices here.
    // For simplicity, we'll just pass through the first vertex's position.
    // A more complete example would involve camera-to-vertex vector math.

    GS_INPUT centerVertex = input[0];

    // Example: Create a simple quad, not camera-aligned for brevity.
    GS_OUTPUT v0, v1, v2, v3;

    v0.pos = centerVertex.pos + float4(-0.5f, -0.5f, 0.0f, 0.0f);
    v0.tex = float2(0.0f, 1.0f);
    output_vertex(v0);

    v1.pos = centerVertex.pos + float4(0.5f, -0.5f, 0.0f, 0.0f);
    v1.tex = float2(1.0f, 1.0f);
    output_vertex(v1);

    v2.pos = centerVertex.pos + float4(-0.5f, 0.5f, 0.0f, 0.0f);
    v2.tex = float2(0.0f, 0.0f);
    output_vertex(v2);

    v3.pos = centerVertex.pos + float4(0.5f, 0.5f, 0.0f, 0.0f);
    v3.tex = float2(1.0f, 0.0f);
    output_vertex(v3);
}

Performance Considerations

While powerful, geometry shaders can be a performance bottleneck if not used judiciously. Each primitive processed by the geometry shader has its vertices processed by the vertex shader first. If the geometry shader generates a large number of primitives or complex geometry, it can significantly increase the workload. In many cases, effects that can be achieved with geometry shaders might be more efficiently implemented using other techniques, such as vertex shader tricks, tessellation, or compute shaders, especially in modern graphics APIs.

For simple instancing or generating a few extra triangles, geometry shaders can be very useful. For complex procedural generation, consider tessellation or compute shaders.

When to Use a Geometry Shader

Consider using a geometry shader for:

Generating particle systems: Creating geometry for particles on the GPU.
Procedural geometry: Generating shapes or details dynamically.
Billboarding: Always facing sprites or billboards towards the camera.
Mesh expansion/contraction: Modifying the shape of existing geometry.
Rendering multiple views: For techniques like shadow mapping or cubemaps (though often specialized techniques exist).

For many common tasks like simple instancing or drawing basic primitives, alternative methods might offer better performance on modern hardware.