Vertex Shading

Vertex shaders are programmable stages in the DirectX rendering pipeline that operate on individual vertices. They are responsible for transforming vertex data from object-local space to screen space and can also perform per-vertex computations like lighting and color calculations. This stage is crucial for defining the shape, position, and initial per-vertex attributes of 3D models.

Purpose of Vertex Shading

The primary role of a vertex shader is to take the input vertex data (position, normals, texture coordinates, colors, etc.) and output transformed vertex data that can be used in subsequent stages of the pipeline. Key tasks performed by vertex shaders include:

Vertex Transformation: Applying model, view, and projection matrices to transform vertices from object space to clip space.
Per-Vertex Lighting: Calculating lighting contributions for each vertex based on its normal, light sources, and material properties.
Texture Coordinate Generation/Manipulation: Generating or modifying texture coordinates for texture mapping.
Color Interpolation: Determining per-vertex colors which will be interpolated across the primitive.
Data Streaming: Outputting processed vertex data to the next stage, often the geometry shader or rasterizer.

HLSL for Vertex Shaders

High-Level Shading Language (HLSL) is used to write DirectX shaders. A typical vertex shader function in HLSL has a specific signature and uses predefined structures for input and output.

Input and Output Structures

Vertex shaders receive input from vertex buffers and output data for the rasterizer. Common input and output structures include:

struct VS_INPUT
{
    float4 position : POSITION;     // Object-space vertex position
    float3 normal : NORMAL;         // Object-space vertex normal
    float2 texCoord : TEXCOORD0;    // Texture coordinates
    float4 color : COLOR0;          // Vertex color
};

struct VS_OUTPUT
{
    float4 position : SV_POSITION;  // Clip-space vertex position (required)
    float3 normal : NORMAL;         // World-space normal (for lighting)
    float2 texCoord : TEXCOORD0;    // Texture coordinates (for pixel shader)
    float4 color : COLOR0;          // Interpolated color
};

A Simple Vertex Shader Example

This example demonstrates a basic vertex shader that transforms the vertex position, passes through texture coordinates, and applies a simple lighting calculation based on the normal and a directional light.

// Global variables (uniforms)
float4x4 worldMatrix : WORLD;
float4x4 viewProjectionMatrix : VIEWPROJECTION;
float3 lightDirection : LIGHTDIRECTION;
float4 lightColor : LIGHTCOLOR;
float4 ambientColor : AMBIENTCOLOR;

VS_OUTPUT VertexShaderFunction(VS_INPUT input)
{
    VS_OUTPUT output;

    // Transform position from object space to clip space
    output.position = mul(float4(input.position.xyz, 1.0), worldMatrix);
    output.position = mul(output.position, viewProjectionMatrix);

    // Transform normal to world space (for lighting)
    // Assuming worldMatrix contains only rotation and translation, not scaling for normals
    output.normal = normalize(mul(input.normal, (float3x3)worldMatrix));

    // Pass through texture coordinates
    output.texCoord = input.texCoord;

    // Simple per-vertex lighting calculation
    // Diffuse lighting: lightColor * max(0, dot(normal, -lightDirection))
    float diffuseFactor = max(0.0f, dot(normalize(output.normal), -normalize(lightDirection)));
    output.color = input.color * (ambientColor + lightColor * diffuseFactor);

    return output;
}

Key Concepts and Considerations

Matrices: Understanding the model, view, and projection matrices is fundamental. The vertex shader combines these to project vertices onto the screen.
Shader Model: DirectX supports different shader models (e.g., Shader Model 4.0, 5.0) which dictate the features and capabilities available to shaders.
Input Semantics: Attributes like POSITION, NORMAL, TEXCOORD, and SV_POSITION are crucial for binding data and defining output.
Performance: Vertex shaders can be a bottleneck if not optimized. Offloading complex calculations to the CPU or using hardware tessellation can be beneficial.
Data Interpolation: Data output by the vertex shader (like color and texture coordinates) is interpolated across the face of triangles by the rasterizer, providing smooth gradients in the pixel shader.

Tip: Vertex shaders are ideal for operations that need to be performed once per vertex. For operations that need to be performed per-pixel or per-fragment, pixel shaders are more appropriate.

Advanced Vertex Shading Techniques

Beyond basic transformations and lighting, vertex shaders can be used for more advanced effects:

Skeletal Animation: Deforming meshes based on bone transformations.
Procedural Geometry Generation: Generating vertices on the fly, though often better suited for geometry shaders.
Vertex displacement: Modifying vertex positions based on textures or other data.

Mastering vertex shaders is a key step in developing sophisticated 3D graphics applications with DirectX.