Azure Pipelines: Running .NET Unit Tests

This guide explains how to integrate .NET unit tests into your Azure Pipelines to ensure code quality and stability. We'll cover setting up the build pipeline to discover and execute your tests.

Prerequisites

An Azure DevOps organization and project.
A Git repository with your .NET project containing unit tests.
Azure Pipelines configured for your repository.

Understanding .NET Test Frameworks

.NET supports several popular unit testing frameworks, including:

MSTest: The default framework for Visual Studio.
NUnit: A widely used and mature testing framework.
xUnit.net: A modern, extensible testing framework.

Azure Pipelines can execute tests from any of these frameworks using the appropriate task.

Configuring Your Pipeline (YAML)

We'll use a YAML pipeline to define the build and test stages. Here's a typical structure:


trigger:
- main

pool:
  vmImage: 'windows-latest' # Or 'ubuntu-latest', 'macos-latest'

steps:
- task: UseDotNet@2
  displayName: 'Use .NET SDK'
  inputs:
    packageType: 'sdk'
    version: '6.x' # Specify your .NET SDK version

- task: DotNetCoreCLI@2
  displayName: 'Restore Dependencies'
  inputs:
    command: 'restore'
    projects: '**/*.sln'

- task: DotNetCoreCLI@2
  displayName: 'Build Project'
  inputs:
    command: 'build'
    projects: '**/*.sln'
    arguments: '--configuration Release'

- task: DotNetCoreCLI@2
  displayName: 'Run Unit Tests'
  inputs:
    command: 'test'
    projects: '**/*[Tt]ests/*.csproj' # Adjust path if your test projects are elsewhere
    arguments: '--configuration Release --logger trx --results-directory $(Agent.TempDirectory)'

- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: 'VSTest' # Or 'trx' if your logger produces trx files
    testResultsFiles: '$(Agent.TempDirectory)/**/*.trx'
    mergeTestResults: true
    failTaskOnFailedTests: true

Explanation of the YAML Steps

1. Trigger

This section defines when the pipeline should run. In this case, it triggers on commits to the main branch.


trigger:
- main

2. Pool

Specifies the agent pool to use for running the job. vmImage: 'windows-latest' uses a Microsoft-hosted Windows agent.


pool:
  vmImage: 'windows-latest'

3. Use .NET SDK

The UseDotNet@2 task ensures the correct .NET SDK version is available on the agent.


- task: UseDotNet@2
  displayName: 'Use .NET SDK'
  inputs:
    packageType: 'sdk'
    version: '6.x'

4. Restore Dependencies

The DotNetCoreCLI@2 task with the restore command downloads all necessary NuGet packages.


- task: DotNetCoreCLI@2
  displayName: 'Restore Dependencies'
  inputs:
    command: 'restore'
    projects: '**/*.sln'

5. Build Project

The build command compiles your .NET solution.


- task: DotNetCoreCLI@2
  displayName: 'Build Project'
  inputs:
    command: 'build'
    projects: '**/*.sln'
    arguments: '--configuration Release'

6. Run Unit Tests

This is the core step for executing tests. The test command finds and runs your unit tests.

projects: '**/*[Tt]ests/*.csproj': This pattern targets project files ending with "Tests.csproj" or "tests.csproj" in any directory. Adjust this if your test projects have different naming conventions or locations.
arguments: '--logger trx --results-directory $(Agent.TempDirectory)':
- --logger trx: Configures the test runner to output results in the TRX format, which is compatible with the PublishTestResults task.
- --results-directory $(Agent.TempDirectory): Specifies where to save the test results files. $(Agent.TempDirectory) is a predefined variable that points to a temporary directory on the agent.


- task: DotNetCoreCLI@2
  displayName: 'Run Unit Tests'
  inputs:
    command: 'test'
    projects: '**/*[Tt]ests/*.csproj'
    arguments: '--configuration Release --logger trx --results-directory $(Agent.TempDirectory)'

7. Publish Test Results

The PublishTestResults@2 task uploads the test results to Azure Pipelines, making them visible in the build summary and test results tab.

testResultsFormat: 'VSTest': Indicates the format of the test results. If you used --logger trx, VSTest is the correct format.
testResultsFiles: '$(Agent.TempDirectory)/**/*.trx': Specifies the files to publish.
failTaskOnFailedTests: true: Configures the pipeline to fail if any tests do not pass.


- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: 'VSTest'
    testResultsFiles: '$(Agent.TempDirectory)/**/*.trx'
    mergeTestResults: true
    failTaskOnFailedTests: true

Running Tests with Different Frameworks

The DotNetCoreCLI@2 task with the test command is framework-agnostic. As long as your test projects are configured correctly for MSTest, NUnit, or xUnit.net, this task will discover and run them.

Example: NUnit Configuration

Ensure you have the NUnit NuGet packages installed in your test project:


dotnet add package NUnit
dotnet add package NUnit3TestAdapter

Example: xUnit.net Configuration

Ensure you have the xUnit.net NuGet packages installed:


dotnet add package xunit
dotnet add package xunit.runner.visualstudio

Viewing Test Results

After a successful pipeline run, you can view detailed test results:

Navigate to your build summary in Azure DevOps.
Click on the "Tests" tab.
Here you'll see a summary of test runs, including passed, failed, and skipped tests. You can drill down into individual tests for more information.

Important: Ensure your .csproj file for the test project correctly references the test framework and the project(s) under test.

For example, a test project's .csproj might look like this:


<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.5.0" />
    <PackageReference Include="xunit" Version="2.4.2" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" />
    <PackageReference Include="coverlet.collector" Version="3.2.0" />
    <ProjectReference Include="..\..\src\MyApplication\MyApplication.csproj" />
  </ItemGroup>

</Project>

Advanced Scenarios

Code Coverage

To get code coverage reports, you can add the coverlet.collector NuGet package to your test projects and include the --collect "Code coverage" argument in your dotnet test command.


- task: DotNetCoreCLI@2
  displayName: 'Run Unit Tests with Coverage'
  inputs:
    command: 'test'
    projects: '**/*[Tt]ests/*.csproj'
    arguments: '--configuration Release --logger trx --results-directory $(Agent.TempDirectory) --collect "Code coverage"'

- task: PublishCodeCoverageResults@1
  displayName: 'Publish Code Coverage Results'
  inputs:
    codeCoverageTool: 'Cobertura' # Or 'JaCoCo' depending on the collector
    summaryFileLocation: '$(Agent.TempDirectory)/**/*.cobertura.xml' # Path might vary

Ensure you have the coverlet.collector package installed in your test project for this to work.

Test Filtering

You can filter which tests are run using the --filter argument.


dotnet test --filter "Category=Integration"
dotnet test --filter "FullyQualifiedName~MyNamespace.MyClass.MyTestMethod"

You can pass these filters via the arguments field in your YAML task.

Troubleshooting

Tests not found: Double-check the projects pattern in the DotNetCoreCLI@2 task to ensure it correctly targets your test project files.
Test execution errors: Examine the build logs carefully. Errors might occur due to missing dependencies, incorrect build configurations, or issues within the tests themselves.
Publishing issues: Verify that the testResultsFiles pattern in the PublishTestResults@2 task correctly matches the output files generated by the test runner.