Azure DevOps Pipelines: Automating Postman API Tests

Prepare Your Postman Collection

Ensure your Postman collection is well-structured and contains all the API tests you wish to automate. Assign meaningful names to your requests and tests.

Export Your Postman Collection and Environment

Export your Postman collection as a JSON file. If you use Postman environments for variables, export your environment as well.

You can do this from Postman by clicking the three dots next to your collection or environment and selecting 'Export'.

Install Newman

Newman is a command-line collection runner for Postman. It allows you to run your Postman collections from your terminal or, in this case, your Azure DevOps pipeline agent.

You can install Newman globally using npm:

npm install -g newman

Configure Azure DevOps Pipeline

In your Azure DevOps project, navigate to 'Pipelines' and create or edit your YAML pipeline definition.

You'll need to add tasks to:

• Download your Postman collection and environment files (if they are stored as pipeline artifacts or in a repository).
• Install Node.js and Newman on the agent.
• Run Newman with your collection and environment.
• Publish test results.

Add Newman Task to Pipeline

Here's a sample YAML snippet for running Postman tests using Newman:

- task: NodeTool@0
  inputs:
    versionSpec: '16.x' # Specify your desired Node.js version
  displayName: 'Use Node.js 16.x'

- script: npm install -g newman
  displayName: 'Install Newman'

- task: DownloadSecureFile@1
  name: postmanCollection
  inputs:
    secureFile: 'your-api-collection.json' # Name of your secure file in Azure DevOps
  displayName: 'Download Postman Collection'

# Optional: Download environment file if used
#- task: DownloadSecureFile@1
#  name: postmanEnvironment
#  inputs:
#    secureFile: 'your-api-environment.json' # Name of your secure file
#  displayName: 'Download Postman Environment'

- script: |
    newman run $(postmanCollection.secureFilePath) --reporters cli junit --reporter-junit-export test-results.xml
    # If using environment:
    # newman run $(postmanCollection.secureFilePath) --environment $(postmanEnvironment.secureFilePath) --reporters cli junit --reporter-junit-export test-results.xml
  displayName: 'Run Postman API Tests with Newman'
  env:
    # Define any environment variables Newman might need, e.g., API URLs
    API_BASE_URL: $(yourApiBaseUrlVariable) # Example: using an Azure DevOps pipeline variable

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: 'test-results.xml'
    mergeTestResults: true
    failTaskOnFailedTests: true
  displayName: 'Publish Postman Test Results'
                    

Explanation:

• NodeTool@0: Ensures a compatible Node.js version is available on the agent.
• npm install -g newman: Installs Newman.
• DownloadSecureFile@1: Downloads your Postman collection (and optionally environment) which you've uploaded to Azure DevOps as secure files. This is a secure way to manage sensitive files.
• newman run ...: This is the core command that executes your collection.
  • --reporters cli junit: Outputs results to the console and generates a JUnit XML report.
  • --reporter-junit-export test-results.xml: Specifies the output file for the JUnit report.
  • --environment: (Optional) Specifies the environment file to use.
  • env: API_BASE_URL: $(yourApiBaseUrlVariable): Demonstrates how to pass pipeline variables to Newman, useful for dynamic configurations.
• PublishTestResults@2: Publishes the JUnit XML results to Azure DevOps, making them visible in the pipeline's 'Tests' tab. failTaskOnFailedTests: true will fail the pipeline if any tests fail.

Commit and Run Your Pipeline

Commit your YAML file and trigger your pipeline. You should see the Newman execution in the logs, and if everything is configured correctly, the test results will appear in the 'Tests' tab of your pipeline run.