Apache Airflow API Documentation (Stable)

Introduction

The Apache Airflow REST API allows you to programmatically interact with your Airflow environment. You can manage DAGs, trigger runs, query task statuses, manage variables, connections, and more.

This API is designed to be idempotent and consistent, providing a reliable way to automate and orchestrate your workflows.

Base URL for the API: /api/v1

Authentication

Airflow's API supports basic authentication. You can authenticate by providing your Airflow username and password in the request headers.

Authorization: Basic

Alternatively, you can use a Personal Access Token (PAT) for more secure and granular access. Refer to the official documentation for details on generating and using PATs.

Authorization: Bearer

Endpoints

DAG Runs

Manage and retrieve information about DAG runs.

`GET /dags/{dag_id}/dagRuns`

GET

Retrieve a list of DAG runs for a specific DAG.

Query Parameters:

Name	Type	Description
`limit`	integer	The number of DAG runs to return.
`offset`	integer	The number of DAG runs to skip.
`state`	string	Filter DAG runs by state (e.g., `success`, `running`, `failed`).

Example Response:

{
  "dag_runs": [
    {
      "dag_id": "example_dag",
      "execution_date": "2023-10-27T10:00:00+00:00",
      "state": "success",
      "run_id": "manual__2023-10-27T10:00:00+00:00"
    },
    // ... more dag runs
  ],
  "total_entries": 15
}

`POST /dags/{dag_id}/dagRuns`

POST

Trigger a new DAG run.

Request Body:

Name	Type	Required	Description
`execution_date`	string (ISO 8601)	No	The execution date for the DAG run. If not provided, Airflow will use the current time.
`conf`	object	No	Configuration overrides for the DAG run.

Example Request:

{
  "execution_date": "2023-10-27T11:00:00+00:00",
  "conf": {
    "param1": "value1"
  }
}

Example Response:

{
  "dag_id": "example_dag",
  "execution_date": "2023-10-27T11:00:00+00:00",
  "state": "running",
  "run_id": "manual__2023-10-27T11:00:00+00:00",
  "conf": {
    "param1": "value1"
  }
}

`GET /dags/{dag_id}/dagRuns/{dag_run_id}`

GET

Retrieve details for a specific DAG run.

Example Response:

{
  "dag_id": "example_dag",
  "execution_date": "2023-10-27T11:00:00+00:00",
  "state": "running",
  "run_id": "manual__2023-10-27T11:00:00+00:00",
  "conf": {
    "param1": "value1"
  },
  "start_date": "2023-10-27T11:01:00+00:00",
  "end_date": null
}

`DELETE /dags/{dag_id}/dagRuns/{dag_run_id}`

DELETE

Delete a specific DAG run.

DAGs

Retrieve information about DAGs.

`GET /dags`

GET

Retrieve a list of all DAGs.

Query Parameters:

Name	Type	Description
`limit`	integer	The number of DAGs to return.
`offset`	integer	The number of DAGs to skip.

Example Response:

{
  "dags": [
    {
      "dag_id": "example_dag",
      "fileloc": "/path/to/dags/example_dag.py",
      "is_paused": false
    },
    // ... more dags
  ],
  "total_entries": 50
}

`GET /dags/{dag_id}`

GET

Retrieve details for a specific DAG.

Example Response:

{
  "dag_id": "example_dag",
  "fileloc": "/path/to/dags/example_dag.py",
  "is_paused": false,
  "owners": ["airflow"],
  "description": "An example DAG for demonstration purposes.",
  "schedule_interval": "0 0 * * *"
}

`PATCH /dags/{dag_id}`

PATCH

Update a DAG (e.g., pause/unpause).

Request Body:

Name	Type	Required	Description
`is_paused`	boolean	Yes	Set to `true` to pause the DAG, `false` to unpause.

Example Response:

{
  "dag_id": "example_dag",
  "fileloc": "/path/to/dags/example_dag.py",
  "is_paused": true
}

Tasks

Retrieve information about tasks within a DAG.

`GET /dags/{dag_id}/tasks`

GET

Retrieve a list of tasks for a specific DAG.

Example Response:

{
  "tasks": [
    {
      "task_id": "start",
      "dag_id": "example_dag",
      "task_type": "BashOperator"
    },
    {
      "task_id": "process_data",
      "dag_id": "example_dag",
      "task_type": "PythonOperator"
    },
    // ... more tasks
  ]
}

`GET /dags/{dag_id}/tasks/{task_id}`

GET

Retrieve details for a specific task.

Example Response:

{
  "task_id": "process_data",
  "dag_id": "example_dag",
  "task_type": "PythonOperator",
  "owner": "airflow",
  "retries": 2,
  "queue": "default"
}

Task Instances

Manage and retrieve information about task instances.

`GET /dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances`

GET

Retrieve a list of task instances for a specific DAG run.

Query Parameters:

Name	Type	Description
`state`	string	Filter task instances by state (e.g., `success`, `running`, `failed`).

Example Response:

{
  "task_instances": [
    {
      "task_id": "start",
      "dag_id": "example_dag",
      "dag_run_id": "manual__2023-10-27T11:00:00+00:00",
      "execution_date": "2023-10-27T11:00:00+00:00",
      "state": "success",
      "start_date": "2023-10-27T11:02:00+00:00",
      "end_date": "2023-10-27T11:03:00+00:00"
    },
    // ... more task instances
  ]
}

`GET /dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}`

GET

Retrieve details for a specific task instance.

Example Response:

{
  "task_id": "process_data",
  "dag_id": "example_dag",
  "dag_run_id": "manual__2023-10-27T11:00:00+00:00",
  "execution_date": "2023-10-27T11:00:00+00:00",
  "state": "running",
  "start_date": "2023-10-27T11:05:00+00:00",
  "end_date": null,
  "try_number": 1
}

`POST /dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/clear`

POST

Clear a specific task instance, allowing it to be re-run.

Example Response:

{
  "message": "Task instance cleared successfully."
}

`POST /dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/run`

POST

Run a specific task instance.

Example Response:

{
  "message": "Task instance triggered to run."
}

Variables

Manage Airflow variables.

`GET /variables`

GET

Retrieve a list of all variables.

Query Parameters:

Name	Type	Description
`limit`	integer	The number of variables to return.
`offset`	integer	The number of variables to skip.

Example Response:

{
  "variables": [
    {
      "key": "MY_API_KEY",
      "val": "************************",
      "description": "My important API key"
    },
    // ... more variables
  ],
  "total_entries": 10
}

`GET /variables/{key}`

GET

Retrieve a specific variable by its key.

Example Response:

{
  "key": "MY_API_KEY",
  "val": "************************",
  "description": "My important API key"
}

`POST /variables`

POST

Create a new variable.

Request Body:

Name	Type	Required	Description
`key`	string	Yes	The key of the variable.
`val`	string	No	The value of the variable.
`description`	string	No	A description for the variable.

Example Response:

{
  "key": "NEW_VARIABLE",
  "val": "some_value",
  "description": "A newly added variable."
}

`PUT /variables/{key}`

PUT

Update an existing variable.

Request Body:

Name	Type	Required	Description
`val`	string	No	The new value of the variable.
`description`	string	No	The new description for the variable.

`DELETE /variables/{key}`

DELETE

Delete a variable.

Connections

Manage Airflow connections.

`GET /connections`

GET

Retrieve a list of all connections.

Query Parameters:

Name	Type	Description
`limit`	integer	The number of connections to return.
`offset`	integer	The number of connections to skip.

Example Response:

{
  "connections": [
    {
      "conn_id": "postgres_default",
      "conn_type": "postgres",
      "host": "localhost",
      "schema": "airflow",
      "login": "user"
    },
    // ... more connections
  ],
  "total_entries": 5
}

`GET /connections/{conn_id}`

GET

Retrieve details for a specific connection.

`POST /connections`

POST

Create a new connection.

Request Body:

Name	Type	Required	Description
`conn_id`	string	Yes	The connection ID.
`conn_type`	string	Yes	The type of connection (e.g., `postgres`, `aws`, `http`).
`host`	string	No	The host.
`schema`	string	No	The schema.
`login`	string	No	The username.
`password`	string	No	The password (use with caution, consider secrets management).
`port`	integer	No	The port number.
`extra`	string	No	Extra parameters as a JSON string.

`PUT /connections/{conn_id}`

PUT

Update an existing connection.

Request Body:

Same fields as POST /connections, but conn_id is specified in the URL.

`DELETE /connections/{conn_id}`

DELETE

Delete a connection.

Pools

Manage Airflow pools.

`GET /pools`

GET

Retrieve a list of all pools.

Example Response:

{
  "pools": [
    {
      "pool_id": "default_pool",
      "slots": 128,
      "description": "Default pool for general tasks."
    },
    {
      "pool_id": "high_priority",
      "slots": 10,
      "description": "Pool for critical tasks."
    }
  ],
  "total_entries": 2
}

`GET /pools/{pool_id}`

GET

Retrieve details for a specific pool.

`POST /pools`

POST

Create a new pool.

Request Body:

Name	Type	Required	Description
`pool_id`	string	Yes	The pool ID.
`slots`	integer	Yes	The number of slots available in the pool.
`description`	string	No	A description for the pool.

`PUT /pools/{pool_id}`

PUT

Update an existing pool.

Request Body:

Name	Type	Required	Description
`slots`	integer	No	The new number of slots.
`description`	string	No	The new description.

`DELETE /pools/{pool_id}`

DELETE

Delete a pool.

Users

Manage Airflow users (requires admin privileges).

`GET /users`

GET

Retrieve a list of users.

Query Parameters:

Name	Type	Description
`limit`	integer	The number of users to return.
`offset`	integer	The number of users to skip.

Example Response:

{
  "users": [
    {
      "username": "admin",
      "email": "admin@example.com",
      "is_active": true,
      "is_admin": true
    },
    // ... more users
  ],
  "total_entries": 3
}

`POST /users`

POST

Create a new user.

Request Body:

Name	Type	Required	Description
`username`	string	Yes	The username for the new user.
`email`	string	Yes	The email address for the new user.
`password`	string	Yes	The password for the new user.
`is_active`	boolean	No	Whether the user is active (defaults to true).
`is_admin`	boolean	No	Whether the user is an administrator (defaults to false).

`GET /users/{username}`

GET

Retrieve details for a specific user.

`PATCH /users/{username}`

PATCH

Update an existing user.

Request Body:

Fields can include email, is_active, is_admin, and password.

`DELETE /users/{username}`

DELETE

Delete a user.

Versioning

The Airflow API is versioned. The current stable version is accessible at /api/v1. Future releases may introduce new versions with backward-incompatible changes.

Always refer to the official Airflow REST API reference for the most up-to-date information.