Triggering DAGs

This guide covers various methods for triggering Airflow DAGs, both manually and programmatically.

Manual Triggering via the UI

The most straightforward way to trigger a DAG is through the Airflow web UI. Navigate to the "Browse" -> "DAGs" view, locate your DAG, and click the "Trigger DAG" button (lightning bolt icon).

You can also provide configuration JSON to customize the trigger, which is useful for passing specific parameters to your DAG runs. For example:

{
  "conf": {
    "execution_date": "2023-10-27T10:00:00",
    "param1": "value1",
    "param2": 123
  }
}

The conf dictionary will be accessible in your DAG code via dag_run.conf.

Triggering DAGs Programmatically (Python API)

You can trigger DAGs from within other Python scripts or even from within another Airflow DAG using the Airflow Python API. This offers more flexibility for complex orchestration scenarios.

Using the `TriggerDagRunOperator`

The TriggerDagRunOperator is the recommended way to trigger one DAG from another.

from airflow.operators.trigger_dagrun import TriggerDagRunOperator
from airflow.models.dag import DAG
from datetime import datetime

with DAG(
    dag_id='parent_dag',
    start_date=datetime(2023, 1, 1),
    schedule_interval=None,
    catchup=False,
) as dag:
    trigger_child_dag = TriggerDagRunOperator(
        task_id='trigger_child_dag_task',
        trigger_dag_id='child_dag_to_trigger',
        conf={"message": "Triggered by parent_dag"},
        execution_date="{{ ds }}",
        wait_for_completion=True,  # Set to True to wait for the triggered DAG to finish
    )

Key parameters for TriggerDagRunOperator:

• trigger_dag_id: The DAG ID of the DAG to trigger.
• conf: A dictionary of configurations to pass to the triggered DAG run.
• execution_date: The execution date for the triggered DAG run. You can use Jinja templating here.
• wait_for_completion: If True, the task will wait until the triggered DAG run completes.
• reset_dag_run: If True, this will reset any existing DAG runs for the specified execution_date before triggering.

Using the Airflow CLI

You can also trigger DAGs using the Airflow command-line interface. This is useful for scripting and automation outside of a running Airflow instance.

# Trigger a DAG with a specific execution date
airflow dags trigger your_dag_id --execution-date 2023-10-27T10:00:00

# Trigger a DAG and pass configuration
airflow dags trigger your_dag_id --execution-date 2023-10-27T10:00:00 --conf '{"param1": "value1", "param2": 123}'

Triggering DAGs via the REST API

Airflow provides a REST API that allows you to interact with your Airflow instance programmatically, including triggering DAGs.

To trigger a DAG, you can send a POST request to the /api/v1/dags/{dag_id}/dagRuns endpoint.

curl -X POST \
  http://localhost:8080/api/v1/dags/your_dag_id/dagRuns \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer YOUR_API_TOKEN' \
  -d '{
    "execution_date": "2023-10-27T10:00:00",
    "conf": {
      "message": "Triggered via REST API"
    }
  }'

Remember to replace YOUR_API_TOKEN with your actual authentication token and adjust the host and port as necessary.