Performance is crucial for large‑scale workflows. In this post I share ten practical ways to speed up your DAGs, from using pools effectively to leveraging task concurrency...
airflow_guru
Data Engineer | Airflow Enthusiast | Sharing tips, tutorials, and best practices for building robust ETL pipelines.
Posts 42
Followers 1.3k
Following 178
Overview
Posts
Comments
Activity
Welcome to my profile! I'm passionate about orchestrating data pipelines with Apache Airflow. Feel free to explore my posts and leave a comment.
The scheduler is the heart of Airflow. This article dives into how the scheduler decides which tasks to run, how it handles backfills, and common pitfalls to avoid.
Kubernetes provides a scalable platform for Airflow. Learn step‑by‑step how to use Helm charts to install, configure, and monitor an Airflow deployment on a K8s cluster.
No recent comments yet. Start a discussion and join the conversation!
- Sep 13, 2025 – Liked a post: Airflow 2.9 Release Highlights
- Sep 12, 2025 – Published a new post: 10 Tips to Optimize Airflow DAG Performance
- Sep 09, 2025 – Commented on Best Practices for DAG Versioning