Apache Airflow is an open source orchestration platform used to schedule, run and monitor chains of data tasks, called pipelines. It was created at Airbnb in 2014, then handed over to the Apache Software Foundation.
Pipelines are written in Python as DAGs (Directed Acyclic Graphs): a graph of tasks and their dependencies (extracting data, transforming it, training a model, producing a report). Airflow runs each task at the right time, in the right order, retries those that fail and provides an interface to monitor everything.
It is a reference tool in data engineering, often paired with dbt for transformations and a warehouse such as Snowflake or BigQuery. It has competitors (Dagster, Prefect) and is available as a managed service from the major cloud providers.
Why it matters when hiring
Airflow is a Data Engineer skill, and sometimes an MLOps Engineer one. Writing a simple DAG is quick to learn: real skill shows in operations (handling retries, idempotent tasks, alerting, scaling). In interviews, ask how the candidate diagnosed a pipeline that kept failing overnight. Do not insist on it at all costs: a profile who masters Dagster or Prefect transfers easily.