Airflow

TechTool

Apache Airflow is an open-source platform for orchestrating data workflows, created at Airbnb in 2014 and later handed over to the Apache Foundation.

Apache Airflow is an open-source platform for orchestrating data workflows, created at Airbnb in 2014 and later handed over to the Apache Foundation.

It lets you define pipelines (extraction, transformation, loading, ML, reporting…) as DAGs (Directed Acyclic Graphs) in Python, then schedule, execute, monitor and manage their dependencies and error recovery.

It is one of the references for modern data engineering, competing with Dagster, Prefect, Argo Workflows and managed services (MWAA on AWS, Cloud Composer on GCP).

Related terms

DeFi
DeFi (Decentralised Finance) refers to the set of financial services - lending, borrowing, swapping, derivatives, insurance - operated through smart contracts on a public blockchain (mostly Ethereum and its Layer 2s), w…
See definition →
Python
Python is a programming language invented by Guido van Rossum.
See definition →
Data
Talked about endlessly since the dawn of the digital era, data is the new black gold - if not the new blue gold.
See definition →
Data Engineer
See the Data Engineer role page.
See definition →
ORM
An ORM (Object-Relational Mapping) is a type of program that sits between an application and a relational database to simulate an object-oriented database.
See definition →
Bun
Bun is a modern JavaScript and TypeScript runtime, written in Zig, that positions itself as an alternative to Node.js and Deno with a focus on performance.
See definition →

A recruitment need?

We take the time to understand your context first. Tell us about your need, we'll get back to you quickly.

Let's talk about your next hire I'm a candidate

Related terms

DeFi

Python

Data

Data Engineer

ORM

Bun

A recruitment need?