Autoplay
Autocomplete
Previous Lesson
Complete and Continue
Fundamentals of Apache Airflow
Introduction
Introduction (7:19)
What Is Apache Airflow? (5:18)
Airflow’s Architecture (3:14)
Exercise: Meet Your Classmates and Instructor
Course Resources
Setup and Useful Resources
[Optional] UNIX CLI Commands
[Optional] Using Windows
Installing Software for the Course
[Optional] What Is a Virtualenv? (6:36)
[Optional] What Is Docker? (11:02)
[Optional] Apache Spark Introduction
Why Apache Spark?
Introduction to PySpark (4:23)
DataFrames Operations (8:50)
Installing Spark (5:50)
Installing Spark on Linux
Implementing Data Pipelines with Apache Airflow
Installing Airflow (6:32)
Defining an Airflow DAG (8:02)
Errors Handling (3:37)
Idempotent Tasks (4:53)
Creating a DAG - Part 1 (4:58)
Creating a DAG - Part 2 (4:41)
Handling Failed Tasks (4:08)
[Exercise] Data Validation (4:30)
[Exercise] Data Validation - Solution (3:26)
Spark with Airflow (3:01)
Using Spark with Airflow - Part 1 (7:38)
Using Spark with Airflow - Part 2 (5:51)
Sensors In Airflow (4:45)
Using File Sensors (4:07)
Data Ingestion (5:49)
Reading Data From Postgres - Part 1 (6:02)
Reading Data from Postgres - Part 2 (5:39)
[Exercise] Average Customer Review (3:52)
[Exercise] Average Customer Review - Solution (4:32)
Advanced DAGs (4:25)
Where To Go From Here?
Let's Keep Learning Together! (1:05)
Review This Byte!
Defining an Airflow DAG