Autoplay
Autocomplete
Previous Lesson
Complete and Continue
PyTorch for Deep Learning in 2024: Zero to Mastery
Introduction
PyTorch for Deep Learning (3:33)
Course Welcome and What Is Deep Learning (5:53)
Exercise: Meet Your Classmates and Instructor
Course Companion Book + Code + More
Machine Learning + Python Monthly
Understanding Your Video Player (notes, video speed, subtitles + more)
Set Your Learning Streak Goal
Section 00: PyTorch Fundamentals
Why Use Machine Learning or Deep Learning (3:33)
The Number 1 Rule of Machine Learning and What Is Deep Learning Good For (5:39)
Machine Learning vs. Deep Learning (6:06)
Anatomy of Neural Networks (9:21)
Different Types of Learning Paradigms (4:30)
What Can Deep Learning Be Used For (6:21)
What Is and Why PyTorch (10:12)
What Are Tensors (4:15)
What We Are Going To Cover With PyTorch (6:05)
How To and How Not To Approach This Course (5:09)
Important Resources For This Course (5:21)
Getting Setup to Write PyTorch Code (7:39)
Introduction to PyTorch Tensors (13:24)
Creating Random Tensors in PyTorch (9:58)
Creating Tensors With Zeros and Ones in PyTorch (3:08)
Creating a Tensor Range and Tensors Like Other Tensors (5:17)
Dealing With Tensor Data Types (9:24)
Getting Tensor Attributes (8:22)
Manipulating Tensors (Tensor Operations) (5:59)
Matrix Multiplication (Part 1) (9:34)
Matrix Multiplication (Part 2): The Two Main Rules of Matrix Multiplication (7:51)
Matrix Multiplication (Part 3): Dealing With Tensor Shape Errors (12:56)
Finding the Min Max Mean and Sum of Tensors (Tensor Aggregation) (6:09)
Finding The Positional Min and Max of Tensors (3:16)
Reshaping, Viewing and Stacking Tensors (13:40)
Squeezing, Unsqueezing and Permuting Tensors (11:55)
Selecting Data From Tensors (Indexing) (9:31)
PyTorch Tensors and NumPy (9:08)
PyTorch Reproducibility (Taking the Random Out of Random) (10:46)
Different Ways of Accessing a GPU in PyTorch (11:50)
Setting up Device Agnostic Code and Putting Tensors On and Off the GPU (7:43)
PyTorch Fundamentals: Exercises and Extra-Curriculum (4:49)
Let's Have Some Fun (+ Free Resources)
Section 01: PyTorch Workflow
Introduction and Where You Can Get Help (2:45)
Getting Setup and What We Are Covering (7:14)
Creating a Simple Dataset Using the Linear Regression Formula (9:40)
Splitting Our Data Into Training and Test Sets (8:19)
Building a function to Visualize Our Data (7:45)
Creating Our First PyTorch Model for Linear Regression (14:09)
Breaking Down What's Happening in Our PyTorch Linear regression Model (6:10)
Discussing Some of the Most Important PyTorch Model Building Classes (6:26)
Checking Out the Internals of Our PyTorch Model (9:50)
Making Predictions With Our Random Model Using Inference Mode (11:12)
Training a Model Intuition (The Things We Need) (8:14)
Setting Up an Optimizer and a Loss Function (12:51)
PyTorch Training Loop Steps and Intuition (13:53)
Writing Code for a PyTorch Training Loop (8:46)
Reviewing the Steps in a Training Loop Step by Step (14:57)
Running Our Training Loop Epoch by Epoch and Seeing What Happens (9:25)
Writing Testing Loop Code and Discussing What's Happening Step by Step (11:37)
Reviewing What Happens in a Testing Loop Step by Step (14:42)
Writing Code to Save a PyTorch Model (13:45)
Writing Code to Load a PyTorch Model (8:44)
Setting Up to Practice Everything We Have Done Using Device-Agnostic Code (6:02)
Putting Everything Together (Part 1): Data (6:07)
Putting Everything Together (Part 2): Building a Model (10:07)
Putting Everything Together (Part 3): Training a Model (12:39)
Putting Everything Together (Part 4): Making Predictions With a Trained Model (5:17)
Putting Everything Together (Part 5): Saving and Loading a Trained Model (9:10)
PyTorch Workflow: Exercises and Extra-Curriculum (3:57)
Unlimited Updates
Section 02: PyTorch Neural Network Classification
Introduction to Machine Learning Classification With PyTorch (9:41)
Classification Problem Example: Input and Output Shapes (9:06)
Typical Architecture of a Classification Neural Network (Overview) (6:30)
Making a Toy Classification Dataset (12:18)
Turning Our Data into Tensors and Making a Training and Test Split (11:55)
Laying Out Steps for Modelling and Setting Up Device-Agnostic Code (4:19)
Coding a Small Neural Network to Handle Our Classification Data (10:57)
Making Our Neural Network Visual (6:57)
Recreating and Exploring the Insides of Our Model Using nn.Sequential (13:17)
Setting Up a Loss Function Optimizer and Evaluation Function for Our Classification Network (14:50)
Going from Model Logits to Prediction Probabilities to Prediction Labels (16:06)
Coding a Training and Testing Optimization Loop for Our Classification Model (15:26)
Writing Code to Download a Helper Function to Visualize Our Models Predictions (14:13)
Discussing Options to Improve a Model (8:02)
Creating a New Model with More Layers and Hidden Units (9:06)
Writing Training and Testing Code to See if Our New and Upgraded Model Performs Better (12:45)
Creating a Straight Line Dataset to See if Our Model is Learning Anything (8:07)
Building and Training a Model to Fit on Straight Line Data (10:01)
Evaluating Our Models Predictions on Straight Line Data (5:23)
Introducing the Missing Piece for Our Classification Model Non-Linearity (10:00)
Building Our First Neural Network with Non-Linearity (10:25)
Writing Training and Testing Code for Our First Non-Linear Model (15:12)
Making Predictions with and Evaluating Our First Non-Linear Model (5:47)
Replicating Non-Linear Activation Functions with Pure PyTorch (9:34)
Putting It All Together (Part 1): Building a Multiclass Dataset (11:24)
Creating a Multi-Class Classification Model with PyTorch (12:27)
Setting Up a Loss Function and Optimizer for Our Multi-Class Model (6:39)
Going from Logits to Prediction Probabilities to Prediction Labels with a Multi-Class Model (11:01)
Training a Multi-Class Classification Model and Troubleshooting Code on the Fly (16:17)
Making Predictions with and Evaluating Our Multi-Class Classification Model (7:59)
Discussing a Few More Classification Metrics (9:17)
PyTorch Classification: Exercises and Extra-Curriculum (2:58)
Course Check-In
Section 03: PyTorch Computer Vision
What Is a Computer Vision Problem and What We Are Going to Cover (11:47)
Computer Vision Input and Output Shapes (10:08)
What Is a Convolutional Neural Network (CNN) (5:02)
Discussing and Importing the Base Computer Vision Libraries in PyTorch (9:19)
Getting a Computer Vision Dataset and Checking Out Its- Input and Output Shapes (14:30)
Visualizing Random Samples of Data (9:51)
DataLoader Overview Understanding Mini-Batch (7:17)
Turning Our Datasets Into DataLoaders (12:23)
Model 0: Creating a Baseline Model with Two Linear Layers (14:38)
Creating a Loss Function: an Optimizer for Model 0 (10:29)
Creating a Function to Time Our Modelling Code (5:34)
Writing Training and Testing Loops for Our Batched Data (21:25)
Writing an Evaluation Function to Get Our Models Results (12:58)
Setup Device-Agnostic Code for Running Experiments on the GPU (3:46)
Model 1: Creating a Model with Non-Linear Functions (9:03)
Model 1: Creating a Loss Function and Optimizer (3:04)
Turing Our Training Loop into a Function (8:28)
Turing Our Testing Loop into a Function (6:35)
Training and Testing Model 1 with Our Training and Testing Functions (11:52)
Getting a Results Dictionary for Model 1 (4:08)
Model 2: Convolutional Neural Networks High Level Overview (8:24)
Model 2: Coding Our First Convolutional Neural Network with PyTorch (19:48)
Model 2: Breaking Down Conv2D Step by Step (14:59)
Model 2: Breaking Down MaxPool2D Step by Step (15:48)
Model 2: Using a Trick to Find the Input and Output Shapes of Each of Our Layers (13:45)
Model 2: Setting Up a Loss Function and Optimizer (2:38)
Model 2: Training Our First CNN and Evaluating Its Results (7:54)
Comparing the Results of Our Modelling Experiments (7:23)
Making Predictions on Random Test Samples with the Best Trained Model (11:39)
Plotting Our Best Model Predictions on Random Test Samples and Evaluating Them (8:10)
Making Predictions Across the Whole Test Dataset and Importing Libraries to Plot a Confusion Matrix (15:20)
Evaluating Our Best Models Predictions with a Confusion Matrix (6:54)
Saving and Loading Our Best Performing Model (11:27)
Recapping What We Have Covered Plus Exercises and Extra-Curriculum (6:01)
Implement a New Life System
Section 04: PyTorch Custom Datasets
What Is a Custom Dataset and What We Are Going to Cover (9:53)
Importing PyTorch and Setting Up Device-Agnostic Code (5:54)
Downloading a Custom Dataset of Pizza, Steak and Sushi Images (14:04)
Becoming One With the Data (Part 1): Exploring the Data Format (8:41)
Becoming One With the Data (Part 2): Visualizing a Random Image (11:40)
Becoming One With the Data (Part 3): Visualizing a Random Image with Matplotlib (4:47)
Transforming Data (Part 1): Turning Images Into Tensors (8:53)
Transforming Data (Part 2): Visualizing Transformed Images (11:30)
Loading All of Our Images and Turning Them Into Tensors With ImageFolder (9:17)
Visualizing a Loaded Image From the Train Dataset (7:18)
Turning Our Image Datasets into PyTorch DataLoaders (9:03)
Creating a Custom Dataset Class in PyTorch High Level Overview (7:59)
Creating a Helper Function to Get Class Names From a Directory (9:06)
Writing a PyTorch Custom Dataset Class from Scratch to Load Our Images (17:46)
Compare Our Custom Dataset Class to the Original ImageFolder Class (7:13)
Writing a Helper Function to Visualize Random Images from Our Custom Dataset (14:18)
Turning Our Custom Datasets Into DataLoaders (6:58)
Exploring State of the Art Data Augmentation With Torchvision Transforms (14:23)
Building a Baseline Model (Part 1): Loading and Transforming Data (8:15)
Building a Baseline Model (Part 2): Replicating Tiny VGG from Scratch (11:24)
Building a Baseline Model (Part 3): Doing a Forward Pass to Test Our Model Shapes (8:09)
Using the Torchinfo Package to Get a Summary of Our Model (6:38)
Creating Training and Testing loop Functions (13:03)
Creating a Train Function to Train and Evaluate Our Models (10:14)
Training and Evaluating Model 0 With Our Training Functions (9:53)
Plotting the Loss Curves of Model 0 (9:02)
Discussing the Balance Between Overfitting and Underfitting and How to Deal With Each (14:13)
Creating Augmented Training Datasets and DataLoaders for Model 1 (11:03)
Constructing and Training Model 1 (7:10)
Plotting the Loss Curves of Model 1 (3:22)
Plotting the Loss Curves of All of Our Models Against Each Other (10:55)
Predicting on Custom Data (Part 1): Downloading an Image (5:32)
Predicting on Custom Data (Part2): Loading In a Custom Image With PyTorch (7:00)
Predicting on Custom Data (Part 3): Getting Our Custom Image Into the Right Format (14:06)
Predicting on Custom Data (Part 4): Turning Our Models Raw Outputs Into Prediction Labels (4:24)
Predicting on Custom Data (Part 5): Putting It All Together (12:47)
Summary of What We Have Covered Plus Exercises and Extra-Curriculum (6:04)
Exercise: Imposter Syndrome (2:55)
Section 05: PyTorch Going Modular
What Is Going Modular and What We Are Going to Cover (11:34)
Going Modular Notebook (Part 1): Running It End to End (7:39)
Downloading a Dataset (4:49)
Writing the Outline for Our First Python Script to Setup the Data (13:50)
Creating a Python Script to Create Our PyTorch DataLoaders (10:35)
Turning Our Model Building Code into a Python Script (9:18)
Turning Our Model Training Code into a Python Script (6:16)
Turning Our Utility Function to Save a Model into a Python Script (6:06)
Creating a Training Script to Train Our Model in One Line of Code (15:46)
Going Modular: Summary, Exercises and Extra-Curriculum (5:59)
Section 06: PyTorch Transfer Learning
Introduction: What is Transfer Learning and Why Use It (10:05)
Where Can You Find Pretrained Models and What We Are Going to Cover (5:12)
Installing the Latest Versions of Torch and Torchvision (8:05)
Downloading Our Previously Written Code from Going Modular (6:41)
Downloading Pizza, Steak, Sushi Image Data from Github (8:00)
Turning Our Data into DataLoaders with Manually Created Transforms (14:40)
Turning Our Data into DataLoaders with Automatic Created Transforms (13:06)
Which Pretrained Model Should You Use (12:15)
Setting Up a Pretrained Model with Torchvision (10:57)
Different Kinds of Transfer Learning (7:11)
Getting a Summary of the Different Layers of Our Model (6:49)
Freezing the Base Layers of Our Model and Updating the Classifier Head (13:26)
Training Our First Transfer Learning Feature Extractor Model (7:54)
Plotting the Loss Curves of Our Transfer Learning Model (6:26)
Outlining the Steps to Make Predictions on the Test Images (7:57)
Creating a Function Predict On and Plot Images (10:00)
Making and Plotting Predictions on Test Images (7:23)
Making a Prediction on a Custom Image (6:21)
Main Takeaways, Exercises and Extra Curriculum (3:21)
Section 07: PyTorch Experiment Tracking
What Is Experiment Tracking and Why Track Experiments (7:06)
Getting Setup by Importing Torch Libraries and Going Modular Code (8:13)
Creating a Function to Download Data (10:23)
Turning Our Data into DataLoaders Using Manual Transforms (8:30)
Turning Our Data into DataLoaders Using Automatic Transforms (7:47)
Preparing a Pretrained Model for Our Own Problem (10:28)
Setting Up a Way to Track a Single Model Experiment with TensorBoard (13:35)
Training a Single Model and Saving the Results to TensorBoard (4:38)
Exploring Our Single Models Results with TensorBoard (10:17)
Creating a Function to Create SummaryWriter Instances (10:45)
Adapting Our Train Function to Be Able to Track Multiple Experiments (4:57)
What Experiments Should You Try (5:59)
Discussing the Experiments We Are Going to Try (6:01)
Downloading Datasets for Our Modelling Experiments (6:31)
Turning Our Datasets into DataLoaders Ready for Experimentation (8:28)
Creating Functions to Prepare Our Feature Extractor Models (15:54)
Coding Out the Steps to Run a Series of Modelling Experiments (14:27)
Running Eight Different Modelling Experiments in 5 Minutes (3:50)
Viewing Our Modelling Experiments in TensorBoard (13:38)
Loading In the Best Model and Making Predictions on Random Images from the Test Set (10:32)
Making a Prediction on Our Own Custom Image with the Best Model (3:44)
Main Takeaways, Exercises and Extra Curriculum (3:56)
Section 08: PyTorch Paper Replicating
What Is a Machine Learning Research Paper? (7:34)
Why Replicate a Machine Learning Research Paper? (3:13)
Where Can You Find Machine Learning Research Papers and Code? (8:18)
What We Are Going to Cover (8:21)
Getting Setup for Coding in Google Colab (8:21)
Downloading Data for Food Vision Mini (4:02)
Turning Our Food Vision Mini Images into PyTorch DataLoaders (9:47)
Visualizing a Single Image (3:45)
Replicating a Vision Transformer - High Level Overview (9:53)
Breaking Down Figure 1 of the ViT Paper (11:12)
Breaking Down the Four Equations Overview and a Trick for Reading Papers (10:55)
Breaking Down Equation 1 (8:14)
Breaking Down Equations 2 and 3 (10:03)
Breaking Down Equation 4 (7:27)
Breaking Down Table 1 (11:05)
Calculating the Input and Output Shape of the Embedding Layer by Hand (15:41)
Turning a Single Image into Patches (Part 1: Patching the Top Row) (15:03)
Turning a Single Image into Patches (Part 2: Patching the Entire Image) (12:33)
Creating Patch Embeddings with a Convolutional Layer (13:33)
Exploring the Outputs of Our Convolutional Patch Embedding Layer (12:54)
Flattening Our Convolutional Feature Maps into a Sequence of Patch Embeddings (9:59)
Visualizing a Single Sequence Vector of Patch Embeddings (5:03)
Creating the Patch Embedding Layer with PyTorch (17:01)
Creating the Class Token Embedding (13:24)
Creating the Class Token Embedding - Less Birds (13:24)
Creating the Position Embedding (11:25)
Equation 1: Putting it All Together (13:25)
Equation 2: Multihead Attention Overview (14:30)
Equation 2: Layernorm Overview (9:03)
Turning Equation 2 into Code (14:33)
Checking the Inputs and Outputs of Equation (5:40)
Equation 3: Replication Overview (9:11)
Turning Equation 3 into Code (11:25)
Transformer Encoder Overview (8:50)
Combining Equation 2 and 3 to Create the Transformer Encoder (9:16)
Creating a Transformer Encoder Layer with In-Built PyTorch Layer (15:54)
Bringing Our Own Vision Transformer to Life - Part 1: Gathering the Pieces of the Puzzle (18:19)
Bringing Our Own Vision Transformer to Life - Part 2: Putting Together the Forward Method (10:41)
Getting a Visual Summary of Our Custom Vision Transformer (7:13)
Creating a Loss Function and Optimizer from the ViT Paper (11:26)
Training our Custom ViT on Food Vision Mini (4:29)
Discussing what Our Training Setup Is Missing (9:08)
Plotting a Loss Curve for Our ViT Model (6:13)
Getting a Pretrained Vision Transformer from Torchvision and Setting it Up (14:37)
Preparing Data to Be Used with a Pretrained ViT (5:53)
Training a Pretrained ViT Feature Extractor Model for Food Vision Mini (7:15)
Saving Our Pretrained ViT Model to File and Inspecting Its Size (5:13)
Discussing the Trade-Offs Between Using a Larger Model for Deployments (3:46)
Making Predictions on a Custom Image with Our Pretrained ViT (3:30)
PyTorch Paper Replicating: Main Takeaways, Exercises and Extra-Curriculum (6:50)
Section 09: PyTorch Model Deployment
What is Machine Learning Model Deployment and Why Deploy a Machine Learning Model (9:35)
Three Questions to Ask for Machine Learning Model Deployment (7:13)
Where Is My Model Going to Go? (13:34)
How Is My Model Going to Function? (7:59)
Some Tools and Places to Deploy Machine Learning Models (5:49)
What We Are Going to Cover (4:01)
Getting Setup to Code (6:15)
Downloading a Dataset for Food Vision Mini (3:23)
Outlining Our Food Vision Mini Deployment Goals and Modelling Experiments (7:59)
Creating an EffNetB2 Feature Extractor Model (9:45)
Create a Function to Make an EffNetB2 Feature Extractor Model and Transforms (6:29)
Creating DataLoaders for EffNetB2 (3:31)
Training Our EffNetB2 Feature Extractor and Inspecting the Loss Curves (9:15)
Saving Our EffNetB2 Model to File (3:24)
Getting the Size of Our EffNetB2 Model in Megabytes (5:51)
Collecting Important Statistics and Performance Metrics for Our EffNetB2 Model (6:34)
Creating a Vision Transformer Feature Extractor Model (7:51)
Creating DataLoaders for Our ViT Feature Extractor Model (2:30)
Training Our ViT Feature Extractor Model and Inspecting Its Loss Curves (6:19)
Saving Our ViT Feature Extractor and Inspecting Its Size (5:08)
Collecting Stats About Our ViT Feature Extractor (5:51)
Outlining the Steps for Making and Timing Predictions for Our Models (11:15)
Creating a Function to Make and Time Predictions with Our Models (16:20)
Making and Timing Predictions with EffNetB2 (10:43)
Making and Timing Predictions with ViT (7:34)
Comparing EffNetB2 and ViT Model Statistics (11:31)
Visualizing the Performance vs Speed Trade-off (15:54)
Gradio Overview and Installation (8:39)
Gradio Function Outline (8:49)
Creating a Predict Function to Map Our Food Vision Mini Inputs to Outputs (9:51)
Creating a List of Examples to Pass to Our Gradio Demo (5:26)
Bringing Food Vision Mini to Life in a Live Web Application (12:12)
Getting Ready to Deploy Our App Hugging Face Spaces Overview (6:26)
Outlining the File Structure of Our Deployed App (8:11)
Creating a Food Vision Mini Demo Directory to House Our App Files (4:11)
Creating an Examples Directory with Example Food Vision Mini Images (9:13)
Writing Code to Move Our Saved EffNetB2 Model File (7:42)
Turning Our EffNetB2 Model Creation Function Into a Python Script (4:01)
Turning Our Food Vision Mini Demo App Into a Python Script (13:27)
Creating a Requirements File for Our Food Vision Mini App (4:11)
Downloading Our Food Vision Mini App Files from Google Colab (11:30)
Uploading Our Food Vision Mini App to Hugging Face Spaces Programmatically (13:36)
Running Food Vision Mini on Hugging Face Spaces and Trying it Out (7:44)
Food Vision Big Project Outline (4:17)
Preparing an EffNetB2 Feature Extractor Model for Food Vision Big (9:38)
Downloading the Food 101 Dataset (7:45)
Creating a Function to Split Our Food 101 Dataset into Smaller Portions (13:36)
Turning Our Food 101 Datasets into DataLoaders (7:23)
Training Food Vision Big: Our Biggest Model Yet! (20:15)
Outlining the File Structure for Our Food Vision Big (5:48)
Downloading an Example Image and Moving Our Food Vision Big Model File (3:33)
Saving Food 101 Class Names to a Text File and Reading them Back In (6:56)
Turning Our EffNetB2 Feature Extractor Creation Function into a Python Script (2:20)
Creating an App Script for Our Food Vision Big Model Gradio Demo (10:41)
Zipping and Downloading Our Food Vision Big App Files (3:45)
Deploying Food Vision Big to Hugging Face Spaces (13:34)
PyTorch Mode Deployment: Main Takeaways, Extra-Curriculum and Exercises (6:13)
Introduction to PyTorch 2.0 and torch.compile
Introduction to PyTorch 2.0 (6:01)
What We Are Going to Cover and PyTorch 2 Reference Materials (1:21)
Getting Started with PyTorch 2.0 in Google Colab (4:19)
PyTorch 2.0 - 30 Second Intro (3:20)
Getting Setup for PyTorch 2.0 (2:22)
Getting Info from Our GPUs and Seeing if They're Capable of Using PyTorch 2.0 (6:49)
Setting the Default Device in PyTorch 2.0 (9:40)
Discussing the Experiments We Are Going to Run for PyTorch 2.0 (6:42)
Creating a Function to Setup Our Model and Transforms (10:17)
Discussing How to Get Better Relative Speedups for Training Models (8:23)
Setting the Batch Size and Data Size Programmatically (7:15)
Getting More Speedups with TensorFloat-32 (9:53)
Downloading the CIFAR10 Dataset (7:00)
Creating Training and Test DataLoaders (7:38)
Preparing Training and Testing Loops with Timing Steps (4:58)
Experiment 1 - Single Run without Torch Compile (8:22)
Experiment 2 - Single Run with Torch Compile (10:38)
Comparing the Results of Experiments 1 and 2 (11:19)
Saving the Results of Experiments 1 and 2 (4:39)
Preparing Functions for Experiments 3 and 4 (12:41)
Experiment 3 - Training a Non-Compiled Model for Multiple Runs (12:44)
Experiment 4 - Training a Compiled Model for Multiple Runs (9:57)
Comparing the Results of Experiments 3 and 4 (5:23)
Potential Extensions and Resources to Learn More (5:50)
Where To Go From Here?
Thank You! (1:17)
Review This Course!
Become An Alumni
Learning Guideline
ZTM Events Every Month
LinkedIn Endorsements
Setting up Device Agnostic Code and Putting Tensors On and Off the GPU
This lecture is available exclusively for ZTM Academy members.
If you're already a member,
you'll need to login
.
Join ZTM To Unlock All Lectures