Example Curriculum
Section 0: Introduction
Available in
days
days
after you enroll
- AI Engineering Bootcamp: Learn AWS SageMaker with Patrik Szepesi (1:35)
- Course Setup
- Course Introduction Part 1: Build, Train, and Deploy Models with AWS SageMaker (8:42)
- Course Introduction Part 2: Fine-Tuning LLMs with QLoRA, AWS, and Open Source (5:19)
- Exercise: Meet Your Classmates and Instructor
- Course Resources
- Understanding Your Video Player
- Set Your Learning Streak Goal
Part 1: Build, Train and Deploy Models with AWS SageMaker
Available in
days
days
after you enroll
Section 1: Introduction to AWS, Environment Setup, and Best Practices
Available in
days
days
after you enroll
- Setting Up Our AWS Account (4:31)
- Set Up IAM Roles + Best Practices (7:39)
- AWS Security Best Practices (7:01)
- Set Up AWS SageMaker Domain (2:22)
- UI Domain Change (0:42)
- Sagemaker Domain Creation Update Part 1 (2:40)
- Sagemaker Domain Creation Update Part 2 (3:06)
- Sagemaker Notebooks Update (11:58)
- Setting Up SageMaker Environment (5:08)
- SageMaker Studio and Pricing (8:44)
- Let's Have Some Fun (+ More Resources)
Section 2: Possible Resource Limit Errors Before Training and Deployment
Available in
days
days
after you enroll
Section 3: A Gentle Introduction to HuggingFace in SageMaker
Available in
days
days
after you enroll
Section 4: Gathering a Dataset for Our Multiclass Text Classification Project
Available in
days
days
after you enroll
Section 5: Exploratory Data Analysis
Available in
days
days
after you enroll
Section 6: Setting Up Our Training Notebook
Available in
days
days
after you enroll
Section 7: Introduction to Tokenizations and Encodings
Available in
days
days
after you enroll
- Creating Our Optional Experiment Notebook - Part 1 (3:21)
- Creating Our Optional Experiment Notebook - Part 2 (4:01)
- Encoding Categorical Labels to Numeric Values (13:24)
- Understanding the Tokenization Vocabulary (15:05)
- Encoding Tokens (10:56)
- Practical Example of Tokenization and Encoding (12:48)
- Course Check-In
Section 8: Setting Up Data Loading with PyTorch
Available in
days
days
after you enroll
Section 9: Choose Your Path
Available in
days
days
after you enroll
Section 10: Mathematics Behind Large Language Models and Transformers
Available in
days
days
after you enroll
- DistilBert vs. Bert Differences (4:46)
- Embeddings In A Continuous Vector Space (7:40)
- Introduction To Positional Encodings (5:13)
- Positional Encodings - Part 1 (4:14)
- Positional Encodings - Part 2 (Even and Odd Indices) (10:10)
- Why Use Sine and Cosine Functions (5:08)
- Understanding the Nature of Sine and Cosine Functions (9:52)
- Visualizing Positional Encodings in Sine and Cosine Graphs (9:24)
- Solving the Equations to Get the Values for Positional Encodings (18:07)
- Introduction to Attention Mechanism (3:02)
- Query, Key and Value Matrix (18:10)
- Getting Started with Our Step by Step Attention Calculation (6:53)
- Calculating Key Vectors (20:05)
- Query Matrix Introduction (10:20)
- Calculating Raw Attention Scores (21:24)
- Understanding the Mathematics Behind Dot Products and Vector Alignment (13:32)
- Visualizing Raw Attention Scores in 2D (5:42)
- Converting Raw Attention Scores to Probability Distributions with Softmax (9:16)
- Normalization (3:19)
- Understanding the Value Matrix and Value Vector (9:07)
- Calculating the Final Context Aware Rich Representation for the Word "River" (10:45)
- Understanding the Output (1:58)
- Understanding Multi Head Attention (11:55)
- Multi Head Attention Example and Subsequent Layers (9:51)
- Masked Language Learning (2:29)
- Exercise: Imposter Syndrome (2:56)
Section 11: Customizing our Model Architecture in PyTorch
Available in
days
days
after you enroll
Section 12: Creating the Accuracy, Training, and Validation Function
Available in
days
days
after you enroll
Section 13: Optimizer Functions, Model Parameters, Cross Entropy Loss Function
Available in
days
days
after you enroll
Section 14: Starting Our Training Job and Monitoring it in AWS CloudWatch
Available in
days
days
after you enroll
Section 15: Deploying our Multiclass Text Classification Endpoint in SageMaker
Available in
days
days
after you enroll
Section 16: Load Testing Our Machine Learning Model
Available in
days
days
after you enroll
Section 17: Production Grade Deployment of Our Machine Learning Model
Available in
days
days
after you enroll
Section 18: Cleaning Up Resources
Available in
days
days
after you enroll
Part 2: Fine-Tuning LLMs with QLoRA, AWS, and Open Source
Available in
days
days
after you enroll
Setting Up AWS Sagemaker Environment
Available in
days
days
after you enroll
Gathering, Chunking, Tokenizing and Uploading our Dataset
Available in
days
days
after you enroll
- Sagemaker Sessions, Regions, and IAM Roles (7:50)
- Examining Our Dataset from HuggingFace (13:29)
- Tokenization and Word Embeddings (9:08)
- HuggingFace Authentication with Sagemaker (4:21)
- Applying the Templating Function to our Dataset (8:43)
- Attention Masks and Padding (15:55)
- Star Unpacking with Python (4:03)
- Chain Iterator, List Constructor and Attention Mask example with Python (10:22)
- Understanding Batching (8:11)
- Slicing and Chunking our Dataset (7:31)
- Creating our Custom Chunking Function (16:06)
- Tokenizing our Dataset (9:30)
- Running our Chunking Function (4:30)
- Understanding the Entire Chunking Process (8:32)
- Uploading the Training Data to AWS S3 (5:53)
- Course Check-In
Understanding LoRA and Setting up HuggingFace Estimator
Available in
days
days
after you enroll
- Setting Up Hyperparameters for the Training Job (6:47)
- Creating our HuggingFace Estimator in Sagemaker (6:45)
- Introduction to Low-rank adaptation (LoRA) (8:11)
- LoRA Numerical Example (10:55)
- LoRA Summarization and Cost Saving Calculation (9:08)
- (Optional) Matrix Multiplication Refresher (4:45)
- Understanding LoRA Programatically Part 1 (12:32)
- Understanding LoRA Programatically Part 2 (5:48)
- Unlimited Updates
Improving Training Speed with Bfloat 16
Available in
days
days
after you enroll
Setting up the QLoRA Training Script with Mixed Precision & Double Quantization
Available in
days
days
after you enroll
- Setting up Imports and Libraries for the Train Script (7:19)
- Argument Parsing Function Part 1 (7:56)
- Argument Parsing Function Part 2 (10:54)
- Understanding Trainable Parameters Caveats (14:30)
- Introduction to Quantization (7:35)
- Identifying Trainable Layers for LoRA (7:19)
- Setting up Parameter Efficient Fine Tuning (4:36)
- Implement LoRA Configuration and Mixed Precision Training (10:34)
- Understanding Double Quantization (4:21)
- Creating the Training Function Part 1 (14:14)
- Creating the Training Function Part 2 (7:16)
- Exercise: Imposter Syndrome
- Finishing our Sagemaker Script (5:09)
- Gaining Access to Powerful GPUs with AWS Quotas (5:10)
- Final Fixes Before Training (3:54)
Running our Fine Tuning Script for our LLM
Available in
days
days
after you enroll
Deploying our Fine Tuned LLM
Available in
days
days
after you enroll
Cleaning up Resources
Available in
days
days
after you enroll
Where To Go From Here?
Available in
days
days
after you enroll