You are NOT signed in as a student! Your progress will not be saved and you won't see milestones on your ZTM Passport.
Make sure you are logged into
Academy
Autoplay
Autocomplete
Previous Lesson
Complete and Continue
Prompt Engineering Bootcamp (Working With LLMs): Zero to Mastery
Section 0: Introduction
Learn To Work With LLMs with Scott Kerr (1:00)
Course Introduction (6:19)
Course Resources + Handbook
Exercise: Meet Your Classmates and Instructor
ZTM Plugin + Understanding Your Video Player
Set Your Learning Streak Goal
Section 1: Introduction to Prompt Engineering
What is Prompt Engineering? (6:24)
Why is Prompt Engineering Even a Thing? (8:31)
Breaking GPT (4:02)
Applied Prompt Engineering (4:24)
Applied Prompt Engineering with NASA (9:29)
Why is Prompt Engineering Important to You? (4:06)
The Latest: Am I Wrong?
Mandatory Homework: AlphaGo
Let's Have Some Fun (+ More Resources)
Section 2: Choose Your LLM
What I'm Using - Part 1 (1:21)
What I'm Using - Part 2 (OpenAI Playground) (4:02)
Optional: Setting Up Your Playground
Multi-Modality and Tools in LLMs (11:41)
Choose Your LLM!
ChatGPT Setup
Getting Started with ChatGPT (1:39)
The Basics of ChatGPT (4:08)
ChatGPT App (1:11)
Optional: ChatGPT Plus (1:55)
Claude Setup
Getting Started with Claude (4:27)
Open Source Option: Llama Setup
Getting Started with Llama
Section 3: Guided Project - Build Your First Game (Snake Game)
Project Introduction (2:13)
Setting Up Your Replit Account (4:44)
Important Point #1 - Fundamental Truths About LLMs (2:58)
Important Point #2 - Models are Non-Deterministic (4:03)
The First Try (4:41)
Building Our Snake Game - Part 1 (6:12)
Building Our Snake Game - Part 2 (5:49)
Exercise: Make It Prettier!
Unlimited Updates
Section 4: How LLMs Work
Introduction to LLMs (5:44)
Tokens (5:17)
Word Guessing Machines? (5:50)
Thinking Like LLMs - Roll a Dice (6:56)
Inside LLMs (22:13)
The Transformer Model (7:38)
GPT Meaning
Exercise: Visualize the LLM Architecture (3:56)
The Training Process (9:57)
Base Model vs. Assistant Model (7:06)
Thinking Like LLMs - The Reversal Curse (5:14)
Exercise: The Reversal Curse
Artificial General Intelligence (AGI) (5:28)
Exercise: Use ChatGPT to Read the Research (4:48)
The World of LLMs (6:05)
Implement a New Life System
Section 5: Challenge - Build Your Second Game (Tic Tac Toe with AI opponent)
Challenge Instructions
Section 6: Our Prompting Framework
Overview of Our Prompting Framework (3:30)
The Standard Prompt (4:33)
Prompt Library
Exercise: Get Hyped to Learn! (4:04)
Section 7: Prompting Fundamentals - The Setup
The Setup (1:01)
The System Message - Part 1 (6:07)
The System Message - Part 2 (5:43)
Exercise: Strengthen Your System Message (6:42)
Claude 3's System Message
Context - What it Is (5:17)
Context - The Context Window (9:21)
Context - Lost in the Middle (11:26)
The Latest: Gemini 1.5's Context Window
Personas and Roles (8:42)
Personas - Tone, Style and Voice (11:49)
Exercise: Talk To Your Hero
Exercise: Write a Movie Scene
Custom Instructions (11:14)
The System Message for Custom Instructions
Thinking Like LLMs - Can GPT Keep a Secret? (4:45)
Exercise: Get ChatGPT to Spill the Tea - Part 1 (1:44)
Exercise: Get ChatGPT to Spill the Tea - Part 2 (1:42)
Exercise: Get ChatGPT to Spill the Tea - Part 3 (4:24)
Exercise: Test Techniques for Securing Secrets
Exercise: Stump Me!
Case Study: Moderna's Use of ChatGPT
Course Check-In
Section 8: Prompting Fundamentals - The Instructions
The User Message (2:09)
Be Clear and Specific (3:52)
Delimiters (5:15)
Exercise: Identify Delimiters (3:25)
X-Y Problem (6:10)
Exercise: Identify X
Solution: Identify X
In-Context Training (1:01)
Zero Shot Prompting (1:47)
One Shot and Few Shot Prompting (9:48)
Language Models Are Few-Shot Learners (3:48)
How Many Examples? (2:23)
Thinking Like LLMs - But Wait... (5:36)
Chain-of-Thought Prompting - Part 1 (5:24)
Chain-of-Thought Prompting - Part 2 (5:45)
Zero Shot CoT (9:29)
Exercise: Imposter Syndrome (2:55)
Section 9: Guided Project - Create Your Own Career Coach
Career Coach Ideation (7:32)
The Setup - Persona (6:02)
The Setup - Context and Commands (5:42)
The Instructions - Modes (12:20)
The Instructions - Chain of Thought (3:51)
The Instructions - Gamify (9:01)
Meet Your Career Coach (19:53)
Exercise: Customize Your Career Coach
Section 10: Prompting Fundamentals - The Output
The Output (1:35)
Length (12:33)
Formats (7:42)
Exercise: Output an Excel File (2:30)
Exercise: Make a Flowchart (4:01)
Section 11: The Dark Side of AI (Guardrails, Jailbreaks, Injections, Hallucinations + more)
Introduction to Guardrails and Jailbreaking (16:32)
Jailbreak! (The DAN Prompt) (7:25)
Many Shot Jailbreaking (18:09)
Prompt Injections - Part 1 (9:36)
Prompt Injections - Part 2 (17:42)
Thinking Like LLMs - Multi-Modal Injection (9:17)
Leaking - Part 1 (Prompt Leaking) (8:35)
Leaking - Part 2 (Data Leaking) (18:07)
Exposure (5:40)
Poisoning (3:18)
Toxicity (4:39)
Hallucinations (13:31)
Thinking Like LLMs - Big vs Small (18:58)
Challenge: Conduct Your Own Mechanistic Interpretability Research on Hallucinations (4:34)
Challenge Instructions
Leaderboard: Mechanistic Interpretability
The Model Card (11:05)
Model Cards Deep Dive (14:43)
Exercise: Explore the Model Card for GPT-o3-mini and Learn Something New!
Section 12: LLM Hyperparameters & The OpenAI Playground
Introduction to Hyperparameters and the OpenAI Playground (3:15)
Note: Setting Up Your Playground
Temperature (10:25)
Top P (4:08)
Frequency and Presence Penalties (4:38)
Stop Sequences (5:22)
Section 13: Open Source Models
What Are Open Source LLMs and Why Are They Important? (6:50)
Chatbot Arena Leaderboard (7:52)
Battle of the Intelligence Tests (14:38)
Exercise: Test The Models With Your Specific Use-Case
Exercise: Create Your Own Test Prompt (Totem) (1:35)
My Totem Prompt
The Latest - The Best Open Source Model
Introduction to LMStudio (2:37)
Setting Up Your Own Model - Part 1 (2:51)
Setting Up Your Own Model - Part 2 (9:38)
Exercise: Explore the Guardrails
Section 14: Advanced Prompting Techniques
Introduction (1:06)
Auto-Priming (10:50)
Chain of Density Prompting (18:16)
Prompt Variables (7:13)
Prompt Chaining (8:38)
Prompt Chaining - Programmatic Visualization (17:40)
Exercise: Prompt Chaining - Customer Support (22:33)
Thinking Like LLMs - Dark Magic (9:57)
Exercise: Use Your Magic Powers
XML Tags - There's More Than Meets The Eye! (9:17)
Emotional Stimuli - Part 1 (18:36)
Emotional Stimuli - Part 2 (4:42)
Emotional Stimuli - Part 3 (But Why?) (7:17)
Self-Consistency (13:44)
Exercise: Implement the Self-Consistency Technique
ReAct Prompting (21:34)
ReAct + CoT-SC (16:47)
Applied Prompt Engineering with CRISPR - Part 1 (13:40)
Applied Prompt Engineering with CRISPR - Part 2 (14:52)
Tree of Thoughts - Part 1 (14:56)
Tree of Thoughts - Part 2 (4:56)
Tree of Thoughts - Part 3 (ToT via Code) (11:33)
Tree of Thoughts - Part 4 (ToT via Chaining) (22:11)
Tree of Thoughts - Part 5 ("Zero Shot" ToT) (12:26)
Section 15: Challenge - Build Your Third Game (Flappy Bird)
Challenge Instructions
The Flappy Bird Hall of Fame
Section 16: Prompt Testing and Model Benchmarks
Introduction to Prompt Testing (8:32)
The Importance of Prompt Testing (4:32)
Thinking Like LLMs - Schrodinger's Prompt (19:17)
Building a Prompt Test (16:49)
The Golden Answer (7:00)
Model Benchmarks (11:29)
Deep Dive: MMLU Benchmark (14:19)
Prompt Tests vs. Model Benchmarks? (4:31)
The Latest: MMLU Pro (18:10)
Exercise: Create Your Own Mini-Benchmark (Totem Benchmark)!
Evaluating Results - Human Judge (7:03)
Evaluating Results - Code Judge (11:23)
Evaluating Results - AI Judge (22:06)
Deep Dive: LLMs as a Judge - Biases (22:11)
Deep Dive: LLMs as a Judge - Prompts (13:01)
LLM Judge Leaderboard
Thinking Like LLMs - Ex Post Facto Reasoning (11:46)
Section 17: Applied Prompt Engineering - Developing Prompt Tests & Evaluations with PromptFoo
Here. We. Go! (6:01)
Introduction to PromptFoo (6:10)
Same As The Pros!
Our Prompt Testing Framework (4:45)
Warning!
Our 1st Prompt Test (Adding Prompts, Metrics + Human Judge) - The Setup (23:09)
Our 1st Prompt Test (Adding Prompts, Metrics + Human Judge) - The Results (10:20)
Housekeeping Matters
Our 2nd Prompt Test (Adding New Prompts + Code Judge) - The Setup (11:40)
Our 2nd Prompt Test (Adding New Prompts + Code Judge) - The Results (7:38)
Our 3rd Prompt Test (Adding 152 Test Cases + AI Judge) - The Setup (25:36)
Our 3rd Prompt Test (Adding 152 Test Cases + AI Judge) - The Results (8:21)
Optional Exercise: Practice Converting Another MMLU Dataset
Exercise: Explore Metrics (Assertions) in the PromptFoo Documentation
Exercise: Use Your Mini-Benchmark (Totem Benchmark)!
Our 4th Prompt Test (Adding Multiple Models) - The Setup (25:58)
Our 4th Prompt Test (Adding Multiple Models) - The Results (13:11)
Our 5th Prompt Test (Adding System Messages) - The Setup (24:31)
Our 5th Prompt Test (Adding System Messages) - The Results (12:22)
Do You Realize What Just Happened? (4:25)
Section 18: Challenge - Become an AI Researcher! (Develop Your Own Prompting Technique)
You Can Wait...Or You Can Get Started Now!
Section 19: AI Research and the Quest for Artificial General Intelligence
Introduction - Through The Looking Glass... (8:40)
Applied Prompt Engineering with The Turing Test (7:33)
Homework: Ex Machina
Mechanistic Interpretability - Part 1 (13:24)
Mechanistic Interpretability - Part 2 (20:59)
Scaling Laws - Model Size (12:55)
Scaling Laws - Dataset Size (8:04)
Scaling Laws - Training Compute (11:49)
Exercise: Current AI Trends
Homework: Situational Awareness - The Decade Ahead
Where To Go From Here?
What's Next?
Thank You! (1:17)
Review This Course!
Become An Alumni
Learning Guideline
ZTM Events Every Month
LinkedIn Endorsements
Optional Appendix: Guided Project - Prompting with Autonomous Agents (AutoGPT)
Project Introduction
A Quick Note
Setup Instructions
Setup Demo: Getting Your API Key (2:23)
Setup Demo: Downloading AutoGPT (1:03)
Setup Demo: Installing Docker (3:26)
Launching Your Autonomous Agent - Part 1 (3:25)
Launching Your Autonomous Agent - Part 2 (1:21)
It's Aliiiiiive! Running Your Autonomous Agent (4:51)
Task 1: Hello World - Your First Website (16:17)
Task 2: Python Program - Palindrome Checker (8:40)
Task 3: Initialize Your Own Task
You Can Wait...Or You Can Get Started Now!
This lecture is available exclusively for ZTM Academy members.
If you're already a member,
you'll need to login
.
Join ZTM To Unlock All Lectures