Setting Up Your Replit Account

Prompt Engineering Bootcamp (Working With LLMs): Zero to Mastery

Section 0: Introduction

Learn To Work With LLMs with Scott Kerr (1:00)
Course Introduction (6:19)
Course Resources + Handbook
Exercise: Meet Your Classmates and Instructor
ZTM Plugin + Understanding Your Video Player
Set Your Learning Streak Goal

Section 1: Introduction to Prompt Engineering

What is Prompt Engineering? (6:24)
Why is Prompt Engineering Even a Thing? (8:31)
Breaking GPT (4:02)
Applied Prompt Engineering (4:24)
Applied Prompt Engineering with NASA (9:29)
Why is Prompt Engineering Important to You? (4:06)
The Latest: Am I Wrong?
Homework: AlphaGo
Let's Have Some Fun (+ More Resources)

Section 2: Choose Your LLM

What I'm Using - Part 1 (1:21)
What I'm Using - Part 2 (OpenAI Playground) (4:02)
Optional: Setting Up Your Playground
Multi-Modality and Tools in LLMs (11:41)
Choose Your LLM!

ChatGPT Setup

Getting Started with ChatGPT (1:39)
The Basics of ChatGPT (4:08)
ChatGPT App (1:11)
Optional: ChatGPT Plus (1:55)

Claude Setup

Getting Started with Claude (4:27)

Open Source Option: Llama Setup

Getting Started with Llama

Section 3: Guided Project - Build Your First Game (Snake Game)

Project Introduction (2:13)
Setting Up Your Replit Account (4:44)
Two Important Points
The First Try (4:41)
Building Our Snake Game - Part 1 (6:12)
Building Our Snake Game - Part 2 (5:49)
Exercise: Make It Prettier!
Unlimited Updates

Section 4: How LLMs Work

Introduction to LLMs (5:44)
Tokens (5:17)
Word Guessing Machines? (5:50)
Thinking Like LLMs - Roll a Dice (6:56)
Inside LLMs (22:13)
The Transformer Model (7:38)
GPT Meaning
Exercise: Visualize the LLM Architecture (3:56)
The Training Process (9:57)
Base Model vs. Assistant Model (7:06)
Thinking Like LLMs - The Reversal Curse (5:14)
Exercise: The Reversal Curse
Artificial General Intelligence (AGI) (5:28)
Exercise: Use ChatGPT to Read the Research (4:48)
The World of LLMs (6:05)
Implement a New Life System

Section 5: Challenge - Build Your Second Game (Tic Tac Toe with AI opponent)

Challenge Instructions

Section 6: Our Prompting Framework

Overview of Our Prompting Framework (3:30)
The Standard Prompt (4:33)
Prompt Library
Exercise: Get Hyped to Learn! (4:04)

Section 7: Prompting Fundamentals - The Setup

The Setup (1:01)
The System Message - Part 1 (6:07)
The System Message - Part 2 (5:43)
Exercise: Strengthen Your System Message (6:42)
Claude 3's System Message
Context - What it Is (5:17)
Context - The Context Window (9:21)
Context - Lost in the Middle (11:26)
The Latest: Gemini 1.5's Context Window
Personas and Roles (8:42)
Personas - Tone, Style and Voice (11:49)
Exercise: Talk To Your Hero
Exercise: Write a Movie Scene
Custom Instructions (11:14)
The System Message for Custom Instructions
Thinking Like LLMs - Can GPT Keep a Secret? (4:45)
Exercise: Get ChatGPT to Spill the Tea - Part 1 (1:44)
Exercise: Get ChatGPT to Spill the Tea - Part 2 (1:42)
Exercise: Get ChatGPT to Spill the Tea - Part 3 (4:24)
Exercise: Test Techniques for Securing Secrets
Exercise: Stump Me!
Case Study: Moderna's Use of ChatGPT
Course Check-In

Section 8: Prompting Fundamentals - The Instructions

The User Message (2:09)
Be Clear and Specific (3:52)
Delimiters (5:15)
Exercise: Identify Delimiters (3:25)
X-Y Problem (6:10)
Exercise: Identify X
Solution: Identify X
In-Context Training (1:01)
Zero Shot Prompting (1:47)
One Shot and Few Shot Prompting (9:48)
Language Models Are Few-Shot Learners (3:48)
How Many Examples? (2:23)
Thinking Like LLMs - But Wait... (5:36)
Chain-of-Thought Prompting - Part 1 (5:24)
Chain-of-Thought Prompting - Part 2 (5:45)
Zero Shot CoT (9:29)
Exercise: Imposter Syndrome (2:55)

Section 9: Guided Project - Create Your Own Career Coach

Career Coach Ideation (7:32)
The Setup - Persona (6:02)
The Setup - Context and Commands (5:42)
The Instructions - Modes (12:20)
The Instructions - Chain of Thought (3:51)
The Instructions - Gamify (9:01)
Meet Your Career Coach (19:53)
Exercise: Customize Your Career Coach

Section 10: Prompting Fundamentals - The Output

The Output (1:35)
Length (12:33)
Formats (7:42)
Exercise: Output an Excel File (2:30)
Exercise: Make a Flowchart (4:01)
Jailbreak! (7:25)
Thinking Like LLMs - Prompt Injection (9:17)

Section 11: LLM Hyperparameters & The OpenAI Playground

Introduction to Hyperparameters and the OpenAI Playground (3:15)
Note: Setting Up Your Playground
Temperature (10:25)
Top P (4:08)
Frequency and Presence Penalties (4:38)
Stop Sequences (5:22)

Section 12: Guided Project - Prompting with Autonomous Agents (AutoGPT)

Project Introduction
A Quick Note
Setup Instructions
Setup Demo: Getting Your API Key (2:23)
Setup Demo: Downloading AutoGPT (1:03)
Setup Demo: Installing Docker (3:26)
Launching Your Autonomous Agent - Part 1 (3:25)
Launching Your Autonomous Agent - Part 2 (1:21)
It's Aliiiiiive! Running Your Autonomous Agent (4:51)
Task 1: Hello World - Your First Website (16:17)
Task 2: Python Program - Palindrome Checker (8:40)
Task 3: Initialize Your Own Task

Section 13: Open Source Models

What Are Open Source LLMs and Why Are They Important? (6:50)
Chatbot Arena Leaderboard (7:52)
Battle of the Intelligence Tests (14:38)
Exercise: Test The Models With Your Specific Use-Case
Exercise: Create Your Own Test Prompt (Totem) (1:35)
My Totem Prompt
The Latest - The Best Open Source Model
Introduction to LMStudio (2:37)
Setting Up Your Own Model - Part 1 (2:51)
Setting Up Your Own Model - Part 2 (9:38)
Exercise: Explore the Guardrails

Section 14: Advanced Prompting Techniques

Introduction (1:06)
Auto-Priming (10:50)
Chain of Density Prompting (18:16)
Prompt Variables (7:13)
Prompt Chaining (8:38)
Prompt Chaining - Programmatic Visualization (17:40)
Exercise: Prompt Chaining - Customer Support (22:33)
Exercise: Create Your Own Prompt Chain!
Thinking Like LLMs - Dark Magic (9:57)
Exercise: Use Your Magic Powers
XML Tags - There's More Than Meets The Eye! (9:17)
Emotional Stimuli - Part 1 (18:36)
Emotional Stimuli - Part 2 (4:42)
Emotional Stimuli - Part 3 (But Why?) (7:17)
Self-Consistency (13:44)
Exercise: Implement the Self-Consistency Technique
ReAct Prompting (21:34)
ReAct + CoT-SC (16:47)
Applied Prompt Engineering with CRISPR - Part 1 (13:40)
Applied Prompt Engineering with CRISPR - Part 2 (14:52)
Tree of Thoughts - Part 1 (14:56)
Tree of Thoughts - Part 2 (4:56)
Tree of Thoughts - Part 3 (ToT via Code) (11:33)
Tree of Thoughts - Part 4 (ToT via Chaining) (22:11)
Tree of Thoughts - Part 5 ("Zero Shot" ToT) (12:26)

Section 15: Challenge - Build Your Third Game (Flappy Bird)

Challenge Instructions
The Flappy Bird Hall of Fame

Section 16: Introduction to Prompt Testing and Model Benchmarks

Introduction to Prompt Testing (8:32)
The Importance of Prompt Testing (4:32)
Thinking Like LLMs - Schrodinger's Prompt (19:17)
Building a Prompt Test (16:49)
The Golden Answer (7:00)
Model Benchmarks (11:29)
Deep Dive: MMLU Benchmark (14:19)
Prompt Tests vs. Model Benchmarks? (4:31)
The Latest: MMLU Pro (18:10)
Exercise: Create Your Own Mini-Benchmark (Totem Benchmark)!
Evaluating Results - Human Judge (7:03)
Evaluating Results - Code Judge (11:23)
Evaluating Results - AI Judge (22:06)
Deep Dive: LLMs as a Judge - Biases (22:11)
Deep Dive: LLMs as a Judge - Prompts (13:01)
LLM Judge Leaderboard
Thinking Like LLMs - Ex Post Facto Reasoning (11:46)

Section 17: Applied Prompt Engineering - Developing Prompt Tests & Evaluations with PromptFoo

Here. We. Go! (6:01)
Introduction to PromptFoo (6:10)
Same As The Pros!
Our Prompt Testing Framework (4:45)
Warning!
Our 1st Prompt Test (Adding Prompts, Metrics + Human Judge) - The Setup (23:09)
Our 1st Prompt Test (Adding Prompts, Metrics + Human Judge) - The Results (10:20)
Housekeeping Matters
Our 2nd Prompt Test (Adding New Prompts + Code Judge) - The Setup (11:40)
Our 2nd Prompt Test (Adding New Prompts + Code Judge) - The Results (7:38)
Our 3rd Prompt Test (Adding 152 Test Cases + AI Judge) - The Setup (25:36)
Our 3rd Prompt Test (Adding 152 Test Cases + AI Judge) - The Results (8:21)
Optional Exercise: Practice Converting Another MMLU Dataset
Exercise: Explore Metrics (Assertions) in the PromptFoo Documentation
Exercise: Use Your Mini-Benchmark (Totem Benchmark)!
Our 4th Prompt Test (Adding Multiple Models) - The Setup (25:58)
Our 4th Prompt Test (Adding Multiple Models) - The Results (13:11)
Our 5th Prompt Test (Adding System Messages) - The Setup (24:31)
Our 5th Prompt Test (Adding System Messages) - The Results (12:22)
Do You Realize What Just Happened? (4:25)

Section 18: Challenge - Become an AI Researcher! (Develop Your Own Prompting Technique)

You Can Wait...Or You Can Get Started Now!

Section 19: AI Research and the Quest for Artificial General Intelligence

Introduction - Through The Looking Glass... (8:40)
Applied Prompt Engineering with The Turing Test (7:33)
Homework: Ex Machina
Mechanistic Interpretability - Part 1 (13:24)
Mechanistic Interpretability - Part 2 (20:59)
Scaling Laws - Model Size (12:55)
Scaling Laws - Dataset Size (8:04)
Scaling Laws - Training Compute (11:49)
Exercise: Current AI Trends
Homework: Situational Awareness - The Decade Ahead

Where To Go From Here?

What's Next?
Thank You! (1:17)
Review This Course!
Become An Alumni
Learning Guideline
ZTM Events Every Month
LinkedIn Endorsements

Setting Up Your Replit Account

This lecture is available exclusively for ZTM Academy members.

If you're already a member, you'll need to login.

Join ZTM To Unlock All Lectures