Reinforcement Learning with Human Feedback (RLHF)

AI Engineering Bootcamp: Building AI Applications (LLM APIs, LangChain + more)

Section 00 - Introduction

AI Engineering Bootcamp: Building AI Applications with LLM APIs, LangChain + much more! (2:01)
Exercise: Meet Your Classmates and Instructor
ZTM Plugin + Understanding Your Video Player
A Note on Course Prerequisites
Course Resources
Set Your Learning Streak Goal

Section 01 - Virtual Environments in Jupyter Notebook

Using Jupyter Notebook (9:23)
Using Virtual Environments (venv) (10:07)

Section 02 - Essential Python Libraries for AI: Requests & Pandas

Getting Started with the requests and httpx Libraries in Python (8:49)
Handling HTTP Errors (4:38)
Managing HTTP Authentication and Headers (OpenAI API) (9:59)
Setting Up the Environment: Jupyter Notebook and Pandas (3:54)
Introduction to Pandas: Series and DataFrames (6:08)
Importing and Exporting Data: Working with CSV Files (6:37)
Exporting Data to Different Formats: Excel, JSON, SQL, YAML (7:46)
Modifying Data: Adding and Dropping Columns and Rows (6:04)
Accessing Data: Using df.iloc[] and df.loc[] (5:42)
Sampling and Previewing Data: Using df.sample() and df.head() (6:14)
Filtering Data: Masks and pandas.Series.between() (7:14)
Sorting Data: Understanding Pandas Sorting Methods (7:10)
Handling Missing Data (4:43)
Aggregations and Grouping Data (4:53)
Project: Analyzing Website Traffic Data (4:32)
Time Series Data Manipulation in Pandas (6:59)
Unlimited Updates

Section 03 - Introduction to LLMs, APIs, and AI Libraries

Foundations of LLMs and Generative AI (8:31)
Tokens, Context Windows and Cost (5:25)
Exploring LLM APIs: AI as a Service (9:22)
OpenAI Playground, Google AI Studio, and Anthropic Workbench (6:05)
Challenges and Limitations of LLMs (9:02)
The State of AI: Present and Future – The Good and the Bad (10:05)
Implement a New Life System

Section 04 - Deep Dive into LLMs

Pretraining Data (Internet) (6:40)
Tokenization (6:06)
Training the Neural Network (9:25)
Post-Training: Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) (8:25)
Reinforcement Learning (RL) (5:29)
Becoming Better than Humans: AGI and ASI with RL (7:31)
Reinforcement Learning with Human Feedback (RLHF) (6:22)
How to Deal With Hallucinations (7:36)
Using Tools: Internet Search, Interpreter, and Deep Search (7:48)
Big Ideas Recap (Core Summary) (9:51)

Section 05 - Diving into OpenAI API with Python

Authenticating to OpenAI using Python Dotenv (8:16)
Chat Completions Endpoint (6:57)
Developer Message (4:30)
Streaming API Responses (4:30)
Using Local Base64 Images as Input (6:43)
Using Online Images as Input (2:04)
Chat Completion API Parameters: Temperature and Seed (6:13)
Chat Completion API Parameters: Top P, Max_Tokens, Penalties (9:49)
Diving into OpenAI’s Reasoning Models (o1 and o3) (7:55)
Best Practices for Prompting Reasoning Models (5:25)
Transcriptions with Whisper (5:47)
Translations with Whisper (3:11)
Text-to-Speech (TTS) API (7:02)
Generating Original Images Using the DALL-E 3 (10:49)
Creating Variations of Images with DALL-E (3:04)
Editing Images with DALL-E (5:39)
Course Check-In

Section 06 - Prompt Engineering for Generative AI

Intro to Prompt Engineering (2:40)
Tactic 1: Position Instruction Clearly with Delimiters (4:12)
Tactic 2: Provide Detailed Instructions for the Context (6:37)
Tactic 3: Use the Rich Text Format (RTF) (7:45)
Tactic 4: Few Shot Prompting (8:12)
Tactic 5: Specify the Steps Required to Complete a Task (5:16)
Tactic 6: Give Models Time to Think (2:12)
Other Tactics and Principles for Better Prompting (5:37)
Avoid Hallucinations Using Guarding (3:06)
Summary (2:06)

Section 07 - OpenAI API Project: Building a Healthy Daily Meal Plan

Project Introduction (2:31)
Creating a Daily Meal Plan Using OpenAI API (5:38)
Creating the Prompt (8:42)
Running the Program (3:23)
Generating Original Images for the Recipes using DALL-E (11:53)
Narrate the Meals using the Text-to-Speech Model (10:23)

Section 08 - Diving into Google’s Gemini API

Setting Up the Python SDK and Authenticating for Gemini API (9:50)
Generating Text From Text Prompts (4:14)
Streaming Gemini Responses (2:58)
Generating Text From Images (5:48)
Gemini API Generation Parameters: Controlling How the Model Generates Responses (6:11)
Gemini API Generation Parameters Explained (10:13)
Building Chat Conversations (7:53)
Project: Building a Conversational Agent Using Gemini Pro (7:18)
System Instructions (5:42)
The File API: Prompting with Media Files (6:08)
Tokens (6:41)
Prompting with Audio (4:20)

Section 09 - Gemini API Project: Talking With an Image

Project Requirements (5:53)
Building the Application (5:22)
Testing the Application (1:48)
Streamlit: Transform Your Jupyter Notebooks into Interactive Web Apps (2:48)
Creating the Web App Layout With Streamlit (11:19)
Saving and Displaying the History Using the Streamlit Session State (5:19)
Exercise: Imposter Syndrome (2:55)

Section 10 - Gemini API Project: Building an AI-Powered Image Renaming Tool

Project Introduction (0:56)
Getting Images Using a Generator (6:17)
Renaming Images Using Gemini (9:34)

Section 11 - Diving into LangChain

LangChain Demo (5:05)
Introduction to LangChain (5:09)
Working with the OpenAI Models (8:42)
Caching LLM Responses (4:56)
LLM Streaming (2:57)
Prompt Templates (5:35)
ChatPromptTemplate (5:54)
Understanding Chains (7:47)
Installing the Python Libraries for Gemini and Authenticating to Gemini (4:30)
Integrating Gemini with LangChain (6:01)
Using a System Prompt and Enabling Streaming (6:31)
Multimodal AI With Gemini (14:12)
LangChain Tools: DuckDuckGo and Wikipedia (11:07)
Creating a React Agent (13:29)
Testing the React Agent (4:49)

Section 12 - Diving into Embeddings

Intro to OpenAI's Text Embeddings (3:15)
Generating Simple Embeddings (5:53)
Embedding the Dataset for Similarity Searches (4:51)
Estimating Embedding Costs With Tiktoken (5:11)
Performing Semantic Searches (7:04)

Section 13 - RAG Project: Q&A Application on Your Private Documents (Pinecone and Chroma)

Project Introduction (6:08)
Loading Your Custom (Private) PDF Documents (7:27)
Loading Different Document Formats (5:12)
Public and Private Service Loaders (4:37)
Chunking Strategies and Splitting the Documents (6:38)
Intro to Vector Stores and Authenticating to Pinecone (9:02)
Working with Pinecone Indexes (9:31)
Working with Vectors (8:42)
Pinecone Namespaces (6:43)
Embedding and Uploading to a Vector Database (Pinecone) (13:52)
Asking and Getting Answers (11:42)
Using Chroma as a Vector DB (11:10)
Adding Memory to the RAG System (Chat History) (9:25)
Using a Custom Prompt (8:09)

Section 14 - Building a Simple ReAct Agent from Scratch

Introduction to Agents and ReAct (4:19)
Creating the Agent Class (2:41)
Creating the ReAct Prompt (2:30)
Creating the Tools (2:40)
Testing the Agent (6:05)
Automating the Agent (7:00)

Section 15 - Diving into LangGraph

LangGraph Concepts and Core Components (5:42)
Building a Chatbot (5:29)
Visualizing the Graph (2:12)
Running the Chatbot (1:31)
Tavily AI (8:28)
Enhancing the ChatBot with Tools (8:16)
Adding Memory to the Chatbot (7:05)

Section 16 - Project: Tweet Generator (Reflection)

Intro to Reflection (2:13)
Generate (4:15)
Reflect and Repeat (2:32)
Define the Graph - Part 1 (3:43)
Define the Graph - Part 2 (2:48)
Running the App (3:54)

Section 17 - LangSmith: Platform for Building Production-grade LLM Apps

Intro to LangSmith (3:28)
Setting Up LangSmith (1:54)
Tracing with LangSmith (6:16)
Tracing the Reflective Agentic App (3:50)

Section 18 - Project: Essay Writer (Reflexion)

Project Overview (1:47)
Defining the Agent State and the Prompts (7:38)
Implementing Agents and Nodes (9:38)
Defining the Conditional Edge (1:26)
Defining the Graph (4:25)
Running the App (4:06)
Tracing the App with LangSmith (2:50)

Section 19 - Master Project: Build a Research Agent with LangGraph, GPT-4o, RAG, Pinecone, ArXiv, and Google SerpAPI

Note (2:15)
Application Overview (3:33)
Extracting Data from ArXiv with Pandas (12:43)
Downloading Research Papers (4:52)
Loading, Splitting and Expanding Data (9:53)
Building a Knowledge Base for RAG (5:34)
Creating a Pinecone Index (7:16)
Loading the Knowledge Base and Deploying to Pinecone (5:03)
Developing Custom Tools (5:12)
Implementing the ArXiv Fetch Tool (8:00)
Unlocking Web Search with Google SerpAPI (3:28)
Building Google SerpAPI Tools (4:25)
Creating RAG Tools (6:19)
Implementing the Final Answer Generation Tool (2:17)
06_14 Initializing the Oracle LLM (11:01)
Testing the Ecosystem (3:32)
Building a Decision-Making Pipeline (8:33)
Defining the Agent State (3:24)
Defining the Graph (6:35)
Generating Reports (4:26)
Building the Final Research Report (5:19)
Concluding the Project (6:22)

Appendix - Working with Python Modules

Understanding Python Modules (6:16)
The OS Module (7:56)
Advanced Import Techniques and Best Practices (4:10)
Using __name__ == '__main__' for Modular and Reusable Code (6:23)
Mastering Python Package Management with pip (8:34)
Let's Have Some Fun (+ More Resources)

Where To Go From Here?

Thank You! (1:17)
Review This Course!
Become An Alumni
Learning Guideline
ZTM Events Every Month
LinkedIn Endorsements

Reinforcement Learning with Human Feedback (RLHF)

This lecture is available exclusively for ZTM Academy members.

If you're already a member, you'll need to login.

Join ZTM To Unlock All Lectures