Developing Custom Tools

AI Engineering Bootcamp: Building AI Applications (LLM APIs, LangChain + more)

Section 00 - Introduction

AI Engineering Bootcamp: Building AI Applications with LLM APIs, LangChain + much more! (2:01)
Exercise: Meet Your Classmates and Instructor
ZTM Plugin + Understanding Your Video Player
A Note on Course Prerequisites
Course Resources
Set Your Learning Streak Goal

Section 01 - Virtual Environments in Jupyter Notebook

Using Jupyter Notebook (9:23)
Using Virtual Environments (venv) (10:07)

Section 02 - Essential Python Libraries for AI: Requests & Pandas

Getting Started with the requests and httpx Libraries in Python (8:49)
Handling HTTP Errors (4:38)
Managing HTTP Authentication and Headers (OpenAI API) (9:59)
Setting Up the Environment: Jupyter Notebook and Pandas (3:54)
Introduction to Pandas: Series and DataFrames (6:08)
Importing and Exporting Data: Working with CSV Files (6:37)
Exporting Data to Different Formats: Excel, JSON, SQL, YAML (7:46)
Modifying Data: Adding and Dropping Columns and Rows (6:04)
Accessing Data: Using df.iloc[] and df.loc[] (5:42)
Sampling and Previewing Data: Using df.sample() and df.head() (6:14)
Filtering Data: Masks and pandas.Series.between() (7:14)
Sorting Data: Understanding Pandas Sorting Methods (7:10)
Handling Missing Data (4:43)
Aggregations and Grouping Data (4:53)
Project: Analyzing Website Traffic Data (4:32)
Time Series Data Manipulation in Pandas (6:59)
Unlimited Updates

Section 03 - Introduction to LLMs, APIs, and AI Libraries

Foundations of LLMs and Generative AI (8:31)
Tokens, Context Windows and Cost (5:25)
Exploring LLM APIs: AI as a Service (9:22)
OpenAI Playground, Google AI Studio, and Anthropic Workbench (6:05)
Challenges and Limitations of LLMs (9:02)
The State of AI: Present and Future – The Good and the Bad (10:05)
Implement a New Life System

Section 04 - Deep Dive into LLMs

Pretraining Data (Internet) (6:40)
Tokenization (6:06)
Training the Neural Network (9:25)
Post-Training: Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) (8:25)
Reinforcement Learning (RL) (5:29)
Becoming Better than Humans: AGI and ASI with RL (7:31)
Reinforcement Learning with Human Feedback (RLHF) (6:22)
How to Deal With Hallucinations (7:36)
Using Tools: Internet Search, Interpreter, and Deep Search (7:48)
Big Ideas Recap (Core Summary) (9:51)

Section 05 - Diving into OpenAI API with Python

Authenticating to OpenAI using Python Dotenv (8:16)
Chat Completions Endpoint (6:57)
Developer Message (4:30)
Streaming API Responses (4:30)
Using Local Base64 Images as Input (6:43)
Using Online Images as Input (2:04)
Chat Completion API Parameters: Temperature and Seed (6:13)
Chat Completion API Parameters: Top P, Max_Tokens, Penalties (9:49)
Diving into OpenAI’s Reasoning Models (o1 and o3) (7:55)
Best Practices for Prompting Reasoning Models (5:25)
Transcriptions with Whisper (5:47)
Translations with Whisper (3:11)
Text-to-Speech (TTS) API (7:02)
Generating Original Images Using the DALL-E 3 (10:49)
Creating Variations of Images with DALL-E (3:04)
Editing Images with DALL-E (5:39)
Course Check-In

Section 06 - Prompt Engineering for Generative AI

Intro to Prompt Engineering (2:40)
Tactic 1: Position Instruction Clearly with Delimiters (4:12)
Tactic 2: Provide Detailed Instructions for the Context (6:37)
Tactic 3: Use the Rich Text Format (RTF) (7:45)
Tactic 4: Few Shot Prompting (8:12)
Tactic 5: Specify the Steps Required to Complete a Task (5:16)
Tactic 6: Give Models Time to Think (2:12)
Other Tactics and Principles for Better Prompting (5:37)
Avoid Hallucinations Using Guarding (3:06)
Summary (2:06)

Section 07 - OpenAI API Project: Building a Healthy Daily Meal Plan

Project Introduction (2:31)
Creating a Daily Meal Plan Using OpenAI API (5:38)
Creating the Prompt (8:42)
Running the Program (3:23)
Generating Original Images for the Recipes using DALL-E (11:53)
Narrate the Meals using the Text-to-Speech Model (10:23)

Section 08 - Diving into Google’s Gemini API

Setting Up the Python SDK and Authenticating for Gemini API (9:50)
Generating Text From Text Prompts (4:14)
Streaming Gemini Responses (2:58)
Generating Text From Images (5:48)
Gemini API Generation Parameters: Controlling How the Model Generates Responses (6:11)
Gemini API Generation Parameters Explained (10:13)
Building Chat Conversations (7:53)
Project: Building a Conversational Agent Using Gemini Pro (7:18)
System Instructions (5:42)
The File API: Prompting with Media Files (6:08)
Tokens (6:41)
Prompting with Audio (4:20)

Section 09 - Gemini API Project: Talking With an Image

Project Requirements (5:53)
Building the Application (5:22)
Testing the Application (1:48)
Streamlit: Transform Your Jupyter Notebooks into Interactive Web Apps (2:48)
Creating the Web App Layout With Streamlit (11:19)
Saving and Displaying the History Using the Streamlit Session State (5:19)
Exercise: Imposter Syndrome (2:55)

Section 10 - Gemini API Project: Building an AI-Powered Image Renaming Tool

Project Introduction (0:56)
Getting Images Using a Generator (6:17)
Renaming Images Using Gemini (9:34)

Section 11 - Diving into LangChain

LangChain Demo (5:05)
Introduction to LangChain (5:09)
Working with the OpenAI Models (8:42)
Caching LLM Responses (4:56)
LLM Streaming (2:57)
Prompt Templates (5:35)
ChatPromptTemplate (5:54)
Understanding Chains (7:47)
Installing the Python Libraries for Gemini and Authenticating to Gemini (4:30)
Integrating Gemini with LangChain (6:01)
Using a System Prompt and Enabling Streaming (6:31)
Multimodal AI With Gemini (14:12)
LangChain Tools: DuckDuckGo and Wikipedia (11:07)
Creating a React Agent (13:29)
Testing the React Agent (4:49)

Section 12 - Diving into Embeddings

Intro to OpenAI's Text Embeddings (3:15)
Generating Simple Embeddings (5:53)
Embedding the Dataset for Similarity Searches (4:51)
Estimating Embedding Costs With Tiktoken (5:11)
Performing Semantic Searches (7:04)

Section 13 - RAG Project: Q&A Application on Your Private Documents (Pinecone and Chroma)

Project Introduction (6:08)
Loading Your Custom (Private) PDF Documents (7:27)
Loading Different Document Formats (5:12)
Public and Private Service Loaders (4:37)
Chunking Strategies and Splitting the Documents (6:38)
Intro to Vector Stores and Authenticating to Pinecone (9:02)
Working with Pinecone Indexes (9:31)
Working with Vectors (8:42)
Pinecone Namespaces (6:43)
Embedding and Uploading to a Vector Database (Pinecone) (13:52)
Asking and Getting Answers (11:42)
Using Chroma as a Vector DB (11:10)
Adding Memory to the RAG System (Chat History) (9:25)
Using a Custom Prompt (8:09)

Section 14 - Building a Simple ReAct Agent from Scratch

Introduction to Agents and ReAct (4:19)
Creating the Agent Class (2:41)
Creating the ReAct Prompt (2:30)
Creating the Tools (2:40)
Testing the Agent (6:05)
Automating the Agent (7:00)

Section 15 - Diving into LangGraph

LangGraph Concepts and Core Components (5:42)
Building a Chatbot (5:29)
Visualizing the Graph (2:12)
Running the Chatbot (1:31)
Tavily AI (8:28)
Enhancing the ChatBot with Tools (8:16)
Adding Memory to the Chatbot (7:05)

Section 16 - Project: Tweet Generator (Reflection)

Intro to Reflection (2:13)
Generate (4:15)
Reflect and Repeat (2:32)
Define the Graph - Part 1 (3:43)
Define the Graph - Part 2 (2:48)
Running the App (3:54)

Section 17 - LangSmith: Platform for Building Production-grade LLM Apps

Intro to LangSmith (3:28)
Setting Up LangSmith (1:54)
Tracing with LangSmith (6:16)
Tracing the Reflective Agentic App (3:50)

Section 18 - Project: Essay Writer (Reflexion)

Project Overview (1:47)
Defining the Agent State and the Prompts (7:38)
Implementing Agents and Nodes (9:38)
Defining the Conditional Edge (1:26)
Defining the Graph (4:25)
Running the App (4:06)
Tracing the App with LangSmith (2:50)

Section 19 - Master Project: Build a Research Agent with LangGraph, GPT-4o, RAG, Pinecone, ArXiv, and Google SerpAPI

Note (2:15)
Application Overview (3:33)
Extracting Data from ArXiv with Pandas (12:43)
Downloading Research Papers (4:52)
Loading, Splitting and Expanding Data (9:53)
Building a Knowledge Base for RAG (5:34)
Creating a Pinecone Index (7:16)
Loading the Knowledge Base and Deploying to Pinecone (5:03)
Developing Custom Tools (5:12)
Implementing the ArXiv Fetch Tool (8:00)
Unlocking Web Search with Google SerpAPI (3:28)
Building Google SerpAPI Tools (4:25)
Creating RAG Tools (6:19)
Implementing the Final Answer Generation Tool (2:17)
06_14 Initializing the Oracle LLM (11:01)
Testing the Ecosystem (3:32)
Building a Decision-Making Pipeline (8:33)
Defining the Agent State (3:24)
Defining the Graph (6:35)
Generating Reports (4:26)
Building the Final Research Report (5:19)
Concluding the Project (6:22)

Appendix - Working with Python Modules

Understanding Python Modules (6:16)
The OS Module (7:56)
Advanced Import Techniques and Best Practices (4:10)
Using __name__ == '__main__' for Modular and Reusable Code (6:23)
Mastering Python Package Management with pip (8:34)
Let's Have Some Fun (+ More Resources)

Where To Go From Here?

Thank You! (1:17)
Review This Course!
Become An Alumni
Learning Guideline
ZTM Events Every Month
LinkedIn Endorsements

Developing Custom Tools

This lecture is available exclusively for ZTM Academy members.

If you're already a member, you'll need to login.

Join ZTM To Unlock All Lectures