Get a head start on your coding projects with ourPython Code Generator. Perfect for those times when you need a quick solution. Don't wait, try it today!

This tutorial will guide you step-by-step through building a full-stack Retrieval-Augmented Generation (RAG) chatbot using FastAPI, OpenAI's language model, and Streamlit. By the end, you will have a working chatbot that can answer questions based on the content of uploaded PDF documents.

Table of Contents:

• Introduction
• Project Structure
• Setting Up the Environment
• Configuring Environment Variables
• Building the Chatbot Logic (chatbot.py)
• Implementing the FastAPI Server (api.py)
• Creating the Streamlit Application (app.py)
• Running the Application
• Demo
• Conclusion
• Further Reading

Introduction

Retrieval-Augmented Generation (RAG) is a powerful approach that combines information retrieval with generative AI models. In this project, we will build a chatbot that can answer user questions based on the content of uploaded PDF documents. The system uses:

• FastAPI for building a RESTful API backend.
• LangChain for chaining together retrieval and generation logic.
• OpenAI for language model and embeddings.
• Chroma as a local vector database for storing and searching document embeddings.
• Streamlit for a simple, interactive web UI.

Project Structure

Your project should have the following structure:

chatbot-rag/├── data/                # Directory to hold the local vector database├── api.py               # FastAPI server├── app.py               # Streamlit web application├── chatbot.py           # Core chatbot logic├── requirements.txt     # Python dependencies├── README.md            # Project documentation└── .env                 # Environment variables (e.g., API keys)

Setting Up the Environment

1. Create a Virtual Environment

A virtual environment isolates your project dependencies. You can usevenv,conda, oruv:

Usingvenv:

python -m venv venvsource venv/bin/activate  # On Windows: venv\Scripts\activate

Usingconda:

conda create -n chatbot-rag python=3.11conda activate chatbot-rag

Usinguv:

uv initsource .venv/bin/activate  # On Windows: .venv\Scripts\activate

2. Install Required Dependencies

Navigate to your project directory and install dependencies:

Withpip:

pip install -r requirements.txt

Withconda:

conda install --file requirements.txt

Withuv:

uv add -r requirements.txtuv sync

Configuring Environment Variables

To keep sensitive information like API keys secure, store them in a.env file. This file should not be committed to version control.

Create a.env file in your project root and add your OpenAI API key:

OPENAI_API_KEY=your_openai_api_key

Building the Chatbot Logic (chatbot.py)

This file contains the core logic for document storage, retrieval, and question answering.

1. Import Required Libraries

We usedotenv for loading environment variables,langchain for chaining logic, andlogging for monitoring.

import osfrom dotenv import load_dotenv, find_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom langchain_chroma import Chromafrom langchain.prompts import ChatPromptTemplatefrom langchain_core.documents.base import Documentfrom langchain.chains import create_retrieval_chainfrom langchain.chains.combine_documents import create_stuff_documents_chainfrom langchain_community.document_loaders.blob_loaders import Blobfrom langchain_community.document_loaders.parsers import PyPDFParserimport logging

2. Set Up Logging

Logging helps you monitor your application and debug issues.

logging.basicConfig(level=logging.INFO)logger = logging.getLogger(__name__)

3. Load Environment Variables and Initialize OpenAI

We load the API key from.env and initialize the embedding and LLM objects.

load_dotenv(find_dotenv())OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")if not OPENAI_API_KEY:    logger.error("OPENAI_API_KEY is not set")    raise ValueError("OPENAI_API_KEY is not set")embeddings = OpenAIEmbeddings(model="text-embedding-3-large", api_key=OPENAI_API_KEY)llm = ChatOpenAI(model="gpt-4o-mini", temperature=0, api_key=OPENAI_API_KEY)

4. Set Up the Vector Database (Chroma)

A RAG system needs a vector database to store document embeddings for efficient similarity search.Chroma is a simple, local vector database.

chroma = Chroma(    collection_name="documents",    collection_metadata={"name": "documents", "description": "store documents"},    persist_directory="./data",    embedding_function=embeddings,)retriever = chroma.as_retriever(search_kwargs={"k": 2})  # Retrieve top 2 relevant docs

5. Define the Prompt Template

The prompt guides the LLM to answer based on the retrieved context.

TEMPLATE = """Here is the context:<context>{context}</context>And here is the question that must be answered using that context:<question>{input}</question>Please read through the provided context carefully. Then, analyze the question and attempt to find adirect answer to the question within the context.If you are able to find a direct answer, provide it and elaborate on relevant points from thecontext using bullet points "-".If you cannot find a direct answer based on the provided context, outline the most relevant pointsthat give hints to the answer of the question.If no answer or relevant points can be found, or the question is not related to the context, simplystate the following sentence without any additional text:i couldnt find an answer did not find an answer to your question.Output your response in plain text without using the tags <answer> and </answer> and ensure you are notquoting context text in your response since it must not be part of the answer."""PROMPT = ChatPromptTemplate.from_template(TEMPLATE)

6. Create the Retrieval and LLM Chains

These chains connect the retriever and the LLM, so that relevant documents are injected into the prompt before generating an answer.

llm_chain = create_stuff_documents_chain(llm, PROMPT)retrieval_chain = create_retrieval_chain(retriever, llm_chain)

7. Define Core Functions

• Store Document: Adds parsed documents to the vector database.
• Parse PDF: Converts PDF bytes into document objects.
• Retrieve Document: Finds relevant documents for a query.
• Ask Question: Answers a query using the retrieval chain.

def store_document(documents: list[Document]) -> str:    chroma.add_documents(documents=documents)    return "document stored successfully"parser = PyPDFParser()def parse_pdf(file_content: bytes) -> list[Document]:    blob = Blob(data=file_content)    return [doc for doc in parser.lazy_parse(blob)]def retrieve_document(query: str) -> list[Document]:    return retriever.invoke(input=query)def ask_question(query: str) -> str:    response = retrieval_chain.invoke({"input": query})    return response["answer"]

Implementing the FastAPI Server (api.py)

FastAPI provides a modern, fast web framework for building APIs.

1. Import Libraries and Core Functions

from fastapi import FastAPI, UploadFilefrom chatbot import retrieve_document, store_document, parse_pdf, ask_questionfrom pydantic import BaseModelfrom typing import Listimport logging

2. Set Up Logging

logging.basicConfig(level=logging.INFO)logger = logging.getLogger(__name__)

3. Create FastAPI Instance

app = FastAPI(    title="Chatbot RAG",    description="A simple chatbot using OpenAI. Enables asking questions and getting answers based on uploaded documents.",    version="0.1",)

4. Define Pydantic Models

Pydantic models ensure that API requests and responses have the correct structure and types.

class DocumentResponse(BaseModel):    documents: List    total: int    query: str    error: str = Noneclass DocumentUploadResponse(BaseModel):    documents: List    total: int    status: str    error: str = Noneclass AskResponse(BaseModel):    query: str    answer: str    error: str = None

5. Implement API Endpoints

• Root Endpoint: Health check for the API.
• Search Documents: Retrieve relevant documents for a query.
• Upload Documents: Upload and store PDF files.
• Ask Question: Get an answer to a question based on uploaded documents.

@app.get("/")def read_root():    return {        "service": "RAG Chatbot using OPENAI",        "description": "Welcome to Chatbot RAG API",        "status": "running",    }@app.get("/documents/{query}")def search_documents(query: str) -> DocumentResponse:    try:        documents = retrieve_document(query)        return {"documents": documents, "total": len(documents), "query": query}    except Exception as e:        logger.error(f"Error searching documents: {e}", exc_info=True)        return {"error": str(e), "documents": [], "total": 0, "query": query}@app.post("/documents")async def upload_documents(files: List[UploadFile]) -> DocumentUploadResponse:    try:        documents = []        for file in files:            if file.content_type != "application/pdf":                logger.error(f"Unsupported file type: {file.content_type}")                raise ValueError("Only PDF files are supported")            content = await file.read()            parsed_docs = parse_pdf(content)            documents.extend(parsed_docs)        status = store_document(documents)        return {"documents": documents, "total": len(documents), "status": status}    except Exception as e:        logger.error(f"Error uploading documents: {e}", exc_info=True)        return {"error": str(e), "status": "failed", "documents": [], "total": 0}@app.get("/ask")def ask(query: str) -> AskResponse:    try:        answer = ask_question(query)        return {"query": query, "answer": answer}    except Exception as e:        logger.error(f"Error asking question: {e}", exc_info=True)        return {"error": str(e), "query": query, "answer": ""}

Creating the Streamlit Application (app.py)

Streamlit provides a simple way to build interactive web apps for your Python projects.

1. Import Libraries

import streamlit as stimport requests

2. Define Helper Function

This function sends a question to theFastAPI backend and returns the answer.

def ask(query: str) -> str:    with st.spinner("Asking the chatbot..."):        response = requests.get(f"{API_URL}/ask?query={query}")    if response.status_code == 200:        data = response.json()        return data["answer"]    else:        return "I couldn't find an answer to your question."

3. Set Up the Streamlit Page

API_URL = "http://localhost:8000"  # Change if deploying elsewherest.set_page_config(page_title="Chatbot", page_icon="🤖")st.title("Chatbot RAG")

4. File Upload and Document Storage

Allow users to upload multiple PDF files, which are sent to the backend for parsing and storage.

uploaded_files = st.file_uploader(    "Upload your PDF documents", type="pdf", accept_multiple_files=True)if uploaded_files:    files = [        ("files", (file.name, file.getvalue(), "application/pdf"))        for file in uploaded_files    ]    try:        with st.spinner("Uploading files..."):            response = requests.post(f"{API_URL}/documents/", files=files)        if response.status_code == 200:            st.success("Files uploaded successfully")            uploaded_files = None        else:            st.error("Failed to upload files")    except Exception as e:        st.error(f"Error uploading files: {e}")

5. Chat Interface

Provide a chat-like interface for users to interact with the chatbot.

with st.chat_message(name="ai", avatar="ai"):    st.write("Hello! I'm the Chatbot RAG. How can I help you today?")query = st.chat_input(placeholder="Type your question here...")if query:    with st.chat_message("user"):        st.write(query)    answer = ask(query)    with st.chat_message("ai"):        st.write(answer)

Running the Application

1. Start the FastAPI Server

fastapi dev api.py #if you want to run for prodcution run fastapi run api.py

2. Run the Streamlit Application

streamlit run app.py

• The API will be available athttp://127.0.0.1:8000
• The Streamlit app will run athttp://localhost:8501 by default

Demo: Using the Chatbot RAG (with Example Images)

To help you understand how to use the RAG chatbot, this section provides a step-by-step walkthrough with example screenshots from thechatbot-rag/images folder.

1. Uploading PDF Documents

Start by uploading one or more PDF files that the chatbot will use to answer your questions. On theStreamlit web interface, click theUpload your PDF docs button and select your files.

Once uploaded, you should see a confirmation message indicating that your files were uploaded successfully.

2. Asking a Question

After uploading your documents, you can interact with the chatbot using the chat input at the bottom of the page. Type your question related to the content of your uploaded PDFs and press Enter.

The chatbot will process your question, retrieve relevant information from your documents, and display an answer in the chat window.

Conclusion

Congratulations! You have built a full-stack Retrieval-Augmented Generation (RAG) chatbot usingFastAPI,OpenAI, andStreamlit. You can now upload PDF documents and interact with the chatbot to get answers based on the content of those documents.

This project demonstrates how to combine modern Python tools to create a practical, educational AI application. You can extend this project by adding authentication, deploying to the cloud, or supporting more document types.

Further Reading

• FastAPI Documentation
• OpenAI API Documentation
• Streamlit Documentation
• LangChain Documentation
• Chroma Documentation

Found the article interesting? You'll love ourPython Code Generator! Give AI a chance to do the heavy lifting for you. Check it out!

Read Also

3 Best Online AI Code Generators

Explore the top 3 AI code generators—CodeConvert, ZZZ Code AI, and CodingFleet. Compare their features, pros, and cons to find the best tool for your coding needs, with CodingFleet leading for its advanced features and versatility.

How to Build a Full-Stack Web App in Python using FastAPI and React.js

Learn how to build a full-stack notes web app using FastAPI, ReactJS, SQLAlchemy and SQLite.

How to Recover Deleted Files with Python

Learn how to recover deleted files using Python by scanning raw disk data for known file signatures. This step-by-step guide walks you through creating a basic file carving tool to retrieve lost JPGs, PDFs, DOCX files, and more.

Comment panel

Join 50,000+ Python Programmers & Enthusiasts like you!

Subscribe

Tags

New Tutorials

Popular Tutorials