Introduction

Learn how Retrieval-Augmented Generation (RAG) works by building an end-to-end RAG pipeline with Llama 2, LangChain, Hugging Face embeddings, and ChromaDB — and understand how this early architecture compares with modern production RAG systems.

Originally published: August 2, 2024
Updated: September 2026

Introduction

Large Language Models (LLMs) can answer questions, summarize text, generate code, and perform many other language-related tasks. However, an LLM has an important limitation:

It does not automatically know information that was not available during its training.

For example, suppose you have an internal company document, a recently published report, or a private knowledge base. Simply asking an LLM about that information may result in an incorrect or hallucinated answer.

One approach is to fine-tune the model on your data.

But fine-tuning is not always necessary.

Retrieval-Augmented Generation (RAG) provides another approach: instead of modifying the model’s parameters, we retrieve relevant information from an external knowledge source and provide it to the LLM as context.

In this tutorial, we will build a simple RAG application using:

Llama 2 as the language model
LangChain for orchestration
Hugging Face Sentence Transformers for embeddings
ChromaDB as the vector database
Python for implementation

The original implementation uses versions of these libraries from the 2024 RAG ecosystem. The code is intentionally preserved because it provides a useful way to understand the fundamental mechanics of vector-based RAG.

Important: This Is an Earlier RAG Implementation

This tutorial was originally published in 2024 and uses Llama 2, an older LangChain API, and ChromaDB.

The RAG ecosystem has evolved considerably since then.

The implementation in this article represents the fundamental architecture:

			
Documents
    ↓
Text Chunking
    ↓
Embeddings
    ↓
Vector Database
    ↓
Similarity Search
    ↓
Retrieved Context
    ↓
Llama 2
    ↓
Generated Answer

		

Modern production RAG systems typically add several additional layers:

			
User Query
    ↓
Query Understanding / Rewriting
    ↓
┌──────────────────────────────┐
│ Dense Retrieval + BM25       │
│       Hybrid Search          │
└──────────────┬───────────────┘
               ↓
          Reranking
               ↓
      Context Selection
               ↓
      Prompt Construction
               ↓
             LLM
               ↓
       Grounded Response
               ↓
 Evaluation + Guardrails +
 Observability + Monitoring

		

The core RAG idea has not changed:

Retrieve relevant external knowledge and give that knowledge to the LLM as context before generating the answer.

What has changed is how sophisticated production systems have become around that core idea.

If you are new to RAG, I recommend following the RAG Fundamentals learning path alongside this tutorial.

What Is Retrieval-Augmented Generation?

Retrieval-Augmented Generation (RAG) is an architecture that combines information retrieval with text generation.

Instead of relying entirely on the knowledge encoded inside an LLM, a RAG system retrieves relevant information from an external knowledge source and supplies it to the model.

At a high level:

			
User Question
      ↓
Retrieve Relevant Documents
      ↓
Add Documents to Prompt
      ↓
LLM Generates Answer

		

For example, suppose we have a document containing the 2023 State of the Union address.

A user asks:

What was the unemployment rate mentioned in the address?

The LLM itself may or may not know the answer.

A RAG system instead performs:

			
Question
   ↓
Embedding
   ↓
Vector Search
   ↓
Relevant Document Chunks
   ↓
LLM + Retrieved Context
   ↓
Answer

		

The retrieved document contains the relevant information, allowing the model to generate an answer grounded in the source.

Why Use RAG Instead of Fine-Tuning?

A common misconception is that you need to fine-tune an LLM whenever you want it to work with new information.

That is not necessarily true.

Fine-tuning changes the model’s parameters. RAG leaves the model unchanged and supplies additional information at inference time.

Consider a company with thousands of internal documents.

When a policy changes, updating a RAG knowledge base can be significantly more practical than retraining the model.

A simplified comparison is:

Approach	How knowledge is added
Prompting	Knowledge is manually placed in the prompt
Fine-tuning	Knowledge influences model parameters
RAG	Relevant knowledge is retrieved at inference time

RAG is particularly useful when the underlying information:

changes frequently
is private
is too large to put into every prompt
needs to be traceable to source documents
comes from external or organizational knowledge bases

RAG Architecture

A basic RAG system has two major components:

1. Retriever

The retriever finds relevant information from an external knowledge source.

2. Generator

The generator uses the retrieved information to produce the final answer.

In our implementation:

                ┌───────────────┐
                │   Documents   │
                └───────┬───────┘
                        ↓
                Text Chunking
                        ↓
                   Embeddings
                        ↓
                  ChromaDB
                        ↑
                        │
User Query → Retriever ─┘
                 ↓
          Relevant Chunks
                 ↓
              Llama 2
                 ↓
              Answer

Key Technologies Used

Llama 2

Llama 2 is a family of large language models released by Meta.

The original implementation uses the:

Llama 2 7B Chat

model.

For this tutorial, Llama 2 acts as the generation component of our RAG pipeline.

2026 note: Llama 2 is now an older-generation model. New RAG applications would normally consider newer models depending on their requirements, cost, latency, deployment environment, and licensing constraints.

LangChain

LangChain is an orchestration framework for building applications around LLMs.

In this tutorial, it connects:

			
LLM
 ↓
Retriever
 ↓
Documents
 ↓
RAG Chain

		

The original tutorial uses an older LangChain API.

2026 note: LangChain’s APIs have changed significantly since the 0.0.x releases used here. The classes and imports below are intentionally preserved so that the original implementation remains reproducible in its historical context.

Embeddings

An embedding model converts text into a numerical vector.

For example:

"America's unemployment rate was 3.4%"

can be represented as a high-dimensional vector.

Semantically similar pieces of text tend to have similar vector representations.

This allows us to perform semantic similarity search.

In this tutorial we use:

sentence-transformers/all-mpnet-base-v2

ChromaDB

ChromaDB is a vector database that can store embeddings and retrieve documents based on similarity.

In our application, ChromaDB stores the embeddings generated from document chunks.

The simplified process is:

			
Document
   ↓
Chunk
   ↓
Embedding
   ↓
ChromaDB

		

When a user asks a question:

			
Question
   ↓
Question Embedding
   ↓
Similarity Search
   ↓
Relevant Chunks

		

Project Setup

The original implementation was developed in a Kaggle environment.

Install the required versions:

			
!pip install transformers==4.33.0 accelerate==0.22.0 einops==0.6.1 langchain==0.0.300 xformers==0.0.21 \
bitsandbytes==0.41.1 sentence_transformers==2.2.2 chromadb==0.4.12

Compatibility note: These package versions are intentionally retained from the original tutorial. Modern versions of these libraries may require substantially different APIs and dependency combinations.

Import Required Libraries

			
from torch import cuda, bfloat16
import torch
import transformers
from transformers import AutoTokenizer
from time import time
from langchain.llms import HuggingFacePipeline
from langchain.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.chains import RetrievalQA
from langchain.vectorstores import Chroma

		

These libraries provide the main building blocks for our implementation.

Step 1: Initialize Llama 2

First, we define the model location, device, and quantization configuration.

			
model_id = '/kaggle/input/llama-2/pytorch/7b-chat-hf/1'
device = f'cuda:{cuda.current_device()}' if cuda.is_available() else 'cpu'
# Set quantization configuration to load the large model
# using less GPU memory.
bnb_config = transformers.BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type='nf4',
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=bfloat16
)

		

Why 4-bit quantization?

Llama 2 7B contains billions of parameters.

Loading a model in full precision can require significant GPU memory.

Quantization reduces the memory required to store the model.

Here we use:

			
4-bit quantization
+
NF4
+
Double Quantization

		

This makes the model more practical to run in a constrained GPU environment.

Step 2: Load the Model and Tokenizer

Now we load the model.

			
time_1 = time()
model_config = transformers.AutoConfig.from_pretrained(
    model_id,
)
model = transformers.AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    config=model_config,
    quantization_config=bnb_config,
    device_map='auto',
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
time_2 = time()
print(f"Prepare model, tokenizer: {round(time_2-time_1, 3)} sec.")

		

The tokenizer converts text into tokens that can be processed by the language model.

The model then generates text based on those tokens.

Step 3: Create the Text Generation Pipeline

We now create a Hugging Face text-generation pipeline.

			
time_1 = time()
query_pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.float16,
    device_map="auto",
)
time_2 = time()
print(f"Prepare pipeline: {round(time_2-time_1, 3)} sec.")

		

The pipeline provides a convenient interface for generating text.

The original implementation produced approximately:

Prepare pipeline: 1.77 sec.

The exact timing will depend on the hardware and runtime environment.

Step 4: Test the LLM

Before introducing RAG, it is useful to verify that the underlying LLM works correctly.

We can create a helper function:

			
def test_model(tokenizer, pipeline, prompt_to_test):
    """
    Perform a query
    print the result
    Args:
        tokenizer: the tokenizer
        pipeline: the pipeline
        prompt_to_test: the prompt
    Returns
        None
    """
    # adapted from Hugging Face Llama 2 documentation
    time_1 = time()
    sequences = pipeline(
        prompt_to_test,
        do_sample=True,
        top_k=10,
        num_return_sequences=1,
        eos_token_id=tokenizer.eos_token_id,
        max_length=200,
    )
    time_2 = time()
    print(f"Test inference: {round(time_2-time_1, 3)} sec.")
    for seq in sequences:
        print(f"Result: {seq['generated_text']}")

		

Now test the model:

			
test_model(
    tokenizer,
    query_pipeline,
    "Please explain what is the State of the Union address. "
    "Give just a definition. Keep it in 100 words."
)

		

At this stage, we are testing the LLM alone.

No external documents have been provided.

Step 5: Wrap the Pipeline as an LLM

LangChain can use our Hugging Face pipeline through HuggingFacePipeline.

			
llm = HuggingFacePipeline(
    pipeline=query_pipeline
)

We can test it again:

			
llm(
    prompt="Please explain what is the State of the Union address. "
           "Give just a definition. Keep it in 100 words."
)

If this works, our generation component is ready.

Step 6: Load the Knowledge Base

Now we move to the important part of RAG:

providing the LLM with external knowledge.

For this example, we use President Biden’s 2023 State of the Union address.

The document is loaded using LangChain’s TextLoader.

			
loader = TextLoader(
    "/kaggle/input/president-bidens-state-of-the-union-2023/"
    "biden-sotu-2023-planned-official.txt",
    encoding="utf8"
)
documents = loader.load()

		

The loaded document becomes the source of knowledge for our RAG system.

Step 7: Split the Document into Chunks

Large documents should not normally be inserted into a vector database as one enormous block.

Instead, we divide them into smaller pieces called chunks.

			
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=20
)
all_splits = text_splitter.split_documents(documents)

		

Here:

			
chunk_size = 1000
chunk_overlap = 20

The overlap helps preserve some context between neighboring chunks.

For example:

			
Chunk 1
-------------------------
A B C D E F G H
              ↓
          overlap
                ↓
Chunk 2
              G H I J K L

		

Why Is Chunking Important in RAG?

Chunking has a major impact on retrieval quality.

If chunks are too large:

irrelevant information may be retrieved
prompts become unnecessarily large
retrieval becomes less precise

If chunks are too small:

important context may be separated
individual chunks may lack sufficient meaning

The optimal chunking strategy depends on the document type.

Modern RAG systems often use more sophisticated strategies such as:

recursive chunking
semantic chunking
section-aware chunking
sentence-aware chunking
parent-child retrieval
document structure-aware chunking

The RecursiveCharacterTextSplitter used here is a simple and useful starting point.

Step 8: Generate Embeddings

Now we convert our document chunks into vectors.

We use:

sentence-transformers/all-mpnet-base-v2

			
model_name = "sentence-transformers/all-mpnet-base-v2"
model_kwargs = {
    "device": "cuda"
}
embeddings = HuggingFaceEmbeddings(
    model_name=model_name,
    model_kwargs=model_kwargs
)

		

Conceptually:

			
Text
 ↓
Embedding Model
 ↓
Vector

		

For example:

			
"Inflation is coming down"
            ↓
     [0.021, -0.173, ...]

The actual embedding contains many dimensions.

Step 9: Store Embeddings in ChromaDB

Now we create our vector database.

			
vectordb = Chroma.from_documents(
    documents=all_splits,
    embedding=embeddings,
    persist_directory="chroma_db"
)

		

The persist_directory option allows ChromaDB to persist its data locally.

Our architecture is now:

			
State of the Union
       ↓
    Chunks
       ↓
   Embeddings
       ↓
    ChromaDB

		

Step 10: Create the Retriever

The vector database can now act as our retriever.

retriever = vectordb.as_retriever()

When a question is submitted, the retriever searches the vector database for relevant chunks.

For example:

"What was the nation's economic status?"

is converted into an embedding.

The retriever then searches for chunks with similar semantic meaning.

Step 11: Create the RAG Chain

Now we combine:

Llama 2
Retriever
LangChain

using RetrievalQA.

			
qa = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=retriever,
    verbose=True
)

		

The basic workflow becomes:

			
Question
   ↓
Retriever
   ↓
Relevant Documents
   ↓
Stuff Documents into Prompt
   ↓
Llama 2
   ↓
Answer

		

The "stuff" chain type means the retrieved documents are placed into the prompt and passed to the LLM together.

Step 12: Query the RAG System

Let’s create a helper function to test our RAG pipeline.

			
def test_rag(qa, query):
    print(f"Query: {query}\n")
    time_1 = time()
    result = qa.run(query)
    time_2 = time()
    print(
        f"Inference time: {round(time_2-time_1, 3)} sec."
    )
    print("\nResult: ", result)

		

Now let’s ask our first question.

			
query = (
    "What were the main topics in the State of the Union "
    "in 2023? Summarize. Keep it under 200 words."
)
test_rag(qa, query)

		

The RAG chain performs approximately:

			
Question
   ↓
Generate query embedding
   ↓
Search ChromaDB
   ↓
Retrieve relevant chunks
   ↓
Add chunks to prompt
   ↓
Llama 2
   ↓
Generate answer

		

An example result from the original implementation was:

			
The State of the Union in 2023 focused on several key topics,
including the nation's economic strength, the competition with
China, and the need to come together as a nation to face the
challenges ahead.

The exact response can vary because Llama 2 is being used with sampling enabled.

Another RAG Query

Let’s ask about the economic situation.

			
query = (
    "What is the nation economic status? "
    "Summarize. Keep it under 200 words."
)
test_rag(qa, query)

		

The original implementation returned an answer containing information such as:

			
The nation's economic status is strong, with a low unemployment
rate of 3.4%...

Notice something important.

The answer is based on information contained in the supplied State of the Union document.

This is the fundamental value of RAG.

Step 13: Inspect the Retrieved Documents

One of the most important debugging techniques in RAG is checking what the retriever actually retrieved.

A generated answer may look reasonable, but that does not necessarily mean the retrieval step was correct.

We can inspect the retrieved documents directly:

			
docs = vectordb.similarity_search(query)
print(f"Query: {query}")
print(f"Retrieved documents: {len(docs)}")
for doc in docs:
    doc_details = doc.to_json()['kwargs']
    print(
        "Source: ",
        doc_details['metadata']['source']
    )
    print(
        "Text: ",
        doc_details['page_content'],
        "\n"
    )

		

The original implementation retrieved four documents.

The returned chunks contained relevant passages discussing:

unemployment
manufacturing jobs
inflation
economic recovery
job creation

This is an important RAG debugging principle:

Always inspect retrieval quality independently from generation quality.

If the retrieved context is irrelevant, changing the LLM prompt may not solve the problem.

Understanding the Complete Pipeline

At this point, we have implemented a complete basic RAG system.

The entire workflow can be summarized as:

                  OFFLINE / INDEXING
                  ------------------

              Source Document
                     ↓
                Text Loader
                     ↓
                 Chunking
                     ↓
               Embedding Model
                     ↓
                 ChromaDB


                  ONLINE / QUERY
                  --------------

                User Question
                     ↓
               Query Embedding
                     ↓
                ChromaDB Search
                     ↓
             Relevant Documents
                     ↓
              LangChain RAG
                     ↓
                  Llama 2
                     ↓
                 Answer

This distinction between indexing time and query time is fundamental to understanding RAG systems.

Indexing vs. Query Time

Indexing Time

Documents are processed before users ask questions.

			
Documents
   ↓
Chunk
   ↓
Embedding
   ↓
Vector Database

		

This can be performed periodically whenever documents are added or updated.

Query Time

When a user asks a question:

			
Question
   ↓
Embedding
   ↓
Vector Search
   ↓
Retrieved Context
   ↓
LLM
   ↓
Answer

		

Understanding this separation becomes particularly important when designing production RAG pipelines.

Why Does RAG Work?

LLMs are powerful generators, but they are not databases.

A model may have learned information during training, but that does not mean it can reliably answer questions about:

private documents
newly created information
rapidly changing information
proprietary company knowledge
large document collections

RAG provides an external knowledge layer.

Conceptually:

			
LLM Knowledge
      +
External Knowledge
      ↓
Grounded Generation

		

This is why RAG is so widely used for enterprise knowledge applications.

Limitations of This Implementation

The implementation above is excellent for understanding the fundamentals, but it has several limitations.

1. Vector Search Only

We use semantic similarity search.

Modern systems frequently combine:

			
Dense Retrieval
       +
Sparse Retrieval / BM25
       ↓
Hybrid Search

		

This helps when exact keywords, names, identifiers, or terminology matter.

2. No Reranking

The retrieved documents are passed directly to the LLM.

Production RAG systems often use a reranker:

			
Initial Retrieval
       ↓
Top N Documents
       ↓
Reranker
       ↓
Top K Documents
       ↓
LLM

		

This can improve the relevance of the final context.

3. Basic Chunking

We use:

			
RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=20
)

This works for a demonstration, but document structure can be important.

For example, a technical document may contain:

			
Chapter
 ├── Section
 │    ├── Subsection
 │    └── Subsection
 └── Section

		

A production system may preserve this hierarchy in metadata.

4. No Metadata Filtering

Our example does not use metadata filters.

Production systems may need queries such as:

			
department = "finance"
document_type = "policy"
year >= 2025
access_level = "employee"

This can reduce irrelevant retrieval.

5. No Evaluation Pipeline

We manually inspect the results.

Production systems should measure retrieval and generation quality using appropriate evaluation datasets and metrics.

Useful dimensions include:

retrieval relevance
context precision
context recall
answer correctness
faithfulness
latency
cost

6. No Production Observability

The original notebook measures inference time, but production systems generally need much more:

			
Request
 ↓
Retrieval latency
 ↓
Reranking latency
 ↓
LLM latency
 ↓
Token usage
 ↓
Cost
 ↓
Answer quality

		

Tracing and observability become especially important when RAG systems serve real users.

How Modern RAG Differs

The original architecture is:

			
Query
 ↓
Vector Search
 ↓
LLM

		

A more sophisticated architecture might look like:

                     User Query
                          ↓
                  Query Rewriting
                          ↓
              ┌───────────┴───────────┐
              ↓                       ↓
        Dense Retrieval          BM25 Search
              ↓                       ↓
              └───────────┬───────────┘
                          ↓
                    Hybrid Results
                          ↓
                       Reranker
                          ↓
                  Context Selection
                          ↓
                   Prompt Builder
                          ↓
                         LLM
                          ↓
                 Grounded Response
                          ↓
             Evaluation / Guardrails

This does not mean that every RAG application needs every component.

The architecture should match the problem.

A simple internal prototype may only need:

Embedding → Vector DB → LLM

A large enterprise application may need:

			
Hybrid Search
+ Reranking
+ Metadata Filtering
+ Query Rewriting
+ Access Control
+ Evaluation
+ Observability
+ Guardrails

		

What Happened to Llama 2 and ChromaDB?

It is important to distinguish concepts from specific tools.

The RAG concept remains highly relevant.

The exact implementation has changed.

Llama 2

Llama 2 was an important open-weight model generation and remains useful for understanding the history of open LLM applications.

However, newer models generally provide better capabilities for many current workloads.

LangChain

The LangChain ecosystem has evolved significantly.

The APIs used in this article:

from langchain.chains import RetrievalQA

and:

qa.run(query)

come from an older version of LangChain.

Modern applications should consult the current LangChain documentation rather than blindly copying these imports into a new project.

ChromaDB

ChromaDB remains useful for local development and experimentation.

However, production vector search architecture depends on factors such as:

scale
latency
filtering requirements
availability
infrastructure
operational requirements
cost

The important architectural concept is therefore not:

“Use ChromaDB.”

It is:

“Use an appropriate retrieval/indexing system for your application’s requirements.”

When Should You Use RAG?

RAG is particularly useful when your application needs access to external knowledge.

Common examples include:

Enterprise Knowledge Assistants

			
Company Documents
      ↓
RAG
      ↓
Employee Assistant

		

Customer Support

			
Product Documentation
       ↓
      RAG
       ↓
Customer Question
       ↓
Support Answer

		

Legal Document Search

			
Contracts
   ↓
RAG
   ↓
Question
   ↓
Relevant Clauses

		

Financial Research

			
Reports
Filings
Research
   ↓
RAG
   ↓
Analyst Question

		

Technical Documentation

			
API Documentation
Architecture Docs
Runbooks
   ↓
RAG
   ↓
Developer Assistant

		

RAG vs. Fine-Tuning

RAG and fine-tuning solve different problems.

Requirement	RAG	Fine-Tuning
Add frequently changing knowledge	✅	❌
Private document Q&A	✅	Sometimes
Ground answers in documents	✅	❌
Change model behavior/style	Limited	✅
Teach a specific output format	Sometimes	✅
Update knowledge without retraining	✅	❌
Need source attribution	✅	Limited

They can also be combined.

For example:

			
Fine-Tuned Model
       +
RAG Knowledge Base
       ↓
Specialized RAG Application

		

Key Takeaways

We built an end-to-end Retrieval-Augmented Generation system using:

			
Llama 2
   +
LangChain
   +
Hugging Face Embeddings
   +
ChromaDB

		

The pipeline consists of:

Loading documents
Splitting documents into chunks
Generating embeddings
Storing embeddings in ChromaDB
Retrieving relevant chunks
Passing those chunks to Llama 2
Generating an answer

The most important concept is the separation between:

			
Retrieval
    +
Generation

The retriever provides relevant external knowledge, while the LLM generates the response.

Conclusion

This tutorial demonstrated a complete RAG pipeline using Llama 2, LangChain, Hugging Face embeddings, and ChromaDB.

Although these specific libraries and APIs represent an earlier generation of the RAG ecosystem, the underlying architecture remains fundamental:

			
Documents
    ↓
Chunking
    ↓
Embeddings
    ↓
Retrieval
    ↓
Context
    ↓
LLM
    ↓
Answer

		

Modern production RAG systems build on this foundation with techniques such as hybrid search, BM25, reranking, metadata filtering, query rewriting, evaluation, observability, access control, and guardrails.

Therefore, this tutorial should be viewed as a practical implementation of the foundational vector-search RAG architecture, rather than a recommendation to build every modern production system exactly this way.

The important lesson is not Llama 2 or ChromaDB.

The important lesson is:

RAG allows an LLM to use external knowledge at inference time without requiring that knowledge to be encoded into the model’s parameters.

Continue Learning: RAG Fundamentals

If you want to move from this basic implementation toward modern RAG architectures, continue with the GeekyCodes RAG Fundamentals series.

A recommended learning path is:

			
What Is RAG?
      ↓
Why LLMs Hallucinate
      ↓
How RAG Works
      ↓
Dense vs. Sparse Retrieval
      ↓
BM25
      ↓
Hybrid Search
      ↓
Reranking
      ↓
Advanced RAG
      ↓
RAG Evaluation
      ↓
Production RAG

		

This article fits into the series as the hands-on implementation layer:

			
RAG Fundamentals
       ↓
Understand the concepts
       ↓
Understand retrieval
       ↓
Understand hybrid search
       ↓
Understand reranking
       ↓
      THIS
       ↓
Build a basic RAG system
       ↓
Production RAG

		

Explore the complete RAG Fundamentals learning path →

References

Murtuza Kazmi — Using LLaMA 2.0, FAISS and LangChain for Question-Answering on Your Own Data
Patrick Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Minhajul Hoque — Retrieval Augmented Generation: Grounding AI Responses in Factual Data
Fangrui Liu — Discover the Performance Gain with Retrieval Augmented Generation
Andrew — How to use Retrieval-Augmented Generation (RAG) with Llama 2
Yogendra Sisodia — Retrieval Augmented Generation Using Llama2 And Falcon

References

[1] Murtuza Kazmi, Using LLaMA 2.0, FAISS and LangChain for Question-Answering on Your Own Data, https://medium.com/@murtuza753/using-llama-2-0-faiss-and-langchain-for-question-answering-on-your-own-data-682241488476

[2] Patrick Lewis, Ethan Perez, et. al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, https://browse.arxiv.org/pdf/2005.11401.pdf

[3] Minhajul Hoque, Retrieval Augmented Generation: Grounding AI Responses in Factual Data, https://medium.com/@minh.hoque/retrieval-augmented-generation-grounding-ai-responses-in-factual-data-b7855c059322

[4] Fangrui Liu , Discover the Performance Gain with Retrieval Augmented Generation, https://thenewstack.io/discover-the-performance-gain-with-retrieval-augmented-generation/

[5] Andrew, How to use Retrieval-Augmented Generation (RAG) with Llama 2, https://agi-sphere.com/retrieval-augmented-generation-llama2/

[6] Yogendra Sisodia, Retrieval Augmented Generation Using Llama2 And Falcon, https://medium.com/@scholarly360/retrieval-augmented-generation-using-llama2-and-falcon-ed26c7b14670

Introduction

Introduction

Important: This Is an Earlier RAG Implementation

What Is Retrieval-Augmented Generation?

Why Use RAG Instead of Fine-Tuning?

RAG Architecture

1. Retriever

2. Generator

Key Technologies Used

Llama 2

LangChain

Embeddings

ChromaDB

Project Setup

Import Required Libraries

Step 1: Initialize Llama 2

Why 4-bit quantization?

Step 2: Load the Model and Tokenizer

Step 3: Create the Text Generation Pipeline

Step 4: Test the LLM

Step 5: Wrap the Pipeline as an LLM

Step 6: Load the Knowledge Base

Step 7: Split the Document into Chunks

Why Is Chunking Important in RAG?

Step 8: Generate Embeddings

Step 9: Store Embeddings in ChromaDB

Step 10: Create the Retriever

Step 11: Create the RAG Chain

Step 12: Query the RAG System

Another RAG Query

Step 13: Inspect the Retrieved Documents

Understanding the Complete Pipeline

Indexing vs. Query Time

Indexing Time

Query Time

Why Does RAG Work?

Limitations of This Implementation

1. Vector Search Only

2. No Reranking

3. Basic Chunking

4. No Metadata Filtering

5. No Evaluation Pipeline

6. No Production Observability

How Modern RAG Differs

What Happened to Llama 2 and ChromaDB?

Llama 2

LangChain

ChromaDB

When Should You Use RAG?

Enterprise Knowledge Assistants

Customer Support

Legal Document Search

Financial Research

Technical Documentation

RAG vs. Fine-Tuning

Key Takeaways

Conclusion

Continue Learning: RAG Fundamentals

References

Related Tutorials

More work on the same topic

References

Share this:

Like this:

Related

Leave a ReplyCancel reply

Discover more from Geeky Codes