Artificial Intelligence (AI) · Intermediate level

Intermediate level · Module 4: Retrieval-Augmented Generation · Lesson 2 of 3

Building a RAG Pipeline

Split documents into chunks, index them and generate answers from retrieved passages with citations.

In this lesson you will

  • Chunk and index documents for retrieval
  • Generate cited answers from retrieved passages

Key concepts

chunkingvector indexcontext assemblycitations

The lesson

Retrieval-augmented generation (RAG) has two stages. Before questions arrive, split documents into chunks of a few paragraphs, embed them and store them in an index with their source and page.

When a question arrives, embed it, retrieve the top few chunks, place them in the prompt with their source labels and instruct the model to answer only from them and cite the sources.

Chunk size matters: too small and context is lost; too large and irrelevant text dilutes the prompt and raises cost.

Step by step

  1. Chunk the documents with source metadata.
  2. Embed and index the chunks.
  3. Retrieve chunks for a question and build the prompt.
  4. Generate an answer with citations.
Workflow

How a retrieval-augmented assistant answers a question

From an organisation's documents to a cited answer — and an honest 'I don't know' when the documents are silent.

  1. Prepare documents

    Split approved documents into chunks and record each chunk's source and page.

    once, in advance

  2. Embed and index

    Turn each chunk into a vector and store it in a searchable index.

    once, in advance

  3. Embed the question

    Turn the user's question into a vector with the same embedding model.

    each question

  4. Retrieve

    Find the few chunks whose vectors are most similar to the question.

    each question

  5. Generate with rules

    Give the model the chunks and instructions: answer only from them, cite sources, say 'I don't know' otherwise.

    each question

  6. Check and log

    Verify citations point to retrieved chunks, then log the question, sources and answer.

    each question

When documents change, re-index them; when evaluation finds failures, improve chunking, retrieval or instructions.

Why it works

Retrieval keeps answers current and citable, but a wrong passage retrieved means a confident wrong answer — evaluate retrieval separately.

Practice activity

Implement the FAQ assistant's RAG pipeline.

Expected result: An assistant that answers with citations to the source documents.

Check your understanding

  1. What are the two stages of RAG?

    Show answer

    Indexing documents in advance, then retrieving and generating at question time.

  2. Why keep source metadata with each chunk?

    Show answer

    So answers can cite where information came from.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.