Claremont AI Learning Guide
A practical guide to connecting language models with relevant, current, and permission-aware information—from the first document ingested to a reliable production system.
01 · The foundation
Retrieval-Augmented Generation is a system design pattern that lets a large language model (LLM) consult an external knowledge source before it answers. The system retrieves relevant material for a user's query and supplies that material to the model as context for generating a response.
Unlike relying only on knowledge encoded in model parameters, RAG can use an organization's own documents and can reflect changes to those documents as its index is updated. The model still writes the response; retrieval supplies evidence that may help it answer more accurately and with better grounding.
A useful mental model: RAG is a data-and-application pipeline around an LLM, not just a vector database. Answer quality depends on the full chain: source content, parsing, chunking, retrieval, ranking, prompt construction, generation, and evaluation.
02 · The end-to-end flow
A RAG application has two connected workflows. Indexing prepares knowledge ahead of time; query-time processing retrieves evidence and generates an answer when someone asks a question.
Index-time vs. query-time: parsing, chunking, and document embeddings are usually computed when data is ingested. A user query is embedded and searched when asked. Keeping these paths distinct makes it easier to reason about data freshness, query latency, and failures.
03 · The components
Each component addresses a different part of the problem. Improve the stage that is limiting quality instead of adding complexity without evidence.
RAG is only as useful as its source material. Parsing should preserve headings, sections, dates, tables, and document identifiers where those details matter. Normalize duplicate or malformed content and retain a trace back to the original source.
Chunking determines what can be retrieved as a unit. Fixed-size chunks are straightforward; sentence- or structure-aware chunks preserve natural boundaries; hierarchical approaches can retrieve a focused passage alongside its parent context. Overlap can reduce boundary loss but increases index size and repeated text.
An embedding model maps text into a numeric vector space where semantic similarity can be compared. Document and query embeddings must be compatible. Evaluate models on representative domain queries rather than assuming a general-purpose model is best.
A vector index supports approximate nearest-neighbor (ANN) search over dense embeddings. Store identifiers and metadata with each passage so results can be filtered, traced, updated, and linked back to their source.
Dense retrieval finds semantic matches. Sparse retrieval, such as BM25, is effective for exact words, names, and codes. Hybrid retrieval combines both signals; metadata filters narrow the search to a user, source, date, or other permitted scope.
A first-stage retriever can return a broad candidate set; a reranker then scores query-passage pairs to select a smaller, more relevant set. Deduplication, compression, and ordering help keep useful context within the model's token budget.
The prompt should distinguish instructions from retrieved, untrusted content; state that answers should be grounded in evidence; and define how to handle missing evidence. The generator synthesizes, summarizes, or formats the final response.
The application manages identity, conversation state, source links, feedback, and user experience. It should provide provenance that lets users inspect supporting passages where the task requires it.
Chunk size is not a universal constant. Smaller chunks can improve precision but separate facts that belong together; larger chunks preserve context but can add irrelevant text. Compare alternatives on a labeled query set and measure both retrieval quality and answer quality.
04 · A maturity path
RAG designs are best understood as an evolution of retrieval and control. Start with a simple baseline, measure its limitations, and add capabilities only where they improve the result.
Pattern: chunk documents, embed them, retrieve the top-k nearest passages, and pass them to an LLM. This fixed pipeline is a useful baseline: it is easy to understand, test, and operate.
Typical limitations: a query may not use the same vocabulary as the source; fixed chunks can split meaning; raw top-k results may include noise; and there is no built-in loop to detect poor retrieval.
Enhance one or more stages: rewrite or expand the query before search; combine BM25 keyword search with dense vector search; fuse result rankings; rerank candidates; compress or reorder context before generation.
Examples: HyDE embeds a hypothetical answer to bridge the query-to-document vocabulary gap; RAG-Fusion searches multiple reformulations and combines results; a cross-encoder reranker re-scores the most promising candidates.
A modular system treats retrieval, ranking, filtering, and generation as replaceable stages. Adaptive RAG adds routing so different question types can follow different paths—for example, a direct lookup can use fast retrieval while a complex question gets extra steps.
Corrective RAG evaluates whether retrieved context is useful and can try another retrieval path when evidence is weak. Self-RAG and related designs add explicit reflection or critique signals. Validation can improve resilience but adds cost and latency, and the critique itself must be evaluated.
An agent can decide whether to retrieve, which source or tool to use, and whether another retrieval step is needed. Iterative and multi-hop RAG decomposes complex questions and gathers evidence across steps. This can address harder queries but is typically less predictable, more expensive, and slower than a fixed pipeline.
Pipeline or agent? A fixed pipeline is generally easier to make fast, deterministic, and testable. Agent-style orchestration is useful when a query needs flexible planning, multiple tools, or evidence gathered in stages. A practical path is incremental: validate a pipeline first, then add a targeted feedback or routing step if measurements justify it.
05 · The architecture landscape
The source repository catalogs 52 RAG architectures. Rather than treating every name as a separate foundation, classify designs by their retrieval control, data modality, feedback, and retrieval scope. Architectures often combine multiple ideas.
| Family | What changes | Examples and fit |
|---|---|---|
| Foundational pipelines | Fixed, mostly single-pass retrieval and generation. | Naive, Advanced, Modular RAG; DPR and ColBERT-style dense retrieval techniques. |
| Adaptive and corrective | Route queries or assess retrieval quality; retry or select another path when appropriate. | Adaptive RAG, Corrective RAG (CRAG), Self-RAG, Speculative RAG, Astute RAG. |
| Agentic and reasoning | Plan retrieval and reasoning across multiple steps or tools. | Agentic RAG, FLARE, Iterative / Multi-hop RAG, Search-R1, Deep Research, CoRAG, RQ-RAG. |
| Graph-based | Use entities and relationships to retrieve connected information and support multi-hop queries. | Graph RAG, LightRAG, HippoRAG, KAG, GraphReader, GNN-RAG, LazyGraphRAG. |
| Long-context and memory | Change the retrieval unit, build summaries at multiple levels, or maintain information across dialogue. | Long-Context RAG, RAPTOR, LongRAG, Contextual RAG, Memory / Conversational RAG, MemoRAG. |
| Structured and multimodal | Retrieve from tables, schemas, SQL sources, or non-text content such as images. | Structured / SQL RAG, Table-Aware RAG, SURGE, Multimodal RAG, VisRAG. |
| Efficiency and query transformation | Reduce repeated inference or improve recall by transforming and combining queries. | CAG, REFRAG, RAG-Fusion, HyDE, Few-Shot Example RAG. |
| Trust, privacy, and freshness | Make evidence traceable, constrain exposure, or keep content current. | Verifiable / Citation RAG, Privacy-Preserving RAG, Streaming / Real-Time RAG. |
| Training-time / parametric | Integrate retrieval or retrieved knowledge into training or model architecture. | REALM, RETRO, Atlas, Fusion-in-Decoder, and RAFT. These differ from many inference-time application pipelines. |
Static: one strategy for every request. Dynamic: route based on query characteristics. Agent-driven: an LLM chooses retrieval actions at runtime.
Text-only sources are the simplest starting point. Structured, graph, and multimodal systems retrieve through different representations and may need specialized evaluation.
A one-pass design has no feedback; a corrective design checks evidence or output; more involved approaches use user signals or learned critique to guide retrieval.
Single-hop retrieves once. Multi-hop chains evidence gathering. Iterative continues until a stopping rule or budget is reached.
Interpret architecture labels as patterns, not guarantees. A "graph" or "agentic" label does not determine quality. Corpus shape, query types, latency, operating cost, and evaluation results should determine the design.
06 · Making the choice
Begin with the job users need done and the evidence available. Use a basic retrieval baseline, examine its failure cases, and select the next capability to address the observed gap.
| Need or constraint | Consider | Trade-off to test |
|---|---|---|
| Small, clean corpus; simple factual questions | Basic RAG with sensible chunking and source links | May be insufficient when queries are ambiguous or require several facts. |
| Exact names, identifiers, and conceptual queries both matter | Hybrid sparse + dense search, followed by ranking | More components and tuning; test precision and recall together. |
| Query wording differs from the documents | Query rewriting, multi-query retrieval, or HyDE | Additional inference may increase latency and cost. |
| Question requires relationships across entities | Graph retrieval or a controlled multi-hop approach | Graph construction and maintenance add complexity. |
| Answers come from tables or relational data | Structured retrieval or schema-aware text-to-SQL | Validate generated queries, permissions, and result correctness. |
| Users ask follow-up questions across a conversation | History-aware query rewriting and explicit memory boundaries | Do not let irrelevant or private history leak into later requests. |
| Strict response-time or cost budget | Fixed pipeline, caching, bounded retrieval, model routing | Measure tail latency and quality; caching can affect freshness. |
| Evidence is frequently poor or incomplete | Improve source quality and retrieval first; then consider corrective feedback | Self-checks can be wrong and add extra model calls. |
Do not jump straight to an agent. Agents are not a default quality upgrade. A well-built, measured pipeline may be faster and easier to debug. Add adaptive or agentic control when real query cases demonstrate a need for variable strategies or multi-step evidence gathering.
07 · Production readiness
A production system must do more than return a plausible answer in a demo. It needs to remain useful under component failures, meet its service objectives, reflect data changes, enforce controls, and provide evidence about its behavior.
Plan for embedding, search, reranking, and generation failures. Use explicit timeouts, bounded retries, circuit breakers, and meaningful fallbacks. For example, a reranker timeout may allow a system to return the original retrieval ranking rather than stall indefinitely.
Measure end-to-end and per-stage P50, P95, and P99 latency. Parallel retrieval, caching, smaller candidate sets, and fewer sequential model calls can help. Agent loops and reranking can materially change tail latency.
Track document versions and update status. Use incremental indexing where appropriate, remove superseded content, and define how quickly changes should become searchable. Stale copies can compete with current material.
Operators need ways to filter or suppress sources, constrain outputs, manage namespaces, and apply access rules. These controls belong in application and retrieval design—not just in natural-language instructions.
Trace a request across query rewriting, retrieval, ranking, generation, and response. Record relevant timing, result identifiers, model versions, and errors while respecting privacy and retention requirements.
Account for embedding and indexing, vector storage, retrieval and reranking, and especially generation tokens. Batch indexing work, cache selectively, and route requests by complexity where measurement supports it.
| Observed failure | Possible cause | Response to investigate |
|---|---|---|
| Relevant material is absent from results | Parsing loss, poor chunk boundaries, embedding mismatch, stale index, or too-narrow retrieval. | Inspect indexed text and labels; test chunking, query variants, hybrid search, and re-indexing. |
| Top results contain noise | Weak ranking, ambiguous query, duplicate chunks, or missing metadata filters. | Measure precision; add filtering, deduplication, reranking, or better query interpretation. |
| Evidence is retrieved but the answer is unsupported | Prompt ambiguity, excessive context, model error, or poor citation handling. | Reduce and order context, require evidence-linked claims, test abstention, and evaluate faithfulness. |
| Answers rely on outdated information | Indexing delays, deletion/versioning errors, or old content remains highly ranked. | Monitor index freshness, filter or down-rank old versions, and test update propagation. |
| Requests time out or costs rise | Too many sequential calls, oversized contexts, expensive reranking, or retry storms. | Set per-stage budgets, inspect traces, bound retries, cache safely, and simplify low-complexity paths. |
| Different users see the wrong material | Authorization applied after retrieval or metadata filters are missing or incorrect. | Enforce permissions before results reach the model; test across roles and tenants. |
08 · Trust and governance
RAG changes which information an LLM can access, but retrieved information remains input to a generative system. Treat content, prompts, tools, and outputs as part of one security boundary.
Security principle: prompt wording is not an authorization mechanism. Enforce access in trusted application and data services, and verify that unauthorized records never enter the model's context.
09 · Measure what matters
A correct final answer can hide a weak retriever, while relevant retrieved passages can still be misused by the generator. Evaluate both planes so teams can locate the source of a regression.
Use a representative, labeled set of queries and relevant source passages to compare chunking, embeddings, filters, and retrieval strategies.
Frameworks such as RAGAS and LLM-as-judge methods can help, but should be calibrated and checked against human judgments.
Test parsing, chunk boundaries, metadata, embedding dimensions, permission filters, and source links with fast automated checks.
Run a stable collection of representative queries through retrieval and generation when changing prompts, models, or indexing.
Track latency, errors, feedback, freshness, and sampled quality signals. Alert on meaningful regressions and investigate drift.
Evaluate the whole experience: combine offline quality measures with latency, cost, user feedback, and failure rates. Metrics are diagnostic signals—not a substitute for reviewing representative answers and their evidence.
10 · From prototype to operations
11 · Reference
| Term | Meaning |
|---|---|
| ANN search | Approximate nearest-neighbor search: an efficient way to find vectors close to a query vector. |
| BM25 | A sparse, keyword-based ranking method commonly used for exact or lexical matching. |
| Chunk | A retrievable unit of a source document created during ingestion. |
| Embedding | A numerical representation of text (or other data) used to compare semantic similarity. |
| Hybrid search | A retrieval approach combining signals such as dense vector similarity and sparse keyword search. |
| Reranker | A second-stage model or method that reorders candidate passages for a particular query. |
| RRF | Reciprocal Rank Fusion, a method for combining ranked result lists from multiple retrievers. |
| Grounding | Constraining a response to evidence or other specified context rather than unsupported model knowledge. |
| Golden dataset | A maintained set of representative queries, expected evidence, and sometimes reference answers for evaluation. |
| Multi-hop retrieval | Retrieving evidence in multiple connected steps to answer questions that require several facts. |
This overview synthesizes concepts and architecture patterns from the RAG Interview System repository. It is an independent educational guide; architecture labels and examples are summarized for orientation, not a guarantee of implementation quality.
