RAG (retrieval-augmented generation) retrieves relevant documents at query time and feeds them to an LLM, so answers are grounded in your sources, not guessed.
Last verified · 2026-08-26 · by Moe Ameen
Retrieval-augmented generation (RAG) is a technique that connects a language model to an external body of knowledge at the moment it answers. Instead of relying only on what the model absorbed during training, a RAG system first *retrieves* the passages most relevant to the question — from your documents, a knowledge base, a product catalog, past transcripts — and then hands those passages to the model as context, so the answer is *generated* from real source material rather than from the model's memory alone. The one-line version: retrieve the right facts, then let the model write the answer over them.
The value is straightforward. A base model's knowledge is frozen at its training cutoff, it has never seen your private or proprietary data, and when it doesn't know something it tends to guess fluently — the failure mode called [hallucination](/glossary/ai-slop). RAG addresses all three at once: it can pull in information newer than the training cutoff, it can reach data the model was never trained on, and because the answer is conditioned on retrieved passages you can show the reader *where each claim came from*. That grounding-plus-citation property is why RAG became the default architecture for enterprise chatbots, documentation assistants, customer support, and the answer engines that now cite sources inline.
Mechanically, a classic RAG pipeline has two halves. Offline, you *index* your knowledge: split documents into chunks, convert each chunk into a numeric vector with an embedding model, and store those vectors in a vector database. Online, when a query arrives, you embed the query the same way, run a similarity search to pull the closest chunks, optionally rerank them, and stuff the top results into the model's prompt alongside the question. Retrieval doesn't have to be vector-based, though — plain keyword or full-text search (BM25) is often enough, and hybrid setups combine both. The moving parts are the retriever (what it finds) and the generator (what it writes); RAG is the discipline of getting the right context in front of the model before it opens its mouth.
RAG was named and formalized in the May 2020 paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" by Patrick Lewis and colleagues at Facebook AI Research (now Meta AI), with collaborators from University College London and NYU; it was published at NeurIPS 2020. The paper combined a pretrained sequence-to-sequence generator with a dense retriever over a Wikipedia index and showed the hybrid produced more specific, factual, and diverse output than a comparable model working from its parameters alone, setting state-of-the-art results on several open-domain question-answering benchmarks. Lewis's team described RAG as a "general-purpose fine-tuning recipe" precisely because it could bolt almost any generator onto almost any external knowledge source.
The idea sat mostly in research circles until the ChatGPT era made two of its weaknesses impossible to ignore: models confidently invented facts, and they knew nothing about your business. RAG was the obvious fix that didn't require retraining a model, so between 2023 and 2025 it went from a paper to an industry. Vector databases (Pinecone, Weaviate, and others), embedding APIs, and orchestration frameworks like LangChain and LlamaIndex turned "build a RAG app" into a weekend project, and by 2025 AWS, Google, Microsoft, IBM, NVIDIA, and Oracle were all shipping RAG-as-a-service. A parallel current pushed back on the complexity: practitioners began arguing that many teams reach for embeddings and vector stores when a well-tuned keyword search plus query rewriting would solve most of the problem more cheaply — the "RAG is simpler than you think" correction that gathered steam through 2026.
| Platform | Behavior |
|---|---|
| ChatGPT / OpenAI | Custom GPTs, the Assistants/File Search tooling, and "chat with your docs" features are RAG under the hood — you upload files, OpenAI chunks and embeds them, and retrieved passages are injected into the prompt. The model answers from your uploads rather than from training data alone. |
| Claude / Anthropic | Claude is commonly paired with RAG for document Q&A and support bots; its large context window lets more retrieved passages ride in the prompt at once, and it can cite which retrieved chunk each claim came from when asked to attribute. |
| Perplexity and answer engines | A live-web variant of RAG: rather than a private index, the retriever is a search over the open web, and the model synthesizes an answer from the fetched pages with inline citations. This is why answer engines can cite sources at all — the sources are the retrieved documents. |
| Enterprise search / support | The dominant production use. A RAG layer over a company knowledge base, ticket history, or documentation lets a bot answer with current, company-specific facts and link the source, instead of a generic model guessing at internal policy. |
| Coding assistants | Codebase-aware assistants retrieve the relevant files, symbols, or docs for the task and feed them to the model, so suggestions reflect your actual repository rather than generic patterns — RAG applied to source code instead of prose. |
RAG is one of those ideas that sounds fancy and is really just discipline: don't ask a model to answer from memory when you can put the facts in front of it. The whole enterprise-AI stack of the last few years is downstream of that one move. What I'd push back on is the reflex to over-build it. The 2026 correction is right — most "we need a vector database" projects are actually "our search is bad and our queries are worse," and a keyword index plus an LLM that rewrites the query gets you most of the way for a fraction of the cost. Reach for embeddings when you have a genuine semantic-matching problem, not because the tutorial used them.
The mindset generalizes past chatbots, and it's the one that matters for anyone generating content. The reason generic AI copy reads as [slop](/glossary/ai-slop) is that it's written from the model's frozen, averaged memory — no grounding. The fix is the same as RAG's: condition generation on real source material. That's exactly the shape of how Kompozy works — you feed it your actual inputs (a video transcript, an article, a source URL, past posts) and a [Persona Brief](/glossary/persona-brief) that fixes voice and banned words, and the engine generates the whole spread — [text posts](/glossary/output-buckets), carousels, [shorts](/glossary/clipped-short), a blog, a newsletter — grounded in what you actually said rather than what a base model would guess. It isn't a RAG database, but it's the same principle applied to publishing: retrieve your real material first, then let the model write over it. See [context engineering](/glossary/context-engineering) for the broader craft of getting the right information into a model's window.
RAG is a technique that connects a language model to external knowledge at answer time. Instead of relying only on training data, the system first retrieves the passages most relevant to the question — from your documents, a knowledge base, or the live web — and feeds them to the model as context, so the answer is generated from real source material and can be traced back to its sources.
It fixes three limits of a base model at once: it supplies information newer than the training cutoff, it reaches private or proprietary data the model never trained on, and because the answer is grounded in retrieved passages you can cite where each claim came from — which sharply reduces confident-but-wrong hallucinations. And it does all of that without retraining the model.
In two halves. Offline you index your knowledge: split documents into chunks, embed each chunk into a vector, and store the vectors. Online, when a query arrives, you embed the query, run a similarity search to pull the closest chunks, optionally rerank them, and place the top results in the model's prompt alongside the question so it answers over them. Retrieval can also be plain keyword or hybrid search rather than vectors.
It was named in the May 2020 paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" by Patrick Lewis and colleagues at Facebook AI Research (now Meta AI), with UCL and NYU, published at NeurIPS 2020. The paper paired a generator with a dense retriever over Wikipedia and beat parameters-only models on open-domain question answering, then became mainstream during the ChatGPT era as the standard fix for stale and hallucinated answers.
No. Vector search shines for semantic matching, but for many use cases a well-tuned keyword or full-text search (BM25) plus an LLM step that rewrites the query retrieves as well or better, with less cost, latency, and chunking complexity. A common 2026 recommendation is to build the simple keyword version first and add embeddings only where semantic matching genuinely earns its keep.
It reduces hallucination but does not eliminate it. RAG grounds the answer in retrieved passages, so if the right passage is retrieved the model has the facts in front of it. But if retrieval surfaces the wrong or missing chunk, the model can still answer confidently from bad context — which is why evaluating retrieval quality separately from the generated answer matters.