Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) combines a large language model with an external knowledge retrieval step: relevant documents are fetched from a corpus and injected into the prompt so the model can answer with up-to-date, grounded information rather than relying solely on its training data.
itArtificial intelligence and machine learning | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic — Retrieval-Augmented Generation
Retrieval-augmented generation, or RAG, is what happens when a language model is not expected to remember the entire office library, its replacement policies, and the one PDF everyone fears. Before RAG, the usual choices were to paste documents into a prompt until it protested, or train knowledge into model weights and wait for that knowledge to age. RAG retrieves relevant material when a question arrives, then gives that material to the model as context.
The important split is retrieve, then generate. A knowledge base holds documents chopped into chunks, meaning passages small enough to search and fit in the prompt. Each chunk gets an embedding, a numerical representation that lets similar meanings sit near one another. The question gets one too. Search returns nearby chunks, and the model writes an answer from those selected pieces. The library has not been swallowed whole; it has merely been given a very fast, very literal index card system.
This is why RAG is not a product you install with one heroic button. It is an arrangement of components: an embedding model, a search index, a chunking strategy, a language model, and the orchestration between them. Vector search finds semantic neighbors. Keyword search catches exact product names and error codes. Hybrid search combines both because language has the irritating habit of being precise and indirect in the same sentence. A re-ranker can then put the most relevant evidence nearer the front of the queue.
The surprise is that the generator is rarely the first place to blame. If retrieval chooses the wrong chunks, the answer can be fluent, cited, and wrong with remarkable composure. Chunks that are too large bury the useful sentence; chunks that are too small lose its context. A finite context window means there is no safe setting called all of it. Access control and freshness matter for the same reason: the model must receive only current material that the requester may see.
Read the Intro for the full architecture and the decisions around chunking, embeddings, and security. Use the Slides for the offline indexing path and online query path in one view. Keep the Cheatsheet nearby when comparing keyword, vector, and hybrid search or tuning top-k. The Practice tab turns the uncomfortable question into a useful habit: did the right evidence reach the prompt before the answer began talking?
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview
Supports
- RAG is a pattern that extends LLM capabilities by grounding responses in proprietary content
- Challenges of RAG include query understanding, multi-source data access, token constraints, response time, and security
- Hybrid search combines keyword and vector search for better recall
- Semantic ranking re-scores results based on meaning not just keywords
- Document-level security trimming and access control at query time
- Agentic retrieval decomposes complex questions into focused subqueries executed in parallel
- Content preparation involves chunking, vectorization, and language analysis
- Maximize relevance with hybrid queries and semantic ranking
- https://docs.aws.amazon.com/prescriptive-guidance/latest/retrieval-augmented-generation-options/introduction.html
Supports
- With RAG the foundation model references an authoritative data source outside its training data before generating a response
- RAG addresses challenges of using generative AI models to answer questions from custom documents
- Options include fully managed services and custom RAG architectures
- https://www.ibm.com/think/architectures/patterns/genai-rag
Supports
- RAG is an architectural pattern enabling foundation models to produce factually correct outputs for specialized or proprietary topics not in training data
- RAG connects language models to external knowledge at inference time
- https://arxiv.org/abs/2005.11401
Supports
- RAG combines parametric memory (the pre-trained model) with non-parametric memory (retrieved documents)
- The retrieve-then-generate approach for knowledge-intensive NLP tasks
- RAG models generate responses conditioned on retrieved documents rather than relying solely on parameters
- The 2020 paper introduced RAG models using a dense vector index of Wikipedia
- https://arxiv.org/abs/2002.08909
Supports
- REALM combined language-model pretraining with retrieval from a large corpus
- https://aclanthology.org/2020.emnlp-main.550/
Supports
- Dense Passage Retrieval used dual encoders for dense passage retrieval in open-domain question answering
- https://aclanthology.org/2022.naacl-main.272/
Supports
- ColBERTv2 paired late interaction with residual compression for efficient retrieval
- https://aclanthology.org/2023.acl-long.99/
Supports
- HyDE generated a hypothetical document for zero-shot dense retrieval before retrieving real documents
- https://arxiv.org/abs/2310.11511
Supports
- Self-RAG proposed on-demand retrieval and reflection for relevance and support decisions
- https://arxiv.org/abs/2401.15884
Supports
- Corrective RAG used a retrieval evaluator to assess retrieved-document quality and select retrieval actions
- https://arxiv.org/abs/2401.18059
Supports
- RAPTOR recursively clustered and summarized chunks into a tree for retrieval across long documents
- https://github.com/microsoft/graphrag
Supports
- Microsoft first released GraphRAG in July 2024 as a graph-based RAG research system
- https://aclanthology.org/2025.tacl-1.36/
Supports
- mtRAG provided a human-generated multi-turn RAG benchmark with unanswerable and non-standalone questions
- https://arxiv.org/abs/2401.05856
Supports
- An experience report from three RAG case studies identified failure points and argued that validation is feasible during operation