Retrieval-Augmented Generation (RAG)

Large language models are limited to the data they were trained on: they do not know your company's current price list, internal procedures or an announcement published yesterday. On topics they do not know, they can also produce convincing but wrong answers. RAG addresses this by giving the model relevant documents along with the question. The model then answers from the sources in front of it rather than from memory, and can show those sources.

The approach was named in a research paper published in 2020, and today it underpins most enterprise AI assistants, support bots and documentation search tools. Because many generative search engines also fetch sources from the web and ground their answers in them, RAG explains why GEO work pays off.

• Knowledge sources: The documents, databases and systems the model will consult; which sources count as trustworthy is decided up front.
• Preparation and chunking: Cleaning documents, splitting them into meaningful chunks and tagging them with metadata such as source, date and permissions.
• Embeddings and index: Converting chunks into vector representations and writing them to a searchable index, often alongside keyword search.
• Retrieval and re-ranking: Finding the chunks most relevant to the question and re-ranking them to select the best.
• Answer generation: The model answers using only the retrieved context and cites its sources.

Think of a company whose support team works with hundreds of pages of product manuals and policy documents. Staff spend minutes searching across documents to find the right answer. A RAG-based assistant receives the question, finds the relevant sections, writes an answer grounded in them and shows which part of which document it used.

When a document is updated, the model does not need retraining; only the index is refreshed. Documents the user is not authorised to see are filtered out at the retrieval stage. The assistant can also reach these documents through a standard protocol such as MCP.

RAG is measured in two separate stages. At the retrieval stage, the question is whether the right documents are found: how often truly relevant chunks appear among the top results, and what share of chunks are irrelevant. At the generation stage, the answer is assessed for faithfulness to the sources, whether it actually answers the question and whether sources are cited correctly.

To do this, an evaluation set built from real user questions is prepared and every change is tested against the same set. Response time, cost per query and user feedback are operational metrics monitored on a regular basis.

Fine-tuning retrains the model's weights on additional data; it is suited to changing the model's style, format or behaviour on a particular task. RAG, by contrast, gives the model changing information at run time. If information is updated frequently, sources need to be cited or access permissions matter, RAG is the more suitable and more auditable route. The two methods can also be combined. Which one to choose is decided by use case in our AI-native system design and solutions work.