AI glossary · Building and connecting

RAG

Also called: Retrieval-augmented generation · AI with a knowledge base

What is RAG?

A language model such as Claude or ChatGPT only knows what it read in training. It does not know what is in Tuesday's site report. Ask anyway and it guesses, often convincingly.

RAG changes that. Think of an open-book exam: the AI may look things up in your documents before every answer, such as project files, price lists or manuals. The term comes from a research paper published in 2020.

How does it work?

The documents are prepared once. After that, every question runs the same way:

  • Prepare: The documents are split into short passages and stored so they can be searched by meaning, usually with embeddings.
  • Search: When a question comes in, the system fetches the passages that fit best.
  • Answer: The passages go to the model with the question. It writes the answer from them and can name the source document.

Why does it matter for your business?

Because a business keeps its knowledge in emails, notes, photos and reports. With RAG, someone on the team asks, “Was the new delivery date confirmed in writing?”, and gets the answer with its source instead of searching.

The source is what matters: you can check the answer. The limit is just as clear: RAG is only as good as the documents. If price lists are outdated or documents contradict each other, the AI passes the mess on.

How does OsAI use it?

OsAI builds RAG systems for clearly defined bodies of knowledge, such as the files of one project. The live demo “Ask the project file” on how it works shows this with a made-up example: the answer names its sources, and they light up in the file.

The free AI data readiness check shows whether your documents are good enough. A guide on AI agents and RAG (in German) explains when simple automation will do.

Frequently asked questions

What is the difference between RAG and fine-tuning?

With RAG, the AI looks things up in your documents and the model stays the same. With fine-tuning, the model is trained further on your own examples. For knowledge that changes, such as prices or dates, RAG is usually the better fit: you only swap the document.

Does an AI with RAG stop making things up?

Less often, but it still happens. RAG improves accuracy and allows citations, yet the AI can misread a passage or fill gaps. So the source belongs with the answer, and a person checks anything important.

Do my documents go to the AI provider with RAG?

The matching passages go to the model with every question, so with a cloud service they reach its provider. Under the EU GDPR you need a data processing agreement under Art. 28. For very sensitive documents, a locally run model is an option.

Sources

Last updated: