What is RAG — and when does your business actually need it?
RAG is how an AI system answers from your documents instead of its imagination. Here is the plain-English version, and the honest checklist for whether you need one.
Ask a general-purpose AI model about your refund policy and it will answer fluently — from nothing. It has never seen your policy. The fluency is the danger: wrong answers arrive with the same confidence as right ones.
Retrieval-augmented generation (RAG) is the standard fix. Before the model answers, the system retrieves the relevant passages from your actual documents — policies, contracts, manuals, tickets, product data — and instructs the model to answer from those passages. The model stops being an oracle and becomes a very good reader.
The mechanics, without the jargon
A RAG system does three things. First, it indexes your knowledge: documents are split into passages and stored in a way that lets the system find meaning, not just keywords — a search for 'can I get my money back' finds the refunds clause even though no word matches. Second, at question time it retrieves the handful of passages most relevant to the question. Third, it hands those passages to the model with the question and asks it to answer from them, citing where each claim came from.
That last part — citations back to source — is what turns an impressive demo into something a compliance officer will sign off on. Every answer carries its receipts.
Problems where RAG is the right tool
- Support teams answering from a large, changing knowledge base — policies updated last week, not at training time.
- Internal search across contracts, SOPs and wikis — 'what did we agree with this vendor about SLAs?'
- Onboarding and HR questions answered from the actual handbook, with the paragraph linked.
- Sales teams querying product specs and past proposals instead of pinging engineering.
- Regulated industries where every answer must trace to a governing document.
When you don't need RAG
If the knowledge fits in a few pages, paste it into the prompt — retrieval infrastructure for twenty FAQs is overengineering. If the task is transformation rather than knowledge — drafting, summarising, classifying, extracting — the model alone usually suffices. And if your documents are wrong or outdated, RAG will faithfully cite the wrong answer: retrieval amplifies your documentation quality, it does not repair it.
The honest first question is not 'do we need RAG' but 'is our knowledge worth retrieving'. Sometimes the highest-ROI step is a documentation cleanup with a RAG system as the forcing function.
What separates a demo from production
A weekend RAG demo is genuinely easy now, which is precisely why so many stall before launch. Production is where the engineering lives: keeping the index fresh as documents change, respecting permissions so the intern's chatbot doesn't quote the executive compensation file, handling the questions where retrieval finds nothing, evaluating answer quality continuously, and failing gracefully to a human when confidence is low.
Access control is the one that bites hardest. Your documents have an audience model — the retrieval layer has to honour it, per query, per user, every time.
How we build them
We build RAG systems as boring, observable infrastructure: your documents stay in your cloud, every answer cites sources, permissions are enforced at retrieval time, and quality is measured against a test set drawn from real questions your team actually asks. If you are weighing whether your use case clears the bar, that assessment is exactly what our first conversation covers — and it costs nothing.
Have a process on your automation list?
We turn AI ideas into production-grade systems — the first conversation is free.
Talk to an engineer →