What this covers. Your agent needs facts: hours, pricing, services, policies, a long list of FAQs. You have three places to put them, and beginners reach for the wrong one constantly. This lesson explains the three options and gives you a one-glance decision guide.
The one idea to hold onto. Most small businesses never need RAG. A short prompt plus a small knowledge base injected in full covers the vast majority of voice agents, and it's both faster and more reliable than the fancier option.
This pairs with Build a Voice Agent From a Transcript and the prompt lesson. If you understand the word budget from those, this slots right in.
There are exactly three places a fact can live.
The fact is part of the system prompt itself, read on every single turn. Always available, zero retrieval risk, but it eats your word budget. Best for the handful of facts the agent uses constantly.
A separate document, but its full contents are still injected into every turn. Same reliability as putting it in the prompt, but kept in its own file so it's easier to manage and reuse across agents. Best for a small FAQ or policy doc.
The document is chunked and indexed. On each question, only the most relevant chunks are pulled in via semantic search. This is the only option that scales to large documents, but it adds latency and a retrieval step that can occasionally miss. Best for big manuals.
Match the size of the content to the right home. The thresholds below are the ones the workflow follows.
| What you have | Where it goes | RAG? |
|---|---|---|
| A few facts used every call (hours, service-call fee, service area) | The prompt | No |
| A small FAQ or policy doc (under ~5,000 tokens, ~300 lines) | A knowledge base in prompt mode |
No |
| A medium doc (~5,000 to 50,000 tokens) | A knowledge base in auto mode |
Yes |
| A large manual (50,000+ tokens) | A knowledge base in auto or rag mode |
Yes |
If your content fits in a couple of pages, you do not need RAG. Use prompt mode by default. Reach for RAG only when the content is genuinely too large to fit in context.
Not sure how to split yours? Paste your facts and let Claude apply the rule for you:
Facts the agent leans on in almost every conversation belong in the prompt, where they're always present and never need to be retrieved.
For a dental clinic, that's the opening hours, the new-patient policy in one line, and whether they take walk-ins. For Riverside HVAC, it's the service area, the standard diagnostic fee, and same-day-or-not. These are short, they're load-bearing, and you never want the agent to "miss" them.
Keep these to a few lines. If your "facts in the prompt" section starts running long, that's the signal to move the overflow into a knowledge base.
When you have a real list of questions and answers, twenty financing questions, a return policy, warranty terms, that's a knowledge base, not prompt text. It would blow your prompt budget and slow every turn.
For a small-to-medium business this almost always means a knowledge base in prompt mode: the doc lives in its own file but is still injected in full, so there's no retrieval step that can fail.
Format it as headers plus short paragraphs, not a giant wall of Q&A. Keep answers to one or two sentences, and don't duplicate facts that are already in the prompt.
When you add a knowledge base document, upload it as a file (TXT, PDF, DOCX, HTML). File uploads get proper content extraction. Pasting raw text in through the text option produces messier extraction and worse results in both prompt mode and RAG.
RAG (retrieval-augmented generation) chunks a large document, embeds it, and pulls only the relevant pieces per question. It's the right tool when content genuinely cannot fit in context, a full product catalog, a hundred-page policy manual.
But it comes with costs beginners underestimate:
prompt mode.Built by AI Service Engine · back to the main voice-agent lesson · install Claude Code first