What this covers. A voice agent lives or dies on its system prompt. The prompt is the agent's personality and its rulebook in one document. Get the structure right and the agent stays on-task, asks for the right things, and sounds human. Get it wrong and the agent rambles, invents answers, or sits in dead silence while a caller waits.
The one idea to hold onto. On a phone call, every extra word in your prompt is extra time the caller waits to hear a reply. A lean, well-ordered prompt is a fast prompt. This lesson shows you the structure and the word budget that keep an agent both reliable and quick.
This builds on Build a Voice Agent From a Transcript. If you have the workflow cloned, Claude Code writes prompts in this shape automatically. This lesson is so you understand what it's doing and can edit by hand.
The system prompt controls what the agent says and how it behaves: its personality, its goal, its rules, when to use tools, and what to do when something breaks.
It does not control conversation mechanics like turn-taking, which voice is used, or which languages the agent speaks. Those live in the agent's config, not the prompt. Keep that separation in mind so you don't try to solve a settings problem with prompt text.
A chat user will scroll back and re-read. A phone caller cannot. The agent gets one spoken shot at each turn, so the prompt has to make correct, short behavior the path of least resistance.
Organize the prompt with markdown headings, in this order. The models behind voice agents are tuned to pay extra attention to clear section headings (especially # Guardrails), and clean boundaries stop one instruction from bleeding into another.
Who the agent is, in two or three sentences. Name, the business it represents, and its manner. Example: "You are Riley, the virtual receptionist for Acme Plumbing. You are warm, efficient, and never pushy."
The job, as a short numbered workflow. What to collect and in what order, what tool to call, when the call is done. Example: "Book service appointments. Collect name, phone, address, and the problem, one field at a time."
The non-negotiable rules. What the agent must never do, when to refuse, when to hand off. This is where you stop the agent from quoting prices it shouldn't, or promising things it can't deliver.
How to convert spoken input into written values for tools. "five five five, one two three four" becomes "5551234." "john dot smith at gmail dot com" becomes "john.smith@gmail.com." One or two lines is enough.
When and how to call each tool, with sequencing. "Always check availability before booking." This section also tells the agent to speak before a tool fires so the caller isn't met with silence.
What to say and do when a tool fails. Acknowledge, do not guess, retry once, then offer a callback or a transfer. This single section prevents most hallucinated answers.
This is the rule beginners skip, and it's the one that most affects how the agent feels on a call.
Every token the model has to read before it can answer adds latency. A bloated prompt is the difference between a reply that lands in under a second and one the caller waits four to seven seconds for.
The example agent in the workflow was trimmed from about 2,550 words down to roughly 430. Time-to-first-byte dropped from four-to-seven seconds to under a second. Same behavior, a fraction of the wait. Lean is fast.
Sections that sound useful but mostly add bloat: a separate "Environment" block (it repeats Personality), a separate "Tone" block (fold one line into Personality), long "Data Collection" instructions (one sentence in Goal does it), and worked example dialogues (a capable model handles these fine without them).
You cannot bold your way to reliability with a voice model, but you can flag what matters. The convention is simple: append "This step is important" to the one or two instructions the agent absolutely must follow.
Repeating your single most important rule once more, later in the prompt, also helps reinforce it. Use this sparingly. If everything is marked important, nothing is.
The prompt should explicitly tell the agent to speak the way people speak on the phone: in short turns.
A good default instruction is to keep replies to one or two sentences unless the caller asks for more detail. Long paragraphs that read fine on a screen sound like a monologue out loud, and they invite the caller to interrupt.
Pair this with a filler instruction in the Tools section: "Before calling any tool, briefly tell the caller what you're doing." That one line removes the dead air a caller would otherwise hear while a tool runs.
Here's the shape of a complete, lean prompt for a generic business. Notice how short each section is.
You do not have to write it from scratch. Hand the structure to Claude and let it draft yours, then trim against the checklist below:
Built by AI Service Engine · back to the main voice-agent lesson · install Claude Code first