> ## Documentation Index > Fetch the complete documentation index at: https://help-center.relyable.ai/llms.txt > Use this file to discover all available pages before exploring further. # Building Production-Ready Voice Agents > A comprehensive guide to building, testing, and deploying AI voice agents that excel in production This guide walks you through the complete process of building a production-ready AI voice agent from scratch, following best practices learned from building hundreds of voice agents for real clients. This is not a beginner tutorial. We assume you're familiar with voice AI platforms like Vapi or Retell AI. If you're brand new, check out their beginner tutorials first. ## The Reality of Voice Agent Development Here's what typically happens: 1. You build a voice agent 2. You call it 5-10 times 3. It sounds great! Everything works perfectly 4. You deploy to production 5. It fails 40-50% of the time with real users **Why?** Because manual testing can't simulate: * Different accents and speaking styles * Frustrated or stressed users * Edge cases and unexpected responses * The scale of hundreds or thousands of calls This guide shows you how to build agents that actually work in production, not just in testing. ## Overview: Building an Agent Start-to-Finish We'll build a complete inbound voice agent for a real estate company that can handle: * Property inquiries (buying, selling, renting) * Lead qualification * Information capture (name, address, timeline) * Conditional logic based on user responses Then we'll stress-test it with Relyable to find and fix issues before real customers encounter them. ## Step 1: Plan Your Agent with a Call Flow Before writing a single line of prompt, map out your agent's conversation flow visually. This makes prompt engineering dramatically easier. ### Create a Call Flow Diagram Use a tool like [Whimsical](https://whimsical.com), [Miro](https://miro.com), or any diagramming tool to map out: 1. **First Message** - What the agent says when it answers 2. **Pathways** - Different conversation flows based on user intent 3. **Questions** - Information to capture at each step 4. **Conditions** - Branching logic (if yes → do this, if no → do that) 5. **End States** - How conversations conclude ### Example: Real Estate Agent Flow ``` Inbound Call Started ↓ "Hi, this is Emily. I'm an AI agent from Inflate Real Estate. How can I help you today?" ↓ ┌─────────────┼─────────────┐ ↓ ↓ ↓ BUYING SELLING RENTING ``` **Buying Pathway:** 1. "Great! Which property were you interested in?" 2. "Could you spell out your full name?" 3. "Do you have another property to sell first?" * If YES → "Would you like our assistance selling it?" * If YES → "What's your current address?" * If NO → "Would you like to book a walkthrough?" **Selling Pathway:** 1. "Great! Could you provide your address?" 2. "Just to confirm, your address is \[repeat address]. Is that correct?" 3. "Could you spell out your full name?" 4. "What's your reason for selling?" 5. "Have you done any repairs recently?" 6. "What's your timeline for getting this sold?" **Renting Pathway:** 1. "Great! Which property were you interested in?" 2. "Could you spell out your full name?" 3. "What's your timeline for moving in?" 4. "Would you like to book a walkthrough?" The more detailed your call flow, the easier prompt engineering becomes. Spend 20-30 minutes on this before touching your prompt. ## Step 2: Configure Your Voice Agent Settings Before building your prompt, set up your agent configuration in Vapi or Retell. ### Model Selection **For Vapi:** * **GPT-4o** - Reliable, fast, good for most use cases (recommended for starting) * **GPT-4o-mini** - Cheaper, slightly less capable * **Claude 3.5 Sonnet** - Alternative, good reasoning * **GPT-5** (new) - Test carefully before using in production Start with GPT-4o until you've tested your agent thoroughly. Don't use brand-new models in production right away. ### Voice Selection Choosing the right voice is critical: **Vapi Voices:** * Pre-tuned for phone calls * Sound realistic and not glitchy * Limited selection (\~10-15 voices) * Recommended for production - they're tested and reliable **ElevenLabs Voices:** * Larger selection * Can sound more natural * **Warning:** Some voices sound great but glitch after hundreds of calls * Test extensively if using ElevenLabs Never deploy a voice to production without testing it on at least 50-100 calls. Some voices that sound incredible will screech or glitch under load. ### Transcriber Settings For most agents, the defaults work well: * **Deepgram** for Vapi (recommended) * Start with default settings * Only adjust after you've tested and identified specific transcription issues ## Step 3: Structure Your Prompt A well-structured prompt is critical for reliable agent performance. ### Recommended Prompt Structure ```markdown theme={null} # ROLE You are **Emily**, a friendly AI agent from **Inflate Real Estate Services**. You help answer inbound phone calls, assisting clients with selling, buying, or renting properties. # TASK Follow the pathways below based on what the caller needs. Follow each question step-by-step in order. ## Buying This is the pathway to follow when someone is interested in buying one of our properties. 1. Great! Which of our properties were you interested in? 2. Thank you. Could you please spell out your full name for us? 3. Once their name has been captured, repeat it back to them letter by letter. 4. Do you have another property you need to sell first? If the user responds yes to question 4, please ask: `No worries. Would you like our assistance in selling this property?` If the user responds no to question 4, please ask: `No worries. Would you like to book a walkthrough?` If they said yes to wanting help selling: 5. Great! Could you please provide us with your current address? 6. Thanks. Would you like to book a walkthrough for the new property? ## Selling This is the pathway to follow when someone is interested in selling their property. 1. Great! Could you please provide us with your address? 2. Thank you. Just to confirm, your address is `[read out the address captured]`. Is that correct? 3. Thank you. Could you please spell out your full name for us? 4. Once their name has been captured, repeat it back to them letter by letter. 5. What is your reason for selling? 6. Have you done any repairs recently? 7. What is your timeline for getting this place sold? ## Renting This is the pathway to follow when someone is interested in renting one of our properties. 1. Great! Which of our properties were you interested in? 2. Thank you. Could you please spell out your full name for us? 3. Once their name has been captured, repeat it back to them letter by letter. 4. What is your timeline for moving in? 5. No worries. Would you like to book a walkthrough? # NOTES ## Address Pronunciation When reading out an address, split it into clear words: - "123B" should be read as "one two three B" - "57A Woodson Blvd" should be read as "five seven A, Woodson Boulevard" - Never say "one hundred twenty-three", always "one two three" ``` ### Key Prompt Engineering Principles **Use Markdown Formatting:** AI models are trained on markdown. Using proper formatting helps the AI understand your prompt better: * Use `# Headers` for major sections * Use `## Subheaders` for pathways * Use `**Bold**` for emphasis on key terms * Use numbered lists for sequential steps * Use backticks for exact phrases to say **Be Explicit About Order:** Number your questions explicitly (1, 2, 3...) and tell the agent to follow them step-by-step. Don't assume the AI will figure out the order. **Use Clear Conditionals:** Instead of "If they say yes, help them" use "If the user responds yes to question 4, please ask: `Would you like our assistance?`" - Reference specific question numbers to avoid ambiguity. **Confirm Critical Information:** Always repeat back names (spell back letter by letter), addresses (read back with proper pronunciation), email addresses, and any data going into a CRM. This dramatically improves accuracy. **Define Personality in the Role:** The ROLE section shapes how the agent speaks. Adding "friendly" or "professional" or "empathetic" actually changes behavior significantly. ## Step 4: Test Manually First Before running automated tests, do a quick manual test: Save and publish your agent in Vapi/Retell Call your agent and go through one pathway completely. Check: * Does it read the first message correctly? * Does it follow the question order? * Does the voice sound good? * Is the response latency acceptable? Write down anything that doesn't work. Don't try to fix everything yet - just get a baseline. If your agent completely fails manual testing, fix the obvious issues before automated testing. Automated testing is for finding subtle edge cases, not broken core functionality. ## Step 5: Import to Relyable Follow the [Quick Start guide](/quickstart) to: 1. Create a Relyable account 2. Import your agent 3. Generate test cases 4. Create personas ## Step 6: Generate Comprehensive Test Cases Test cases evaluate specific behaviors. Aim for 15-25 test cases covering: ### Must-Have Test Cases **Identity and Branding:** * Agent introduces with correct name and company * Agent mentions it's an AI system * Agent maintains consistent identity **Conversation Flow:** * Agent asks questions in the correct order * Agent doesn't skip required questions * Agent handles conditional logic correctly **Data Capture:** * Agent captures required information (name, address, email) * Agent confirms captured information back to user * Agent spells back name letter by letter * Agent reads addresses with proper pronunciation **Pathway Handling:** * Agent correctly identifies buying vs selling vs renting intent * Agent follows the correct pathway for each intent * Agent handles pathway changes (e.g., buyer also needs to sell) **Edge Case Handling:** * Agent handles objections ("Why do you need my name?") * Agent handles unclear responses * Agent doesn't give up on difficult customers prematurely ### Setting Test Case Priorities | Priority | When to Use | Impact on Score | | -------- | ---------------------------------------------------------- | --------------- | | Critical | Must work 100% of the time (emergency routing, compliance) | Huge impact | | High | Core functionality (capturing lead info, following flow) | Large impact | | Medium | Important but not critical (tone, using caller's name) | Medium impact | | Low | Nice-to-have (specific phrases, minor details) | Small impact | ## Step 7: Create Diverse Personas Don't just create "normal" callers. Create personas that stress-test your agent: ### Example Personas to Create **The Frustrated Elderly Customer:** Stanley Miller, 79-year-old retired mechanic. Frustrated with life, no patience, speaks in short clipped sentences. Gruff and exasperated. Reluctantly calling because his kids are forcing him to sell his house of 50 years. **The Fast-Talking Young Professional:** Sarah Chen, 28-year-old tech professional. Speaks very quickly, interrupts frequently, expects immediate answers. Impatient with any delays. Used to chatbots and expects perfect performance. **The Non-Native Speaker:** Raj Patel, 45-year-old engineer. Strong Indian accent, speaks slowly and carefully, sometimes struggles with pronunciation. Very polite but needs information repeated sometimes. **The Skeptical Customer:** Mike Johnson, 52-year-old business owner. Doesn't trust AI systems, questions everything, tests whether it's really AI or human. Asks unexpected questions to trip up the system. **The Confused Caller:** Linda Martinez, 67-year-old retiree. Not tech-savvy, easily confused, needs things explained multiple times. Forgets what was just discussed. Very polite but difficult to keep on track. Create 5-7 diverse personas representing your actual customer base plus some worst-case scenarios. This coverage helps you find issues across user types. ## Step 8: Run Automated Tests at Scale Now run your tests: Create 5-10 different scenarios using your personas: * 2-3 buying scenarios * 2-3 selling scenarios * 2-3 renting scenarios * 1-2 complex scenarios (buyer who also needs to sell) Select all scenarios and run them together. This typically takes 10-20 minutes for 5-10 calls. Grab coffee. Relyable will call your agent with each scenario and evaluate against all test cases. ## Step 9: Analyze Results and Find Issues ### Understanding Your Score After tests complete, you'll see an overall score: * **90%+** → Excellent, production-ready * **70-89%** → Good, acceptable for production * **50-69%** → Needs work before production * **Below 50%** → Significant issues, not ready Reality Check: In the video demonstration, an agent that worked perfectly in manual testing scored 51-61% in automated testing. This is normal and exactly why you need automated testing! ### Reviewing Failed Test Cases Click on failed test cases to see: 1. **Which calls failed** - Sometimes a test case fails on 2/5 calls, not all 2. **Why it failed** - AI explanation of what went wrong 3. **The exact conversation** - Full transcript and audio 4. **Suggestions** - How to fix it in your prompt ### Common Failure Patterns **Skipping Questions:** * **Symptom:** Agent jumps to question 5 without asking questions 3 and 4 * **Fix:** Add explicit numbering and: "Follow each question step-by-step in order. Do not skip any questions." **Not Confirming Information:** * **Symptom:** Agent captures name but doesn't spell it back * **Fix:** Make confirmation explicit: "3. Once their name has been captured, repeat it back to them letter by letter." **Giving Up on Difficult Customers:** * **Symptom:** When customer pushes back ("Why do you need that?"), agent says "No worries, have a great day" and ends the call * **Fix:** Add handling: "If the caller objects or questions why you need information, briefly explain the reason and ask again gently. Do not end the call unless they explicitly want to hang up." **Breaking Conditional Logic:** * **Symptom:** Agent asks "Would you like help selling?" even when user said NO to having a property to sell * **Fix:** Be more explicit: "If the user responds NO to question 4, skip to question 6 directly." **Poor Address Pronunciation:** * **Symptom:** Agent says "one hundred twenty-three" instead of "one two three" for address 123 * **Fix:** Add pronunciation guide in a NOTES section showing how to split numbers and addresses. ## Step 10: Iterate and Improve The key to production-ready agents is iteration: Don't try to fix everything. Pick the 2-3 most critical failures and fix those in your prompt. Make your prompt changes in your voice platform, not in Relyable. Relyable is read-only. Click "Sync Prompt" in Relyable to pull the updated prompt. Run the same scenarios again. Did your score improve? Did the specific failures get fixed? Keep iterating. Most production-ready agents take 5-10 iterations to get from 50% to 75%+. ### Tracking Improvements | Iteration | Score | Changes Made | | --------- | ----- | --------------------------- | | 1 | 51% | Baseline | | 2 | 58% | Added step-by-step ordering | | 3 | 63% | Added name confirmation | | 4 | 69% | Fixed conditional logic | | 5 | 74% | Added objection handling | | 6 | 78% | Production Ready | Document what changes you make each iteration. This helps you understand what actually improves performance vs. what doesn't matter. ## Step 11: Enable Live Monitoring Once you reach 70%+ consistently: In Relyable, go to Agent Settings → Enable Call Monitoring Your agent is now ready for real customers Every production call is evaluated against your test cases. You'll see: * Real-time scores * Which test cases are failing in production * Trends over time (is quality improving or degrading?) Get notified when: * Score drops below 70% * Critical test cases fail * Specific issues occur multiple times ## Prompt Engineering Best Practices ### Use Markdown Effectively **Good Example:** ```markdown theme={null} # ROLE You are **Emily**, a friendly AI agent from **Inflate Real Estate**. ## Buying Pathway 1. Ask which property they're interested in 2. Capture their full name ``` **Bad Example:** ``` You are Emily from Inflate Real Estate. When someone wants to buy ask them which property and get their name. ``` ### Preview Your Prompt Use [Markdown Live Preview](https://markdownlivepreview.com/) to see how your prompt renders. This shows you how the AI "sees" your prompt. ### Be Explicit About Everything Don't assume the AI will "figure it out." Be explicit about: * Question order * What to say exactly (use backticks) * When to say it * What NOT to do ### Test One Change at a Time If you change 5 things and the score improves, you don't know which change helped. Change 1-2 things per iteration. ### Use Real Call Examples When you find a failure in testing, paste the transcript into your prompt as an example: ```markdown theme={null} ## Handling Objections If a caller asks "Why do you need my name?", respond with: "I just want to make sure we have the right information to help you. It ensures our team can follow up properly." Example: Caller: "Why do you need my name?" Agent: "I just want to make sure we have the right information to help you. It ensures our team can follow up properly. Could you spell out your full name for us?" ``` ## Advanced: Data Capture and CRM Integration Once your agent is reliable, integrate it with your CRM: ### Capture Data with Functions In Vapi, set up functions to capture: * Name * Phone number * Email * Address * Other lead information ### Confirm Before Sending Always confirm information before sending to CRM: ```markdown theme={null} 5. Repeat back: "Just to confirm, your email is `[email]`. Is that correct?" 6. If yes, save to CRM. If no, ask them to repeat it. ``` ### Test the Integration Run automated tests with the CRM integration enabled. Verify: * Data is captured correctly * Data is confirmed before sending * Invalid data doesn't get sent * The agent handles CRM errors gracefully ## Performance Optimization ### Reducing Latency If your agent is too slow: 1. **Use a faster model** - Try GPT-4o-mini instead of GPT-4o 2. **Shorten your prompt** - Remove unnecessary details 3. **Reduce tool calls** - Fewer function calls = faster responses 4. **Use Vapi's latest features** - They constantly optimize latency ### Improving Speech Quality If speech sounds robotic or glitchy: 1. **Switch voices** - Some voices are more reliable 2. **Adjust WPM (words per minute)** - 150-180 is natural 3. **Add punctuation to responses** - Helps with cadence 4. **Use SSML tags** - For pauses and emphasis (platform-dependent) ## Common Pitfalls **Testing Only Happy Paths:** Don't just test scenarios where everything goes perfectly. Test difficult customers, people who object or refuse to give information, people who go off-script, and multiple intents in one call. **Assuming Manual Testing Is Enough:** You CANNOT catch all issues with manual testing. You'll miss rare edge cases, issues with specific accents, problems at scale, and inconsistencies (works 80% of the time). **Over-Engineering Too Early:** Start simple. Get the basic flow working well before adding complex branching, advanced features, or CRM integrations. A simple agent that works is better than a complex one that's unreliable. **Ignoring Low-Priority Test Cases:** Just because something is "low priority" doesn't mean ignore it. If low-priority test cases fail 100% of the time, fix them. They're still part of the user experience. **Not Documenting Changes:** Keep notes on what you change and why. This helps you understand what works, train team members, debug issues later, and build knowledge for future agents. ## Checklist: Is My Agent Production-Ready? ### Functionality Requirements * Agent introduces itself correctly 100% of the time * Agent follows conversation flow step-by-step * Agent captures all required information * Agent confirms critical data (name, email, address) * Agent handles conditional logic correctly * Agent doesn't skip questions * Agent doesn't give up on difficult customers ### Testing Requirements * Agent scores 70%+ on automated tests consistently * Tested with at least 5 different personas * Tested with 20+ different scenarios * All Critical test cases pass 100% * All High test cases pass 90%+ * Edge cases are handled gracefully ### Quality Standards * Voice sounds natural and doesn't glitch * Latency is under 1 second average * Words per minute is 150-180 (natural pace) * Agent sounds friendly and professional * No awkward pauses or weird inflections ### Production Monitoring * Live monitoring is enabled in Relyable * Alerts are set up for critical failures * Team knows how to check performance * Process for responding to issues is defined ## Next Steps Integrate Relyable into your development workflow Get your first agent tested in 10 minutes Access your Relyable workspace Join other voice AI builders ## Resources * [Vapi Documentation](https://docs.vapi.ai) * [Retell AI Documentation](https://docs.retellai.com) * [Markdown Guide](https://www.markdownguide.org/) * [Whimsical (Call Flow Tool)](https://whimsical.com)