In most sales teams and contact centres, a manager listens to a handful of calls each week. Industry estimates put manual review at just 1–2% of calls (AssemblyAI). The other 98% are a black box: missed upsells, promises customers should never have heard, scripts skipped under pressure, mandatory phrases forgotten. Coaching ends up based on anecdotes and on whichever complaint was loudest.
AI quality control changes the economics. Every recorded call is transcribed, scored against your checklist and summarised, and the patterns are on a dashboard the next morning. Because processing costs are measured in cents per minute, reviewing 100% of calls becomes cheaper than having a supervisor review 2%. Of all voice AI use cases, this is the most mature and the least risky: it never speaks to a customer and involves no outbound calling.
This article covers how AI call QA works, how to design a scorecard that people trust, what the market charges, how to roll it out in 30 days and what EU law allows.
What AI call QA actually does
A working QA pipeline has six stages:
- Capture. Recordings arrive from your telephony provider, cloud phone system or CRM.
- Transcription with speaker separation. The system knows who is the agent and who is the customer, which matters for every criterion.
- Scoring. A language model checks the transcript against your scorecard and returns a yes/no or a rating for each criterion, together with the quote and timestamp that justify it.
- Summary and fields. Call reason, objections raised, competitor mentions and the agreed next step are extracted and written back to the CRM.
- Dashboards and alerts. Scores by rep, team and criterion; alerts for compliance misses or customers at risk of leaving.
- Coaching loop. Managers review flagged calls, not random ones, and each rep gets two or three concrete examples a week.
The quoted evidence is what makes the system usable. A score of 6 out of 10 starts an argument; a score with the exact sentence where pricing was misquoted starts a coaching conversation.
Designing a scorecard people trust
Most QA projects fail at the scorecard, not the technology. Here is an example for an inbound sales call:
| Criterion | Weight | How the AI checks it |
|---|---|---|
| Greeting with name and company | 5% | Required phrase within the first 30 seconds |
| Discovery: at least three qualifying questions | 20% | Questions about need, timing and budget are present |
| Customer need summarised back | 10% | Agent restates the problem in their own words |
| Correct prices and terms | 20% | Stated prices compared with the approved price list |
| Objection handled according to playbook | 15% | Objection detected, followed by the agreed response pattern |
| Concrete next step agreed | 20% | Date, time or specific action confirmed |
| Mandatory disclosures | 10% | Recording notice and required terms are present |
Four rules keep scorecards useful:
- Keep it short. Eight to twelve criteria. A forty-line checklist produces noise, and nobody coaches on forty things.
- Score observable behaviour. “Asked about timing” can be verified; “showed empathy” cannot, and in the EU it drifts towards territory you must avoid (see the compliance section).
- Prefer binary criteria. Yes/no with evidence beats a 1–10 scale for consistency.
- Calibrate before going live. Have the AI and your sales or support lead score the same set of around 100 calls, compare the results criterion by criterion and rewrite the wording until they agree. Skip this and reps will dismiss the scores within a week.
Sales calls and support calls need different scorecards
The pipeline is the same, but what “good” means differs. Mixing both into one checklist is a common reason why scores feel arbitrary.
| Dimension | Sales scorecard | Support scorecard |
|---|---|---|
| Primary outcome | Next step agreed, deal advanced | Issue resolved on first contact |
| Key behaviours | Discovery, value framing, objection handling | Verification, accurate diagnosis, clear instructions |
| Accuracy checks | Prices, discounts, contract terms | Policies, refund rules, troubleshooting steps |
| Risk flags | Over-promising, unapproved discounts | Wrong advice, missing escalation, repeat contact |
| Useful extracted fields | Budget, timing, competitors mentioned | Product, root cause, follow-up needed |
| Who calibrates | Head of sales | Support team lead |
Start with whichever team has the clearer business metric, prove the value there, then copy the setup with a separate scorecard for the second team. Questions the dashboard should answer within the first month are simple: which criteria fail most often, which reps improve after coaching, which objections recur, and which call types end without a next step.
What it costs: the vendor landscape
Published prices and third-party estimates as of October 2026:
| Vendor | Pricing |
|---|---|
| Gong | $1,300–1,600 per user per year, plus a platform fee of $5,000–50,000 per year and onboarding of about $7,500; annual increases of 5–10% are reported (MarketBetter) |
| Observe.AI | Minimum of 100 seats, roughly $60,000–180,000 per year for 100 seats; a separate module on AWS Marketplace costs $69 per agent per month (CloudTalk) |
| CallMiner | Average contract about $88,900–101,900 per year, with a minimum of around 60 agents (Macha) |
| NICE CXone | From $110 per agent per month (CloudTalk) |
| Level AI | Estimated $75–185 per agent per month plus $1,500 per integration (Balto) |
| evaluagent / Zendesk QA | $35–65 per user per month / $50 per agent per month (Intryc) |
| Convin | About $0.04–0.07 per minute (PricingNow) |
| Retell AI QA (for AI agents) | $0.10 per minute after 100 free minutes (Retell AI) |
Two patterns stand out. First, the enterprise leaders are built and priced for large organisations: minimum seat counts of 60 or 100 put them out of reach for a sales team of twelve. Second, the underlying processing is cheap. Twilio lists transcription at $0.024 per minute (Twilio), batch speech-to-text can be cheaper, and an LLM scoring pass with an efficient model adds relatively little on top. In practice, raw processing lands at a few cents per minute. That gap between cost and price is why teams of 5–50 people, the segment the big vendors serve poorly, can often get a better fit from a lean custom setup.
A quick sizing example: a team of ten reps producing 15,000 call minutes a month would pay roughly $600–1,500 a month at usage-based market rates of $0.04–0.10 per minute. Compare that with the time a manager currently spends listening to calls, and with the revenue impact of one extra percentage point of conversion.
Buy, build or hybrid
| Option | Best for | Watch out for |
|---|---|---|
| Enterprise platform (Gong, Observe.AI, CallMiner) | Large sales or contact-centre organisations with dedicated enablement teams | Seat minimums, platform fees, annual increases |
| Helpdesk-native QA (Zendesk QA and similar) | Support teams already living in that helpdesk | Mostly text-first; voice depth varies |
| Usage-based QA tools | Teams that want to start fast with standard criteria | Limited control over scoring logic and data location |
| Custom pipeline | SMB and mid-market teams with specific checklists, CRM write-back, several languages or EU data residency needs | Needs an owner for maintenance and calibration |
Our broader framework for this decision is in build vs buy AI.
A 30-day rollout plan
Week 1: data. Connect recordings and transcription. Pull 100 recent calls covering good, average and bad examples. Check audio quality and speaker separation.
Week 2: scorecard and calibration. Draft the scorecard with your sales or support lead. Score the 100 calls with both the AI and the lead, compare and refine the wording until results match closely on every criterion.
Week 3: go live. Switch on scoring for 100% of calls. Send a daily digest to managers: lowest-scoring calls, compliance misses, recurring objections.
Week 4: coaching rhythm. Each rep receives two or three flagged calls with timestamps and one clear focus for the week. Team-level patterns go into the weekly sales or support meeting.
A good weekly coaching session takes 30 minutes and follows the same pattern: one team-wide trend from the dashboard, two short call excerpts that show the gap and the target behaviour, and one agreed focus for the next week. Reps who see their own scores improve on that focus become the strongest advocates for the system.
Fix the pilot metrics in advance: share of calls reviewed (from 1–2% to 100%), average scorecard result, conversion or resolution rate, and hours managers spend on call review.
QA for your AI voice agents
If you run an AI receptionist or other voice agents, the same pipeline monitors them. Typical bot failures are easy to score automatically: an invented discount or policy, a transfer that never reached a person, a misheard phone number or date, an agent that talked over the caller. Reviewing bot calls with the same scorecard logic as human calls closes the loop between launch and ongoing quality. For the testing side before launch, see evals: measuring AI quality before and after launch.
Compliance: what the EU and the US require
- No emotion recognition of employees in the EU. Article 5(1)(f) of the AI Act has prohibited emotion recognition in the workplace since 2 February 2025 (AI Act, Article 5). Prohibited practices carry fines of up to €35 million or 7% of worldwide turnover (ActuIA). For QA this means no “agent sounded stressed” scores: assess script adherence, accuracy and outcomes. If you are considering any sentiment features, get legal advice first.
- GDPR basics. You need a legal basis for recording (legitimate interest or consent), a notice at the start of the call, data processing agreements with every vendor in the chain (US platforms via standard contractual clauses or the Data Privacy Framework) and defined retention periods. Transcripts are personal data too. Call recording also typically calls for a data protection impact assessment.
- Germany. Recording the spoken word without consent is a criminal offence under §201 of the German Criminal Code, punishable by up to three years in prison (§201 StGB).
- United States. Several states require all-party consent to record, so the standard practice is a clear “this call may be recorded” notice at the start of every call. If your calls are outbound, read our guide to legal AI phone calls.
Be transparent with your own team as well. Explain what is scored, why and how scores are used. QA that feels like surveillance gets gamed; QA that feels like coaching gets adopted.
Common mistakes
- A scorecard with forty criteria. Nobody can act on it.
- Skipping calibration. Reps stop trusting the scores and managers stop opening the dashboard.
- Using QA as surveillance. Ranking people publicly without coaching creates resistance, not improvement.
- Scoring emotions of employees in the EU. It is prohibited; design it out from the start.
- Trusting transcripts blindly. Real-world word error rates are often two to three times worse than lab benchmarks, and 8 kHz phone audio makes it worse (AssemblyAI). Require quoted evidence and spot-check.
- Dashboards without a rhythm. If no meeting and no one-to-one uses the data, the project quietly dies.
- No CRM write-back. Summaries and next steps belong in the deal record, where follow-up actually happens.
Conclusion
AI call QA is the most dependable entry point into voice AI. It shows value within weeks (“we listened to every call, here is where deals and customers are lost”), carries little risk because it never speaks to customers, and costs a fraction of enterprise licences when it is built around a focused scorecard. The software is the easy part; the method of short criteria, calibration and a coaching rhythm is what moves conversion and service quality.
If you would like to see what 100% call coverage would reveal in your team, book a free 15-minute AI audit. We will look at your call volumes, tools and goals, and tell you honestly whether a platform or a lean custom setup fits better. More about our voice and call QA services.