How to score the quality of sales calls: a criteria set and a sample scorecard

Call scorecards are often built in a hurry: dozens of vague criteria, each reviewer reading them differently, and scores then used to rank staff. The number looks objective but reflects the reviewer more than the person reviewed.

This article covers call scoring for sales and customer care teams. The main sources are Amazon Connect and Google Cloud documentation, McHugh's 2012 paper on rater agreement, Kluger and DeNisi's 1996 review of feedback, and OpenAI's Whisper paper, checked on October 21, 2026.

It answers 6 questions: what scoring is for, how to sample, how to design the scorecard, how to calibrate reviewers, how to give feedback, and how to tell if the scorecard is useful. Click Star already has an article on what AI call analysis is; this one focuses on the scorecard.

Key points

• Score calls to coach and fix process, not to punish: over 1/3 of feedback interventions lowered performance (Kluger and DeNisi, 1996).
• Each criterion is an observable behavior answered with pass, fail or not applicable, with clear instructions.
• Weights are a management choice, not an industry standard. Amazon Connect requires them to total 100%. Split serious errors out as critical fails.
• Measure reviewer agreement with kappa. McHugh treats kappa below 0.60 as inadequate, Google Cloud targets no less than 0.2: the two thresholds differ.
• AI should only assist. Amazon Connect says AI is not 100% accurate, and per its language table Vietnamese has no automated performance evaluation yet.
• Test usefulness with data: compare scores with CSAT, FCR or conversion rate, and do not infer causation from correlation.

What call scoring is for

Call scoring means reviewing a sample of calls against one form to learn what staff do well, what they miss, and which processes cause repeat errors. A coaching form needs detailed behaviors; a compliance form needs few questions and a clear line between right and wrong.

Coaching, compliance control and staff evaluation are 3 purposes that often get mixed. Click Star advises against using scores alone to decide pay while the form is uncalibrated.

How to sample calls so the results can be trusted

Score a random sample for a fair picture, plus calls chosen on purpose (complaints, lost deals) for lessons. Click Star's reference framework, not from an outside source: 5 calls per person per month, 3 random and 2 chosen.

By the normal approximation for a proportion (Click Star's own calculation), 30 random calls at an 80% pass rate give a margin of about 14 percentage points at 95% confidence; 100 calls give about 8. A few calls a month help choose what to coach but cannot rank people.

Designing the scorecard: criteria groups, weights and critical fails

Each criterion has a question, answering instructions and a weight. Per Amazon Connect, a weight shows how much a question or section affects the total, and weights always sum to 100%.

The sample below is Click Star's reference framework; weights are examples, not an industry standard. It has 12 criteria in 5 groups and 3 critical fails:

• Group 1, Opening (10%): greets and gives company and own name in the first sentence; asks why the customer called or whether it is a good time.
• Group 2, Needs discovery (30%): asks at least 1 open question about the goal; asks about 1 deciding condition such as budget or timing; restates the need before proposing.
• Group 3, Advice and concerns (30%): ties benefits to the need just heard; acknowledges an objection before answering; states price or conditions clearly when asked.
• Group 4, Next step (20%): states the next step with a date or time; repeats the agreement for the customer to confirm.
• Group 5, Compliance and records (10%): follows the recording-notice procedure; logs key information in the CRM after the call.

Critical fails sit outside the weights: promising or stating untruths about price, results or conditions; asking for or exposing sensitive data such as an OTP or card number; insulting or threatening the customer.

Amazon Connect has an "automatic fail" option: choosing that answer drops the section or form score to 0. Click Star prefers to record the normal score and flag the critical fail separately for a manager, so the rest of the call's information is kept.

Writing criteria as observable behavior

A good criterion has 1 behavior and can be answered by ear or from the transcript. Google Cloud advises instructions that "leave no room for interpretation" with examples for each answer. An example from Click Star's framework, the criterion "The agent states the next step with a date or time":

• Pass when: the agent says something like "I will send the quote on Zalo before 5 p.m. today and call you back tomorrow morning".
• Fail when: the agent only says "Contact us if you need anything".
• Not applicable when: the customer declines at the start of the call or dialed the wrong number.

Scale and calculation

For most criteria, Click Star advises 3 answers: pass, fail, not applicable. Amazon Connect advises few, short options, since a long list from "Great" to "Horrible" lowers AI accuracy.

A call's score is the weight of passed criteria divided by the weight of applicable criteria. Amazon Connect redistributes weights proportionally when 1 question becomes not applicable, for example from 60% to 75%.

An example with Click Star's framework (invented numbers): group 1 earns 10, group 2 earns 20, group 3 earns 20, group 4 earns 10, and group 5, with 1 pass and 1 not applicable, earns 5 of 5. The total is 65 of 95 applicable points, or 68.4%.

Click Star found no sourced industry pass-mark. Amazon Connect lets you set your own thresholds, so set them after calibration. Do not edit the form mid-period and keep comparing scores.

Calibration between reviewers

Calibration means several reviewers score the same call independently, then compare and agree on how to read the criteria. Google Cloud advises computing Cohen's kappa.

Kappa measures agreement after removing chance agreement. Per McHugh (Biochemia Medica, 2012), values run from -1 to 1 and 0 is chance level. It comes from Jacob Cohen's 1960 paper; Click Star could not open it and relies on McHugh's description.

An illustration (invented numbers): 2 reviewers score 20 calls on 1 criterion. Both pass 14, both fail 3, and disagree on 3 (A passes and B fails on 2, the reverse on 1). Raw agreement is 85% but kappa is only about 0.57, because most calls would pass for both anyway.

McHugh (writing for health care) treats kappa below 0.60 as inadequate. Google Cloud targets no less than 0.2 for annotators. Click Star found no threshold specific to sales calls, so it advises computing kappa per criterion and rewording the low ones.

Feedback for staff and coaching rhythm

Feedback must address specific behavior, not the person. Kluger and DeNisi (Psychological Bulletin, 1996) pooled 607 effect sizes: feedback raised performance on average (d = 0.41) but more than 1/3 of interventions lowered it. Click Star read only the abstract (full text paywalled).

Click Star's reference coaching framework:

• Pick 1 to 2 behaviors per session, not all 12.
• Let staff score their own call first.
• Name what went well, then 1 behavior to fix with a replacement phrase, then schedule a recheck.

Reference rhythm: 1-on-1 coaching every 2 weeks, calibration monthly, scorecard review quarterly. Click Star's proposal; no source states a standard.

Using AI to assist scoring: limits and the final human check

AI can pre-fill answers from the transcript, but a person must still review them. Per Amazon Connect, AI returns an answer with a reason and transcript evidence. The documentation says AI evaluations are not 100% accurate, and advises reviewing a sample before rewards or coaching and keeping manual scoring to catch drift.

Limits to know:

• Speech recognition: Amazon Connect says transcript quality affects scoring, and multilingual calls or overlapping speakers are not transcribed accurately.
• Use it only for criteria answerable from words: Amazon Connect and Google Cloud say AI does not judge tone of voice.
• Vietnamese: in Amazon Connect's language table (read October 21, 2026), vi-VN has post-call analytics but the automated evaluation column is empty.
• Public figures: per the Whisper paper (2022, appendix D.2.4), Vietnamese word error rate on FLEURS is 60.0% (tiny), 40.5% (base), 21.2% (small), 12.7% (medium), 10.7% (large). FLEURS is read speech, not noisy phone audio.
• Model version: Amazon Connect says a newer version can behave differently; pin the version when comparing scores over time.

Click Star found no public figures for Vietnamese sales calls, nor research on bias by regional accent. Compare AI and human scores by accent group, and have the final reviewer relisten to every critical fail the AI raises.

Is the scorecard useful? How to measure

A scorecard is useful when reviewers score alike and scores track what the business cares about. The first test is kappa; the second is whether total and per-criterion scores relate to CSAT, FCR or conversion on the same calls.

Be careful about causation: strong agents score high and close well, and CSAT surveys reflect only customers who respond. Compare within one call type, look per criterion, and drop or reword criteria unrelated to outcomes.

Call recording and personal data

Scoring needs recorded calls, and that touches personal data law. Per a copy on ketoananpha.vn (read October 21, 2026), Law No. 91/2025/QH15 takes effect on January 1, 2026. Click Star found no rule specific to recording sales calls in the sources it read.

Click Star already has an article on the personal data protection law. This is not legal advice: check the original text. Operationally, Click Star advises notifying the customer at the start of the call and masking card numbers and OTPs.

What this means for Vietnamese sales teams

Click Star's view. The greatest value of a scorecard is making the team agree on what a good call is, more than the final number. A small team should start with a short, manual form and consider AI later; in Vietnamese, phone numbers and amounts need a human listen.

Disclosure: Click Star offers Speech Insight AI (AI call analysis). Per the published service page, it scores against the business's own scorecard, people decide the criteria and pass mark, and accuracy is measured on the customer's own calls during a trial.

30-day plan

• Week 1: fix the purpose, call type and 8 to 12 criteria; check current recording.
• Week 2: 2 reviewers score 20 calls independently, compute kappa per criterion, reword the ones that diverge.
• Week 3: second calibration, freeze version 1, score a real sample, first coaching.
• Week 4: compare scores with CSAT, FCR or conversion rate; if you trial AI, compare it with people on the same 20 calls.

What is not known

Click Star found no sourced industry pass-mark. Kappa thresholds conflict across sources, and none is specific to sales calls.

There are no public accuracy figures for speech recognition or AI scoring on Vietnamese sales calls, only FLEURS figures on read speech. There is no Vietnamese evidence that a scorecard raises conversion.

Click Star could not open Cohen's 1960 paper or the original CSAT and FCR definitions.

Common mistakes

1. Writing vague criteria like "good attitude", so each reviewer reads them differently.
2. Setting weights by feel and treating them as an industry standard.
3. Having no critical fails, so serious errors get diluted in the average.
4. Skipping calibration, or looking only at raw agreement.
5. Ranking staff from a handful of calls.
6. Trusting AI scores without comparing them with people, or switching model versions mid-period.
7. Using scores for pay before the scorecard is validated, or scoring without giving feedback.