Most QA scorecards are written by experienced reviewers for experienced reviewers. Lines like “good call control” or “professional tone” make sense to the person who wrote them. Hand the same scorecard to a new reviewer, or to software, and you get different answers.
A scorecard that can be checked automatically is simply one where every line has a single, observable answer. The good news is that rewriting for that also makes your human reviews more consistent.
The test for every line
Read each line and ask three questions:
- Can you point at it? Could a reviewer highlight the exact part of the conversation that proves it happened, or show where it should have happened and didn’t?
- Would two people agree? Given the same conversation, would two reviewers reach the same answer without discussing it?
- Does it always apply? If not, is it clear when it applies?
If a line fails any of these, rewrite it.
Five rewrites that fix most scorecards
1. Replace qualities with behaviours
“Showed empathy” is a quality. You can’t point at it. Turn it into something you can see:
- Before: Showed empathy
- After: The agent acknowledges the customer’s problem before offering a solution.
You lose some nuance. Keep the nuance for coaching conversations, and keep the scorecard for things you can check.
2. One behaviour per line
“Greeted the customer, confirmed their identity and stated the reason for the call” is three checks. If one is missed, the whole line fails and nobody knows which part. Split it.
3. Say when it applies
Many checks only make sense in some conversations. A refund rule means nothing on a call where no refund came up. Write the condition into the line:
- Before: Refund approved by supervisor
- After: When a refund over £50 is offered, the agent says it has been approved by a supervisor.
Without a condition, the check either fails unfairly or gets marked “not applicable” by hand, which is exactly the judgement you were trying to remove.
4. Name the thing, not the feeling
“Explained the fees clearly” depends on what clear means. Name what must be said:
- Before: Explained the fees clearly
- After: The agent states the monthly price and any setup or cancellation fee before the customer agrees.
5. Separate requirements from warnings
Some lines are things that must happen. Others are things you want to know about when they happen, such as a customer mentioning they are in financial difficulty or asking for a manager. Treat them differently:
- Requirements fail when they are missing.
- Warnings don’t fail the agent. They flag the conversation for a person to look at.
Mixing the two makes pass rates meaningless, because an agent can “fail” for something the customer said.
Mark how serious each line is
Not every miss is equal. A missed greeting and a missed identity check shouldn’t count the same. Give each line a severity, such as info, warning or critical, and let critical failures stand out in reports. That also tells reviewers which failures to look at first.
Test the wording before you trust it
Once the lines are rewritten, try them on real conversations:
- Pick 20 conversations your team has already reviewed.
- Check them against the new wording, by hand or automatically.
- Look at every disagreement with the original review.
Disagreements usually point at wording that is still vague. Fix the line, not the result. When a line agrees with your reviewers on nearly every conversation, it is ready.
A before and after
| Before | After |
|---|---|
| Professional opening | The agent says their name and the company name in their first message. |
| Verified the customer | Before discussing the account, the agent confirms at least two security details. |
| Handled the complaint well | When the customer complains, the agent says the complaint has been logged and when they will hear back. |
| No false promises | The agent does not promise a refund, credit or outcome they haven’t confirmed. |
| Good close | Before ending, the agent summarises what happens next and when. |
Where Assay fits
In Assay, each line of your scorecard becomes a rule written in plain English, with an optional “only when” condition and a severity. If a line is hard to check, Assay suggests clearer wording. Every conversation is then checked against each rule, and every result points at the line that decided it. For ready-made examples, see 20 customer service QA checks you can copy.