86.23%. It looks precise. It feels scientific. It feels safe.
But according to Gabriel Fairman, it may be one of the biggest illusions in the localization industry.
“This number is, in itself, a hallucination.” — Gabriel Fairman
Why We Fell in Love With Scores
For decades, translation quality has been reduced to numbers.
Percentages, grades, scores, thresholds.
They made language feel manageable.
- Numbers feel objective
- Numbers fit neatly into spreadsheets
- Numbers help justify budgets and decisions
“If I tell management our translation quality went up 2% this quarter,” Gabriel explains, “everyone nods.”
The messiness of human language suddenly looks clean.
Computers Couldn’t Understand Meaning
The obsession with numbers didn’t come from nowhere. It came from a real technical limitation.
“Until recently, computers couldn’t understand words. They could only count them.”
Because machines couldn’t process meaning, the industry relied on math.
- Rule-based MT focused on grammar rules
- Statistical MT focused on probabilities
- Quality meant matching patterns, not intent
Numbers weren’t ideal. They were necessary.
BLEU, Fuzzy Matches, and a Dangerous Illusion
Metrics like BLEU promised clarity. They compared word chunks. They counted overlap.
But they missed something critical. Meaning.
“BLEU measures surface similarity, not semantic equivalence.” — Gabriel Fairman
A classic example:
- Reference: The system is unavailable
- Translation: The system is available
BLEU sees an 80% match.
Humans see a catastrophic failure.
When High Scores Hide High Risk
This is where the danger starts.
A translation can look “good” on paper and be deeply wrong in practice.
- Legal meaning can flip
- Brand voice can drift
- Instructions can mislead users
“If you’re managing quality based on 86.23%, you’re not managing quality. You’re managing a mirage.” — Gabriel Fairman
Scores give comfort. But comfort is not the same as control.
It’s 2026. The Limitation Is Gone.
Today’s AI is different. Large Language Models don’t just count words.
They compute semantics.
“Now we can interact with meaning, not just math.” — Gabriel Fairman
This changes everything.
For the first time, we can analyze what is wrong, not just how similar something looks.
Gabriel proposes a shift inspired by software engineering.
Instead of grades, think diagnoses.
Instead of scores, think symptoms.
“We need to move from asking ‘Is this right or wrong?’ to ‘What does this smell like?’” — Gabriel Fairman
This is where the idea of Translation Smells comes in.
What Are Translation Smells?
Translation smells are qualitative signals.
They don’t say “pass” or “fail.”
They say “pay attention.”
Examples include:
- Ambiguity smells
- Terminology inconsistencies
- Tone shifts
- Semantic drift
- Brand voice misalignment
“Smells are whiffs of things that could be off with language.” — Gabriel Fairman
They are not percentages. They are insights.
A score tells you something is wrong.
A smell tells you what to fix.
- “Tone is defensive instead of apologetic”
- “This sentence can be interpreted in two ways”
- “Terminology conflicts with the glossary”
“It doesn’t give you a number. It gives you a diagnosis.” — Gabriel Fairman
This turns the reviewer from a grader into a doctor.
The Trojan Horse of Confidence Scores
Ironically, even Bureau Works uses a confidence score. Not because it’s ideal.
But because the market demands it.
“People want something neat. Something clean. Something they can decide with.” — Gabriel Fairman
Procurement needs numbers. Managers need reports.
Spreadsheets still run the world.
Why the Score Is Not the Point
The score is only the map. The smells are the territory.
“The value isn’t that it’s 85% confident. The value is knowing why it isn’t 100%.” — Gabriel Fairman
When uncertainty is explained in words, teams can act.
- Rewrite for clarity
- Escalate legal risk
- Block publication
- Send focused human review
This is impossible with a single number.
The Danger Zone: Letting AI Fix Everything
If AI can detect problems, why not let it fix them automatically?
Gabriel calls this the danger zone.
“Authorship is choice.” — Gabriel Fairman
If AI writes, reviews, and corrects everything:
- Who owns the voice?
- Who owns the intent?
- Who is accountable?
Without human choice, language becomes noise.
Human judgement is the final mile. Translation smells don’t replace humans.
They support them.
- AI flags issues
- Humans decide what matters
- Humans choose when to ignore
“The decision to ignore or apply is where the human value lies.” — Gabriel Fairman
That decision is authorship.
A translation can be accurate and still fail. It can be correct and ineffective.
Ultimately, it’s about the effectiveness of communication. If users don’t click, trust, or engage, something is wrong.
Language exists to do things. Not just to be correct.
A Transition Point for the Industry
The industry stands at a crossroads. One path leads to better scores.
The other leads to better diagnostics.
“The future isn’t about better numbers. It’s about better understanding.” — Gabriel Fairman
Words analyzing words are finally more powerful than decimals.
Treat Language Like a Living System
When we stop grading language like an exam, something changes.
We make room for judgment.
We make room for context.
We make room for humans.
“If we’re not keeping these decisions for ourselves, we’re losing control over the production of language.” — Gabriel Fairman
And once meaning is lost, it’s very hard to get it back.
In 2026, words are smarter than numbers.
The question is whether we’re ready to let them be.
Ready to Move Beyond Scores?
At Bureau Works, we built Smells for exactly this moment.
Smells don’t grade your translations.
They diagnose them.
Instead of reducing meaning to a percentage, Smells surface what actually matters:
- Where meaning may have shifted
- Where tone doesn’t match intent
- Where terminology, context, or voice may be off
This gives teams clarity without taking control away from humans.
“The score is the map. The smells are the territory.” — Gabriel Fairman
If you’re ready to stop managing quality through numbers and start engaging directly with meaning, Smells can help you get there.
👉 Explore Smells and see how teams can make better language decisions without losing authorship.
Subscribe to Bureau Works and experience translation quality that’s diagnosed, not guessed.
Unlock the power of glocalization with our Translation Management System.
Unlock the power of
with our Translation Management System.
.avif)
