AI vs Human Interpreter Accuracy: What Studies Measure

How accurate is AI interpretation compared with a human interpreter? Published comparisons do not give one transferable percentage. Their results depend on the language pair, whether the task is live speech or recorded material, who interprets, and whether the evaluator counts wrong words, unclear delivery or errors that could change a decision.

Why One Accuracy Percentage Misleads

An industry article quotes 82–88% for AI and 98–99% for human interpreters while attributing the pair to a Stanford NLP study without linking it or defining “accuracy.” We could not verify that citation. Do not use those percentages as a benchmark without the original study, its scoring method and its sample.

Even a sourced number can answer the wrong question. A word-level error rate does not tell you whether a listener understood a warning. A share of errors with possible clinical consequences does not tell you what share of all utterances was wrong. The studies below measure different outcomes, so their percentages cannot be pooled or ranked as if they were one test.

What Four Published Comparisons Actually Found

These papers cover different systems and settings. Three compare outputs directly; the figurative-language study compares two interview methods with the same six patients, so it does not isolate the translation engine on identical utterances.

Clinical errors: check the denominator

Feinberg and colleagues began with 14 real physician speech segments. Each was translated into six languages by an app in voice mode and by qualified medical interpreters, yielding 84 translations in each condition. The table reports 28 clinically wrong AI translations versus four interpreter translations. The original draft called this “14 segments” without explaining the 84-output denominator, making the percentages look broader or narrower than the actual comparison. Performance also differed across languages: the Spanish row recorded two clinically wrong app outputs and zero interpreter outputs out of 14 each. The result should not be applied to every AI product or language.

Meaning and delivery can diverge

In Singh and colleagues’ LingualAI study, bilingual clinicians rated anonymized audio from scripted English–Spanish ENT scenarios. With a prespecified 0.30-point non-inferiority margin on five-point scales, the in-house system met the threshold for terminology (difference 0.07) and adequacy of meaning (0.13), but missed it for clarity (0.50). Fluency, prosody and pacing also missed the margin. Overall quality and clinical confidence favored interpreters. The study used nine raters with uneven participation and reported modest inter-rater agreement (Krippendorff’s alpha 0.31). It was a simulation, not proof of performance in unscripted care.

Conference speech and figurative language

Fantinuoli and Prandi assessed five conference speeches using a framework focused on intelligibility and informativeness. Human interpreters scored 84.84% versus 73.49% for machine intelligibility; machine output scored 74.63% versus 72.82% for human informativeness. These are scores under this study’s rubric, not a general “accuracy percentage.” The authors warn against generalizing from the small corpus.

Tougas and colleagues studied figurative language in psychiatric interviews. Six patients had one interview mediated by a human interpreter and another in Spanish with AI-assisted interpretation. Both methods frequently rendered idioms and other figurative expressions inaccurately. Because the source utterances differed between interviews, the study is useful for finding a difficult failure mode, not for a clean product-versus-interpreter accuracy score.

Human Interpreter Quality Also Needs a Definition

Flores and colleagues analyzed 57 pediatric emergency encounters and recorded 1,884 interpretation errors across professional, ad hoc and no-interpreter conditions. Eighteen percent of those errors had potential clinical consequences. The corresponding shares were 12% for professional interpreters, 22% for ad hoc interpreters and 20% without an interpreter. These figures are not percentages of all speech translated correctly.

Within the professional group, the share of errors with potential clinical consequences was 2% for interpreters with at least 100 hours of training and 12% for those with fewer hours. This association does not mean either group had a 98% or 88% overall accuracy rate. It does mean a buyer should ask about training and how consequential errors are identified, instead of treating “human” as one uniform baseline.

A Practical Evaluation Checklist

1. Match the setting. Test your actual languages, dialects, audio quality, turn taking and technical vocabulary. A scripted ENT dialogue does not establish conference or emergency performance.
2. Define the denominator. Count source utterances, translated outputs, detected errors and consequential errors separately.
3. Score more than words. Have qualified bilingual reviewers assess terminology, preserved meaning, intelligibility, omissions and consequences, preferably without knowing the source of each output.
4. Record correction paths. Decide when a participant can challenge a caption, pause, repeat a statement or bring in a qualified interpreter.
5. Report uncertainty. State the sample size and language-specific results; do not turn a small experiment into an all-language guarantee.

For AI tool-to-tool metrics, see our real-time translation accuracy guide. For a written record rather than live cross-language speech, see the transcription accuracy comparison. Those pages answer different questions from staffing a human interpreter.

Where MirrorCaption Fits

MirrorCaption offers live transcription and translation so people can inspect the original and translated text side by side. Speak Translations can read translated speech aloud. These features can help in ordinary meetings and conversations, but this article’s studies did not evaluate MirrorCaption. The product is not a qualified human interpreter or a clinical interpretation service.

For clinical decisions and other high-stakes conversations, arrange an appropriately qualified interpreter under the relevant organization’s language-access process. HHS language-access guidance describes qualified interpreters and separate rules for machine-translated documents; the latter should not be read as approval of unsupervised AI speech interpretation. Check current policy and the particular communication before choosing a workflow.

Frequently Asked Questions

How accurate is AI interpretation compared with a human interpreter?

There is no one transferable percentage. Published results differ by system, language, task and scoring method. In one scripted English–Spanish study, AI met a meaning and terminology threshold but missed clarity and speech-delivery thresholds; a separate six-language clinical-speech test found more clinically wrong outputs from its app than from qualified interpreters.

Where does the 82–88% AI versus 98–99% human statistic come from?

An industry article attributes that pair to a Stanford NLP study without linking the study or defining accuracy. We could not verify the source. Ask for the original paper and scoring method before citing the numbers.

Are trained human interpreters 99% accurate?

The Flores study does not report a general 99% accuracy rate. It found that 18% of 1,884 recorded interpretation errors had possible clinical consequences; among professional interpreters, the share of errors with possible consequences differed by training hours.

Can AI replace a qualified interpreter in a medical appointment?

These studies do not establish that it can. Use the care organization’s language-access process for consequential clinical communication. AI may assist where appropriate, with a clear route to qualified human interpretation and review.

How should a team compare AI and human interpretation?

Test the languages and real tasks you need. Report the number of source utterances and outputs, score meaning and intelligibility separately, identify consequential errors, and document how a participant gets a correction or a qualified interpreter.