AI Detector Accuracy: What Happens When the Writer Is Not a Native English Speaker
Anyone who has pasted their own writing into an AI detector and watched it come back flagged has asked the same question about AI detector accuracy, usually in a slightly panicked way: how can this be wrong about something I know I wrote?
There is a published answer, it is specific, and it is worse than most people assume. In a study of seven widely used detectors, essays written by non-native English speakers were misclassified as machine-generated more than half the time. Essays by American schoolchildren were classified almost perfectly.
The detectors were not measuring what they claimed to measure. What follows is what they were measuring instead, how that was proven, and what a second study found when it tested fourteen tools against the opposite failure.
The Claim These Tools Make
The commercial pitch is simple: paste text, receive a probability that a machine wrote it. Schools buy it to catch cheating. Editors buy it to screen freelancers. Clients increasingly run it over work they have paid for.
The pitch assumes the tool can separate two populations — human writing and machine writing — on properties intrinsic to each. The research tested that assumption directly, and it does not hold.
AI Detector Accuracy Collapses When the Writer Is Not American
Researchers at Stanford ran seven widely used GPT detectors over several sets of text whose authorship was known. The paper, GPT detectors are biased against non-native English writers, was published in the Cell Press journal Patterns.
The result that matters:
| Text set, all human-written | Average false positive rate |
|---|---|
| TOEFL essays (non-native English writers), n=91 | 61.22% |
| US 8th-grade essays, n=88 | near-perfect accuracy |
More than six in ten essays by people writing English as a second language were labelled machine-generated. Essays by American thirteen-year-olds were, in the paper’s words, classified with “near-perfect accuracy.”
Two further figures make the scale concrete. 18 of the 91 TOEFL essays — 19.78% — were flagged as AI-authored by all seven detectors simultaneously. And 89 of 91, or 97.80%, were flagged by at least one.
Unanimity among seven independent tools is the thing worth sitting with. If you are a non-native English writer and you run your own work through several detectors to be safe, the agreement between them is not evidence. They share the same blind spot.
The Experiment That Explains the Mechanism
This is the part that turns a worrying result into an explanation, and it is why the paper matters more than the headline.
The researchers took the same TOEFL essays and asked ChatGPT to enhance their word choices — richer vocabulary, more varied phrasing, nothing else. Same authors, same ideas, same arguments.
| TOEFL essays | Average false positive rate | Unanimously flagged |
|---|---|---|
| As originally written | 61.22% | 18 of 91 (19.78%) |
| After vocabulary enrichment | 11.77% | 1 of 91 (1.10%) |
A drop of 49.45 percentage points, from a text edit that changed no claim and no argument.
Then they ran it backwards. They simplified the word choices in the US 8th-grade essays — the set the detectors had handled almost perfectly. Misclassification rose from an average of 5.19% to 56.65%.
Make an American child’s essay simpler and the detectors start calling it artificial. Make a non-native speaker’s essay richer and they stop. The authorship never changed in either direction.
What the Detectors Are Actually Measuring
The mechanism is now visible, and it is not subtle.
These tools lean heavily on text perplexity — roughly, how surprising each word is given the words before it. Language models produce statistically likely words, so low surprise is treated as a machine signature.
But low surprise is also what a limited vocabulary produces. Someone writing in their fourth year of English reaches for the common word because it is the word they have. A language model reaches for the common word because it is the probable one. The statistic cannot tell those two situations apart, because at the level the statistic operates, they are the same situation.
So the detector is not measuring whether a machine wrote the text. It is measuring whether the vocabulary is plain. Those correlate in one population and come apart badly in another, and the population where they come apart is everyone who learned English as an adult.
The Second Failure: They Also Miss the AI
A separate team tested the opposite direction, and the result is the other half of the problem.
Weber-Wulff and colleagues, in the International Journal for Educational Integrity, evaluated fourteen detection tools— twelve publicly available ones plus the commercial systems Turnitin and PlagiarismCheck.
Their conclusion, stated plainly in the paper: the available detection tools are “neither accurate nor reliable.”
And the direction of their error is the interesting part. The tools have, in the authors’ words, a “main bias towards classifying the output as human-written rather than detecting AI-generated text.” The study also found that content obfuscation techniques significantly worsen performance — paraphrasing machine text makes it harder, not easier, to catch.
Why Both Failures Happen at Once
Read those two studies together and the picture is worse than either alone.
One finds the detectors falsely accusing human writers, concentrated among non-native speakers. The other finds them letting actual machine text through, and failing further when that text is lightly paraphrased.
That sounds contradictory. It is not. Both follow from the same mechanism. A tool keyed to plain, predictable language will flag humans who write plainly and miss machines told to write unpredictably. The error is not random noise that more data would reduce. It is the measure being wrong about what it measures.
Which produces the outcome nobody designing these tools intended: the person most likely to be falsely accused is a non-native speaker writing honestly, and the person least likely to be caught is someone deliberately evading detection. Exactly inverted.
Institutions have begun saying so in public. Brandeis University’s AI steering council states that AI detection tools “are unreliable and can be biased against non-native speakers and students who are underrepresented in higher education,” and that they “have high false positive rates and are easy to evade.”
What Google Actually Does About AI Content
Worth separating two fears that get collapsed into one, because the second is where most people’s money actually is.
Google’s published guidance on generative AI content makes no statement at all about detecting AI authorship. It does not describe a classifier, a penalty for machine-written text, or a detection system.
What it describes instead is a quality and scale test. Using AI to generate many pages “without adding value for users may violate” the spam policy on scaled content abuse. It warns that generative models “don’t retrieve facts,” which makes it “critical to manually factcheck and review all AI-generated content.”
The distinction is the whole thing. Google is not asking who typed it. It is asking whether the page is accurate, useful, and not one of ten thousand near-identical pages. That is consistent with what the rest of its documentation on AI content says, and it means the detector panic and the ranking question are two separate problems that happen to share a vocabulary.
Platform monetization is a third separate problem again — some platforms require disclosure of synthetic content and penalise its absence, which is a rule about labelling, not about detection.
What This Means If You Write in a Second Language
Four practical consequences, stated without reassurance that the research does not support.
You are in the high-risk group, and it has nothing to do with what you did. A 61.22% false positive rate is not a rare malfunction. For non-native writing it is the normal behaviour of the tool.
Running your own work through several detectors proves nothing. 97.80% of the TOEFL essays were flagged by at least one of seven. Checking more tools raises your chance of a false alarm; it does not raise your confidence.
Vocabulary range is the variable. Richer, more varied word choice cut the false positive rate by nearly fifty points in the study. That is not a trick to game a detector — it is a description of what the detector is sensitive to, and a reason to build vocabulary for its own sake.
The underlying disadvantage is real and separate. Writing in a second language carries genuine costs and some genuine advantages — research on decision-making in a foreign language has documented both. Detector bias is one more cost, and the only one here that is an artefact of a bad instrument rather than a feature of the work.
If Your Work Gets Flagged
There is no appeal process at a detector, so the argument has to be made to the person holding the result.
What the research supports you saying: the tool has a published false positive rate above 60% for non-native English writing; seven detectors agreed wrongly on nearly a fifth of a known-human sample; and a peer-reviewed evaluation of fourteen tools concluded they are neither accurate nor reliable. All of that is citable, and the two studies are linked at the end of this article.
What helps more than argument is evidence of process — drafts, version history, notes, sources. A document’s revision history is the thing a detector score cannot contradict.
What does not help is running the text through another detector and hoping. You already know how that goes.
What the Research Does Not Say
Four honest limits.
The study is specific. Seven detectors, a particular set of TOEFL essays, a particular moment in the tools’ development. Detector vendors iterate, and these figures are a snapshot, not a permanent property.
“Neither accurate nor reliable” is a conclusion about tools tested, not a law. A future detector could be better. Nothing in either paper rules that out; both rule out the ones available when the work was done.
None of this says AI-written text is fine to pass off as your own. The research is about whether a specific instrument can tell. Whether you should disclose is a separate question with its own answers, and so is who owns what a model produces — some platforms and some legal systems answer both with rules rather than preferences.
The vocabulary finding is not advice to write ornately. Clear writing beats decorated writing for a reader. The finding explains a detector’s behaviour; it is not a style guide.
Frequently Asked Questions
Is AI detector accuracy good enough to trust? Not according to the published evaluations. A study of fourteen tools in the International Journal for Educational Integrity concluded they are “neither accurate nor reliable,” and a Stanford study found an average 61.22% false positive rate on human essays by non-native English writers.
Why do AI detectors flag writing I wrote myself? Most rely on how statistically predictable the word choices are. Plain, common vocabulary looks predictable, and so does machine output. The tools cannot separate the two, so writing with a limited word range gets read as machine writing.
Are AI detectors biased against non-native English speakers? The data says yes. In the Stanford study, TOEFL essays by non-native writers were falsely flagged 61.22% of the time on average, while US 8th-grade essays were classified near-perfectly. Brandeis University’s AI steering council states plainly that the tools “can be biased against non-native speakers.”
Does checking several detectors make the result more reliable? No. Among 91 human-written TOEFL essays, 97.80% were flagged by at least one of seven detectors and 19.78% were flagged by all seven at once. The tools share the same mechanism, so they share the same error.
Does Google penalise AI-generated content? Google’s own guidance makes no statement about detecting AI authorship. It warns that generating many pages “without adding value for users may violate” its scaled content abuse policy, and that AI output must be fact-checked. The test is quality and scale, not who typed it.
Can AI-generated text evade these detectors? The fourteen-tool evaluation found content obfuscation techniques significantly worsen detection performance, and that the tools are already biased toward calling text human-written. Deliberate evasion is the case they handle worst.
What should I do if my work is wrongly flagged? Take it to the person who ran the test, not to another detector. Cite the published false positive rates, and show process evidence — drafts, revision history, sources. A document’s version history is harder to dismiss than a probability score.
Does richer vocabulary really change the result? In the study it did, dramatically. Enhancing word choices in the same TOEFL essays cut the average false positive rate from 61.22% to 11.77%, and unanimous misclassification from 19.78% to 1.10%, without changing a single argument.
Conclusion
The honest summary of AI detector accuracy is that these tools do not do the job they are sold for. They fail in both directions at once — accusing humans who write plainly and missing machines told to write otherwise — because the property they measure is vocabulary range, not authorship.
The cost of that error is not evenly distributed. It falls on people writing English as a second language, who are already carrying a disadvantage in every room where this matters, and it falls on them in a way that looks like evidence to anyone who does not know what the number means.
Two peer-reviewed studies have now said this clearly, and institutions have started repeating it. The tools are still being sold, still being bought by schools and clients, and still producing a percentage that looks authoritative precisely because it has a decimal point.
If you write in a second language and a detector has flagged you, you are not being caught. You are being measured by an instrument that cannot tell the difference between a machine and someone using the words they have.
A Note from Lucas
I write this site in English and English is not my first language, so I went into this research with a personal stake and came out angrier than I expected. The number that stayed with me is not the 61% — it is the 19.78%, the share of real human essays that all seven detectors agreed were machine-written, because agreement is exactly what makes a false accusation feel like proof. What I had not understood until reading the paper is that these tools were never measuring authorship at all; they measure how plain your vocabulary is, which means they are closer to an English fluency test than a cheating detector. My direct opinion: any school or client making a decision on one of these scores is making it on a number that the published research says is unreliable, and they should be told so with the citations attached. That is why both studies are linked above rather than summarised.
About the Author
Lucas Palma is the founder and editor of Info Is The Key. He writes about online business, entrepreneurship, and the practical mechanics of building income independently, working from primary sources and official data rather than recycled advice.
Last Updated: October 2026 | Reading Time: 11 minutes
Editorial note: This article reports findings from two peer-reviewed studies and the published positions of Google and Brandeis University, all accessed in October 2026. Detection tools are updated frequently, and the figures describe the tools as tested at the time of each study rather than a permanent property of all detectors. Nothing here is advice on academic or professional conduct, and the research concerns whether these instruments work, not whether undisclosed AI use is acceptable in any given setting.
Sources
- Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. — GPT detectors are biased against non-native English writers (published in Patterns, Cell Press; full text via arXiv)
- Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1), 26
- Google Search Central — Google Search’s guidance on using generative AI content on your website
- Brandeis University, AI Steering Council — Limitations of AI Detection Tools
One Comment