Do AI detectors flag human writing?

How often real writing was flagged

Detector: MELD, at the line its makers set so that 1% of human writing is flagged. Sources: report.txt, report_v2.txt, report_v3.txt, public_report.txt and humanizers_report.txt. On random web text it was right about that. On polished, formal or old-fashioned writing it was far off, and books written before 1940 can't have come from a chatbot.

Why polished writing gets flagged

Detectors learn what AI text looks like, and AI text is tidy, even and confident. So is a Wikipedia opening, and so is a LinkedIn post written to impress. The more a genre rewards a smooth, neutral voice, the more of its human writing crosses the line.

The fix is to set the line per genre. When we set it from 1,044 real LinkedIn posts, the share of real posts flagged dropped from 9.9% to 0.7%, while still catching about half of the AI drafts at that line and about 90% at a looser one. One caveat: the posts we checked that on came from the same 12 authors, so it's the line agreeing with itself, not a fresh test.

Non-native English writers are flagged more

A Stanford study ran seven detectors on 91 TOEFL essays written by non-native English speakers and 88 essays by US eighth-graders. The detectors were near-perfect on the US essays but flagged the TOEFL essays as AI 61.22% of the time on average. All seven agreed on 18 of the 91, and 89 of the 91 were flagged by at least one (Liang et al., 2023).

The authors traced it to perplexity: a writer with a smaller working vocabulary makes more predictable word choices, which is exactly what perplexity-based detectors read as machine-like. When ChatGPT was used to enrich the essays' word choice, the average false-positive rate fell to 11.77%. The essays hadn't become more human; they'd become less predictable.

We haven't measured this with MELD, which is a trained classifier rather than a pure perplexity score, so we can't say how much of the gap it shares. The general lesson holds for any detector: plain, careful, predictable writing looks more like AI.

Even detector makers say it

OpenAI released its own AI-text classifier in January 2023 and withdrew it on July 20, 2023, citing its low rate of accuracy. A detector's score is a probability from a model, not a record of how the text was made.

If you've been flagged and didn't use AI

1. 01Ask which detector was used and at what threshold. A score with no false-positive rate behind it isn't evidence.
2. 02Show your process: version history in Google Docs or Word, drafts, notes, sources you opened, and earlier writing in the same voice.
3. 03Point to the genre problem. Formal, encyclopedic or corporate writing is flagged far more often than casual writing, as the table above shows.
4. 04If English isn't your first language, say so and point to the research on non-native writers.
5. 05Offer to talk through the piece or write a short section in person. Knowing your own argument is the strongest evidence there is.