Every other article about AI detectors makes one of two claims: they can tell when you used a language model, or they can't. Both arguments quietly rest on the same handful of technical terms - perplexity, burstiness, token probability, watermarking - and almost none of them define those terms in plain English. That makes it nearly impossible to judge which detector claims deserve belief.

This glossary defines the language of AI text detection the way the researchers who build these systems use it. Every definition is tied to a primary source: a peer-reviewed paper, official documentation, or a published standard, linked at the point of use. If you write with AI tools, teach students who do, or evaluate detectors for your organization, this page is built to be referenced, not just read once.

Language model fundamentals

Language model

A program trained to predict text. Google's machine learning documentation states the crucial limitation plainly: generative models "don't retrieve facts, but predict a likely sequence of words based on their training data." Every term below describes some property of that prediction process - and every detector tries to exploit one of them.

Token

The unit a model actually reads and writes. A token may be a whole word, part of a word, or a punctuation mark: "detection" might be one token to one model and two ("detec" + "tion") to another. At every step, a model produces a probability distribution over its entire vocabulary. That is why detectors do their work at the token level - it is the only layer where the machine's confidence is visible.

Temperature

A setting that controls how random a model's choices are. At low temperature, the model almost always picks its highest-probability next token, producing very predictable text. At high temperature, it takes chances, and writing becomes more varied - and more prone to nonsense. Temperature is where the statistical gap between machine and human text is created: text generated at low temperature is, by construction, more predictable than most human writing.

Log probability (logprobs)

How confident a model was in each token it chose, expressed on a logarithmic scale. A token with a log probability near zero was nearly certain; a strongly negative value means the model selected something unlikely. These values disappear in finished prose, but they are the raw evidence that many detectors and visualization tools analyze after the fact.

Perplexity

The classic measure of how "surprised" a language model is by a text. Google's Machine Learning Glossary defines perplexity for a prediction task as "approximately the number of guesses you need to offer in order for your list to contain the actual word the user is trying to type," and relates it to cross-entropy as P = 2-cross-entropy.

For detection, the intuition runs in reverse: text a model could have predicted easily - full of common phrasing and expected word sequences - has low perplexity. Human writing, with its digressions, rare constructions, and idiosyncratic word choices, tends to score higher. That gap is one of the signals detectors are built to exploit, but it cuts both ways. Formal, templated human writing can also be low-perplexity, which is part of why detectors flag careful editors and non-native English writers (see "False positive" below).

The signals detectors look for

Burstiness

The tendency of meaningful words to appear in clusters rather than evenly across a text. A human writing about, say, watershed management will use the word "watershed" several times in one paragraph, then not use it again for pages. Function words like "the" and "of" spread evenly by contrast. Human writing is highly bursty for content words; machine-generated text tends to smooth those clusters out, because the model distributes its vocabulary more uniformly.

Burstiness is frequently listed alongside perplexity when detection tools explain their methods, though implementations vary and exact formulas are rarely published. The practical takeaway holds regardless: rhythm and repetition patterns are part of what gets measured, which is why stripping a text's natural repetitions can make it look more machine-like, not less.

Token ranking (the GLTR method)

GLTR - "Statistical Detection and Visualization of Generated Text," built by researchers at Harvard and MIT and presented at ACL 2019 - is the clearest published illustration of token-level detection. GLTR color-codes every token in a passage by how highly the model ranked it in its vocabulary distribution. Text sampled from a model at typical settings is dominated by top-ranked, high-probability tokens; human text shows a much wider spread. Notably, GLTR was designed as a human-in-the-loop aid: an analyst reads the statistics, rather than the statistics deciding on their own. Its paper remains the best window into why simple statistical tests work as well as they do.

Stylometry

The long-established practice of fingerprinting authors by their statistical habits - sentence length distributions, favorite function words, punctuation patterns. AI detection is essentially stylometry with a machine as the author: detectors trained on large corpora learn the fingerprint of model output and compare new text against it. The connection has a practical consequence. Anything that flattens a writer's habits toward the average - translated text, heavily edited text, formulaic writing - moves their fingerprint toward the machine centroid, which is exactly the failure mode documented in the bias research below.

Watermarking

Instead of guessing from statistics, watermarking marks text at generation time. The best-known scheme, from Kirchenbauer et al. (ICML 2023), selects a randomized set of "green" tokens before each word is generated, then softly promotes green tokens during sampling. The bias is invisible to human readers, but a detector with no access to the model can run a statistical test - with interpretable p-values - and flag the text from a short span of tokens, with negligible impact on text quality, per the authors. Watermarking is the most reliable detection signal that exists. The catch is adoption: it works only when a model's operator enables it, so it can never cover every model - and Sadasivan et al. later showed watermarked text can be spoofed in the opposite direction, which creates its own risk (see "Spoofing").

Compression-based detection

Compressors exploit exactly the statistical regularity that differs between texts. Benedetto, Caglioti, and Loreto's "Language Trees and Zipping" (Physical Review Letters, 2002) showed that a compression-based measure of the "remoteness" between texts can discriminate languages and authors with high accuracy, using nothing exotic - just how predictable a string is to a general-purpose compressor. The same principle underlies a family of AI-detection approaches: if a text compresses about as well as typical model output, that is evidence it shares the model's statistics.

How detectors fail

False positive

Flagging human text as AI-generated. False positives are the failure mode that actually harms people: a student accused on the strength of a score, a writer's credibility put in question. The most cited evidence comes from Liang et al. (2023), who tested widely used GPT detectors with writing samples from native and non-native English writers and found the detectors "consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are accurately identified." The same study showed simple prompting strategies could effectively bypass those detectors - the bias punished constrained writers while doing little to stop determined evaders. The authors' published recommendation is caution against using detectors in evaluative or educational settings.

False negative

Missing AI-generated text - scoring it as human. Every deployed detector trades false positives against false negatives: tune a detector to catch everything, and it flags more humans; make it cautious, and it misses more machines. Published accuracy figures describe one point on this trade-off, measured on the test set the developers chose, which is why cross-testing the same passage on several independent detectors tells you far more than any single vendor's headline number.

Paraphrase attack

Editing AI text to defeat detectors - by paraphrasing, translating, or recursively re-generating it. Sadasivan et al. ("Can AI-Generated Text be Reliably Detected?", published in Transactions on Machine Learning Research) stress-tested detectors with recursive paraphrasing and found it could "significantly reduce detection rates" while "only slightly degrades text quality in many cases." Their analysis goes deeper than an exploit demo: they connect the best achievable detector's performance (AUROC) to the statistical distance between human and AI text distributions, arguing that as models get better at mimicking human text, perfect detection becomes mathematically harder - not just practically harder. It is the strongest published argument against treating any single detector as ground truth.

Spoofing

The mirror-image attack: forging watermark signatures onto human text to make it look machine-generated. Sadasivan et al. demonstrate that an attacker can infer hidden AI text signatures without white-box access to the detection method, then use them to misclassify human writing as AI. The victim is the human writer, and the reputational risk lands on whoever operates the model whose pattern was forged. Spoofing is why watermarking, despite being the strongest signal available, cannot by itself settle disputes about authorship.

AUROC

The standard head-to-head metric for detectors: the probability that a detector ranks a randomly chosen AI text as more machine-like than a randomly chosen human text. A perfect detector scores 1.0; a coin flip scores 0.5. AUROC is threshold-free, which makes it the honest way to compare systems - "accuracy" numbers hide the cutoff the vendor picked. In the AI-detection literature, AUROC is also where theory bites: the best possible detector's AUROC is bounded by how statistically close the two text distributions are, and modern models keep closing that gap.

What this means when you use detectors

Read the terms above together, and a fairly clear picture of honest detector use emerges:

  • A detector score is evidence, not a verdict. Watermark spoofing, recursive paraphrasing, and the non-native-writer bias study all point the same way: scores are probabilistic signals that deserve human context before any decision rides on them.
  • Short texts are unreliable almost everywhere. Token-level statistics need tokens. Most detectors degrade sharply below a few hundred words, which is why our own AI detector enforces a minimum input length instead of returning a meaningless number.
  • The safest workflow is review, not screening. Google's own guidance for publishers asks for manual factchecking and review of AI-assisted content rather than trust in any automated proxy - a standard that applies to detection scores as well (see our guide to Google's October 2026 quality guidance).
  • Improving a draft is authorship; laundering provenance is not. Editing machine text until it is accurate, specific, and genuinely yours - the workflow our AI humanizer supports - is not the same act as pushing unreviewed output through to escape a score. The second is what detectors were built to catch, and, as the paraphrase research shows, increasingly fail to catch anyway.

Frequently asked questions

What does low perplexity mean?

Low perplexity means a language model found the text highly predictable - it could have produced similar word sequences with many choices to spare. Predictability correlates with common phrasing, which is why very low perplexity is associated with machine-generated text; it also correlates with formulaic human writing. Perplexity alone proves nothing. It is one signal among several, and its meaning depends entirely on the rest of the evidence.

Can AI-generated text be reliably detected?

Not perfectly. Published research shows that recursive paraphrasing significantly reduces detection rates, watermarks can be spoofed onto human text, and detectors systematically mis-flag non-native English writers. Detection produces probabilistic evidence, not certainty. The mathematically honest framing comes from work published in Transactions on Machine Learning Research showing the best achievable detector is bounded by how statistically similar human and machine text have become - a gap modern models keep shrinking.

Why do AI detectors flag non-native English writers?

Because constrained, careful phrasing - limited vocabulary, simple sentence structures, textbook-like organization - is statistically closer to model output than the varied writing of native speakers. Liang et al. (2023) documented this across widely used detectors: non-native writing samples were consistently classified as AI-generated while native samples were identified correctly. It remains the most serious documented fairness problem in the field, and it is why detector scores should never be the sole basis for an accusation.

What is burstiness in AI writing?

Burstiness describes how human writers cluster their content words - repeating an important term several times within a passage, then dropping it - versus the more even word distribution typical of model output. It is one of the statistical patterns detection tools use, alongside perplexity-style scores. The irony worth knowing: a common editing tactic for making text feel less repetitive can lower burstiness and push a text toward looking machine-made.

Is watermarking the future of AI detection?

It is the strongest signal available. Statistical tests over green-token patterns can flag watermarked text with interpretable confidence and negligible effect on readability. But watermarking depends on model vendors adopting it, it covers only their own output, and the signatures can be spoofed onto human text. Expect watermarking to become one important provenance channel alongside human review - not a single source of truth.

Sources