How Much Are LLMs Being Used in Peer Review?

The study

The paper, “Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews,” proposes a method for detecting AI-generated text based on the adjectives and adverbs preferred by AI systems. When the authors applied it to machine-learning paper reviews, they found a substantial estimated share of LLM use.

The authors frame their method against three existing approaches to AI-text detection. Zero-shot methods are difficult to improve in a setting dominated by closed models. Training-based methods can overfit and are vulnerable to adversarial attacks and bias. Watermarking requires service providers to implement the feature, and adding a watermark may reduce output quality.

Method

The researchers collected human-written reviews. They also gave an AI system papers and reviewer instructions, asking it to produce reviews, thereby constructing human and AI corpora.

For each adjective, they calculated its frequency in human writing, P(token), and in AI writing, Q(token). If a corpus is mixed at a ratio of α, the probability that a given adjective appears in a sampled document is (1 − α)P + αQ. Using multiple tokens and documents, they estimated α through maximum likelihood estimation.

They first validated whether the estimate matched the known proportion in synthetic data, then applied the method to real-world data.

Findings

The estimated proportion of AI-generated content increased after ChatGPT was released. The study estimates that approximately 7–15% of machine-learning conference reviews involved LLM use. It found no such evidence in Nature reviews.

The estimated α value did not change substantially when reviewers used ChatGPT only for proofreading. The method also detected text as generated in a setting where a person prepared an outline and an LLM turned it into full sentences. This makes it difficult to state exactly how much a reviewer relied on generation, rather than merely whether generated phrasing was present.

LLM use was estimated to be higher for reviews submitted during the final three days before a deadline. Reviews containing “et al.” had lower α values, possibly because they more often discussed references, an area in which LLMs are weak.

Reviewers who posted more replies during the review-discussion stage had lower α values. Higher estimated AI use was also associated with greater agreement among reviewers. The concern is that, rather than receiving diverse human judgments, authors may receive similar LLM-mediated feedback. This could reduce the input of creative ideas into peer review and, if such text is reused as training data, may further reduce diversity.

The conferences in the study required reviewers to report a confidence score. When reviews were divided into low-confidence scores of 1–2 and higher-confidence scores of 3–5, the higher-confidence group had lower α values. It is unclear whether reviewers with low confidence seek more LLM assistance, or whether relying more heavily on LLMs lowers their confidence.

What the paper does and does not claim

The study finds higher estimated LLM use in situations that have plausible explanations, and it identifies a possible risk of reduced diversity as LLM use becomes more common in peer review. It does not attempt to judge whether LLM use in reviews is inherently good or bad.

Its results also raise an interpretive problem. If a reviewer creates the substance of a review as an outline and asks an LLM only to turn it into prose, the text can still be detected as generated. Detection alone therefore cannot settle the question of how much of the intellectual work was delegated.

My view

This is a timely warning in a period when more papers may be written with LLMs and more reviews may be produced with them as well. One strength of the study is that it examines how the estimated α value changes across several independent variables, addressing many of the questions readers are likely to have.

Is it acceptable for a human to prepare an outline and use an LLM only to turn it into sentences? For a paper itself, I think this raises at least a mild ethical issue, and I am not sure that increasing the writer’s dependence on such tools is desirable. A paper also has value as writing produced by its author; the selection and arrangement of words are part of the identity of the work.

A review is different. Its value lies less in the prose itself than in its role as a means of epistemic communication. Writing a review in bullet points and having an LLM convert it into sentences could help both the writer and the reader. This seems especially relevant when reviewers are unpaid and there is a chronic shortage of reviewers.