Can GPT-4 Provide Useful Peer Review?

The Study

The paper, “Can Large Language Models Provide Useful Feedback on Research Papers? A Large-Scale Empirical Analysis,” by Weixin Liang and colleagues, investigates GPT-4-generated peer-review comments in two ways.

The retrospective study compared GPT-4 reviews with existing human reviews. The dataset included 3,096 papers and 8,745 reviews from Nature-family journals, as well as 1,709 papers and 6,506 reviews from ICLR.

The prospective study allowed researchers to upload their own papers, receive a GPT-4 review, and then evaluate that feedback through a survey.

How Closely Did GPT-4 Match Human Reviewers?

For Nature-family journals, human reviewers overlapped with one another by 29% in the content they raised. GPT-4 overlapped with at least one of two human reviews by 58%, and its overlap with each individual reviewer was 31%. In other words, its similarity to a single human reviewer was roughly at the human-to-human level (Figure 2a).

For ICLR papers, overlap between human reviewers was 35%. GPT-4 overlapped with at least one human review by 77%, and with each individual reviewer by 39% (Figure 2b).

Agreement between AI and humans, as well as agreement among humans, declined for papers selected for oral presentations (the top 5%) and spotlights (25%). Agreement was higher for rejected papers. This may mean that rejected papers have clearer, more obvious problems that both AI and human reviewers tend to identify. It suggests that AI may be useful for catching major issues early in the process.

Was the Feedback Generic?

One concern is that an LLM might produce only generic comments rather than paper-specific feedback. The researchers mixed GPT-4 reviews across papers and then measured their alignment with human reviews. The overlap fell from 31% to 0.43%. This suggests that GPT-4 was providing non-generic feedback tied to the particular paper (Figures 2a and 2b).

The more human reviewers independently identified a particular issue—whether it was raised by one, two, or three reviewers—the more likely the LLM was to identify it as well (Figures 2e and 2f).

Reviewers typically present more important issues earlier in their comments. When reviews were divided into four sections, content appearing earlier in a human review was more likely to overlap with GPT-4 feedback (Figures 2g and 2h).

Different Areas of Attention

The authors categorized ICLR review comments into 11 areas. Human reviewers most often focused on novelty, at 10.7 times the baseline, and ablation experiments, at 6.7 times the baseline. The LLM focused more heavily on the implications of the research, at 7.3 times the baseline, and requests for additional data experiments, at 2.2 times the baseline.

Because humans and the LLM emphasized different areas, the difference need not be interpreted as purely negative. It may indicate an opportunity for collaboration in which each compensates for what the other misses.

Prospective Study Results

The survey results were broadly consistent with the retrospective findings. Seventy percent of respondents said the GPT-4 review had at least partial alignment with human feedback (Figure 4b).

Regarding specificity, 33% said GPT-4 was more specific than some human reviewers, 17% said it was as specific as human reviewers, and 14% said it was more specific than most human reviewers (Figure 4d).

Fifty percent said the feedback was helpful, and 7% said it was very helpful (Figure 4a). Twenty percent said it was as helpful as human feedback, while another 20% said it was more helpful than human feedback (Figure 4c). Fifty-one percent said they would like to use the system again (Figure 4g).

Respondents were optimistic that the system could help the human peer-review process (Figures 4e and 4f), although they thought authors, rather than reviewers, would benefit the most from it (Figure 4h).

Limitations and Discussion

Sixty-five percent of respondents reported receiving GPT feedback on issues that had not been pointed out by human reviewers. At the same time, some respondents felt that the system lacked concrete and actionable feedback.

The paper discusses how LLMs could help researchers outside the mainstream, including those who attend fewer conferences or face desk rejection without receiving substantive reviews. However, the authors argue that LLMs should primarily be used by authors before submission. If they are used mainly within the formal review process, the complementary synergy between human and LLM feedback may not emerge.

The study contributes to recent work suggesting that LLMs can assist at multiple stages of research. Further experiments are needed, including tests in which errors are deliberately introduced into well-written papers to see whether an LLM can detect them. Designing effective AI-assisted review systems will require a precise understanding of LLM weaknesses. More research is also needed on researchers whose first language is not English.

My Take

The study design is particularly impressive because it combines retrospective and prospective research in one project.

I also find its interpretation of the difference between LLM and human review especially strong. Rather than treating those differences only as a failure, the authors frame them as a possible basis for collaboration: humans and LLMs may cover one another’s blind spots.

I have often argued from experience that LLMs can be useful for obtaining feedback after writing a paper. It is encouraging to have empirical results that can support that view.

I tried the prompt “Be thoughtful and constructive,” and it worked quite well. I plan to use it more often.