Is it a research ethics violation to insert a secret phrase aimed at AI? (2025.7)

English summary

The video examines hidden prompts embedded in academic manuscripts to manipulate large language model based peer review, together with the related research integrity and security issues. It explains that altering text to distort evaluations for personal benefit may constitute falsification or manipulation of research materials or processes, while also emphasizing the need to distinguish legitimate review assistance from malicious interference.

Key points

• Prompt injection hides instructions in inputs, such as invisible white text or subtle text in images, to override earlier instructions and control model output.
• The video presents examples of hidden instructions seeking favorable manuscript evaluations and relates this practice to an analogy with LLM based search optimization.
• In the described demonstration, hidden text in a manuscript prompted the model to include a specified word and metaphor in its review. The model otherwise produced conventional review comments but still followed the embedded instruction.

Audience

Academic authors, peer reviewers, journal editors, research integrity professionals, and artificial intelligence and natural language processing researchers

Scope and limitations

This summary is based only on the supplied description of the video, its cited materials, and its reported demonstration. The news reports, preprint, claims about review bias and disciplinary accuracy, and other cited evidence were not independently verified here. A single demonstration cannot be generalized to all LLMs or review settings.

Research context

The topic is thematically connected to the use of LLMs in medical and scholarly data environments, including prompt injection, evaluation reliability, privacy, and data security. The video identifies section summarization, assistance with finding strengths and limitations, and checks of internal consistency, such as matching participant counts across an abstract and the main text, as potential legitimate uses. It also describes positive bias, superficial criticism, homogenized feedback, and weak citation verification as limitations of LLM based review. Proposed safeguards include adjusting service data controls, using temporary chats, changing activity settings, relying on policies that do not use standard inputs for training by default, and running local models. Rather than an outright ban, the video proposes usage guidance and reviewer education, along with pre-review detection and quarantine of hidden instructions.