A guide to evade hallucinations and maintain reliability when using large language models for medical research: a narrative review.

English summary

This narrative review explains that although large language models (LLMs) are increasingly used in medical research, their autoregressive architecture and computational limitations create inherent reliability problems, particularly in specialized medical settings. Mitigation approaches such as Chain-of-Thought prompting and Retrieval-Augmented Generation (RAG) are discussed, but they do not fully remove the underlying reliability issues. Effective integration of LLMs into medical research requires task-appropriate tool selection, suitable verification procedures, and human oversight, especially for complex biological systems.

Key points

• LLM reliability limitations arise from autoregressive prediction mechanisms and computational constraints related to undecidability, making perfect accuracy unattainable.
• Chain-of-Thought prompting and Retrieval-Augmented Generation may help address hallucinations and reliability concerns but do not resolve the fundamental limitations.
• Meta-analyses of human–AI collaboration experiments found that LLMs can augment individual human capabilities, but they are most effective in contexts where humans can verify the outputs.

Audience

Medical researchers, clinicians, medical AI and data-science researchers, and advanced students

Scope and limitations

The supplied source is a narrative review summarizing technical limitations of LLMs and principles for their use in medical research. This summary is limited to the provided text; specific search methods, included studies, meta-analytic estimates, and study-level findings are not reported.

Research context

The topic has a neutral thematic connection to Sangzin Ahn’s portfolio in medical AI and data science through its focus on AI reliability, human–AI collaboration, and verifiable use of tools and data.