연구용 AI의 한계와 윤리적 고민 (2025.3)

English summary

This presentation addresses the current capabilities of research-oriented artificial intelligence, its limitations and ethical risks, and policy responses for researchers and institutions. Examples involving AI Scientist, natural-language data analysis, and multi-agent search tools illustrate the potential for research automation, while medical hallucinations, privacy and security, authorship, plagiarism and originality, and bias in peer review are presented as major concerns. Its central position is that institutions should not rely on blanket bans, but should strengthen verification, transparent disclosure, and accountable human oversight.

Key points

• AI Scientist is presented as an automated research pipeline linking idea generation, literature-based novelty checking, code and experiment generation and execution, result visualization, manuscript drafting, and automated review. According to the presentation notes, three papers generated without human intervention were submitted to an ICLR 2025 workshop, and one was accepted with an average review score of 6.3.
• OpenAI Data Analyst and comparable tools can perform data exploration, machine-learning modeling, statistical evaluation, and visualization from natural-language instructions alone. The presentation demonstrated linear regression, random forest, and gradient boosting on a warfarin dosing prediction dataset.
• Research-oriented large language models may generate medical hallucinations involving incomplete reasoning, factual errors, outdated references, spurious correlations, and fabricated sources or guidelines. The reuse and resynthesis of unverified open-access or predatory-journal content may propagate errors into new research outputs. The presentation showed a case in which search results about moderate alcohol consumption overrepresented positive associations relative to current medical consensus, as reported in the presentation notes.

Audience

Medical and life-science researchers, journal editors, peer reviewers, research ethics committees, and academic policy makers.

Scope and limitations

This summary is based only on the supplied evidence notes from the presentation. The original sources, analytical methods, samples, and context of the cited studies and numerical findings were not independently verified, and the reported examples and results cannot be assumed to generalize to all research AI tools. Privacy settings, synthetic data, and locally deployed models were presented as risk-reduction options, not as guarantees of complete security.

Research context

The material is thematically connected to medical AI and data science through its focus on research automation, data governance, algorithmic validation, and research ethics, providing a neutral thematic context for Sangzin Ahn's portfolio.