연구에 Agent가 다가온다 (2025.3)

English summary

The video describes recent advances in autonomous AI agents and automated scientific research systems. It focuses on Sakana AI's The AI Scientist, OpenAI's agent-building tools, and Anthropic's Model Context Protocol, discussing automation from idea generation through experimentation, manuscript preparation, and review, as well as changing roles for human researchers.

Key points

• Sakana AI's The AI Scientist uses large language models to generate research ideas, check novelty through Semantic Scholar, and score and archive candidate ideas.
• The system fills experiment templates, writes and modifies code, runs experiments, analyzes numerical data, produces plots, drafts manuscripts with citations, and uses language-model-based review to provide iterative feedback.
• In an ICLR 2025 workshop experiment, three AI-generated papers were anonymously reviewed with 40 human submissions after prior approval and institutional ethics clearance. One paper received an average reviewer score of 6.33, corresponding to approximately the top 45 percent of workshop submissions, and passed the acceptance threshold; the other two did not. Under the prespecified experimental and disclosure rules, all AI-generated papers were withdrawn after review. This was a workshop-level result, and the source does not generalize it to overall scientific performance. Analysis against higher-level conference standards identified citation-origin errors and imprecise mathematical definitions, including an example that incorrectly attributed the foundations of LSTM to Goodfellow et al., 2016. The stated limitation is therefore that the system can accelerate the generation of experiments and manuscript drafts, but scientific validity and accuracy still require separate human verification.

Audience

Graduate students, medical AI and data science researchers, software engineers, machine learning practitioners, and readers interested in automated scientific discovery and large language model agent frameworks

Scope and limitations

OpenAI's Responses API combines conversational capabilities with web search, file search through vector stores, and computer-use tools. The Agents SDK supports agents with configurable instructions and tools, multi-agent orchestration and handoffs, input and output validation, and execution tracing. Anthropic's Model Context Protocol standardizes connections between natural-language clients and tool servers for databases, file systems, and web APIs. The video discusses the possibility that expansion into real-world laboratory automation could shift human work toward planning, oversight, and validation, but states that the scientific community has not reached consensus on ethics, authorship, submission criteria, or review standards for purely or heavily AI-generated manuscripts. The evidence presented is limited to a specific workshop experiment, selected benchmark examples, and tool demonstrations; it is not a systematic evaluation of long-term effects or general research performance.

Research context

The topic is thematically connected to medical AI and data science through agent-based research automation, standards for connecting computational tools, reproducibility, and human verification and accountability.