Critique of Bengio's Scientist AI Proposal
a critique of yoshua bengio's 'scientist ai' proposal argues that restricting ais to non-agentic roles is both unsafe and impractical for scientific discovery.
a detailed critique of yoshua bengio's 'scientist ai' proposal argues the plan is flawed for both safety and practical reasons. the author asserts that a non-agentic ai cannot reliably perform science without causal inference, which requires the active data collection found in reinforcement learning. furthermore, the critique suggests that restricting an ai from taking actions fundamentally limits its intelligence compared to agentic models. despite these criticisms, the author endorses bengio's short-term focus on fine-tuning llms to hypothesize risks regarding user requests.