OpenAI's Astra sparks safety fears with opaque design
openai's upcoming astra model uses a more opaque technique, sparking concerns among researchers that it could be harder to monitor and detect undesirable behavior.
openai is on the cusp of releasing its most powerful ai model yet, astra, following weeks of delays to shore up safety protocols. astra uses a more opaque technique known as a recurrent depth or looped transformer, which cycles information through internal layers before producing an output, making it harder to monitor. researchers, including redwood research's chief scientist ryan greenblatt, warn that this could lead to a safety disaster and a 'race to the bottom' in ai safety. openai has limited its use of the looped transformer technique with astra to allow for continued monitoring of the model's reasoning.