Reka AI's Rho-1 handles text, images, video, and robot control in one model

reka ai has released a research preview of rho-1, a 19-billion-parameter omni-model that processes and generates text, images, video, and robot control actions in a single neural network. unlike most ai systems, rho-1 runs all modalities as tokens in one shared context window with no tool calls or external models. rho-1 was trained on 320 h100 gpus over about three months and uses an inverse dynamics model to pull control signals from ordinary internet videos. this release builds on reka ai's previous work, including the april 2024 release of reka core, a multimodal language model that competed with gpt-4, claude 3, and gemini ultra on benchmarks.