ai agents build 3d scenes from photos but lack self-assessment
researchers from university of maryland and aws created lego-anything, an ai agent that builds editable 3d scenes from single photos, but struggles with geometric accuracy and self-assessment. this impacts applications in image analysis and 3d reconstruction.
researchers at university of maryland and aws developed lego-anything, a project that uses a coding agent to build editable 3d scenes from single photos. the agent writes code for blender in an iterative process, producing a usable scene but struggling with geometric accuracy and self-assessment. the team introduced lego-bench, a benchmark with 208 images from 104 indoor and outdoor scenes, to measure agent performance. gpt-6 astra, the best tested model, scored 53.4% on indoor scenes and 39.6% on outdoor scenes. the agents' poor self-assessment and inability to judge geometric accuracy are major issues, suggesting that refinement should rely on concrete measurements rather than the agent's own judgment.