Why top LLMs still fail at simple riddles and visual puzzles

mit technology review examined current ai limitations using a series of logic and spatial puzzles, citing a columbia university study where models improved from solving 18% to nearly 100% of nyt connections puzzles between late 2024 and early 2025. despite this progress, the report highlights that frontier models continue to fail at spatial rotation tasks often found in iq tests and are easily tricked by slight variations of classic logic puzzles like knights and knaves due to over-reliance on memorized training data. these tests reveal that while llms excel at trivia, they lack the robust cognitive flexibility and 3d manipulation capabilities inherent in human spatial thinkers, suggesting that 'world models' have yet to master physical reasoning.