Running 27B models on an iPhone kills the cloud
prismml released bonsai 27b, a 3.9gb reasoning model that fits on an iphone, retaining 90% of the base qwen performance while enabling zero-latency, local agents.
prismml, founded by caltech researchers, released bonsai 27b, a 27b-parameter model based on alibaba's qwen that compresses weights to just 1-2 bits to fit entirely on an iphone 17 pro max. the small variant uses 3.9gb of space and generates roughly 11 tokens per second, allegedly maintaining 90% of the original model's performance and near-perfect accuracy in math and coding tasks. ceo babak hassibi confirmed that apple and other companies are testing the technology for speed and power efficiency. this compression strategy allows complex multi-step agents to run locally without cloud latency or data exposure, potentially enabling a new class of always-on, privacy-first ai applications.