在 iPhone 上运行 27B 模型正在扼杀云端
prismml 發布了 bonsai 27b，這是一款僅 3.9gb 的推理模型，可安裝在 iphone 上，保留了基礎 qwen 90% 的性能，同時實現了零延遲的本地代理。
prismml, founded by caltech researchers, released bonsai 27b, a 27b-parameter model based on alibaba's qwen that compresses weights to just 1-2 bits to fit entirely on an iphone 17 pro max. the small variant uses 3.9gb of space and generates roughly 11 tokens per second, allegedly maintaining 90% of the original model's performance and near-perfect accuracy in math and coding tasks. ceo babak hassibi confirmed that apple and other companies are testing the technology for speed and power efficiency. this compression strategy allows complex multi-step agents to run locally without cloud latency or data exposure, potentially enabling a new class of always-on, privacy-first ai applications.