iPhoneで27Bモデルを動かせばクラウドは不要になる
PrismMLが、iPhoneに収まる3.9GBの推論モデル「Bonsai 27B」をリリース。ベースのQwenの性能の90%を維持しながら、ゼロレイテンシのローカルエージェントを実現。
prismml, founded by caltech researchers, released bonsai 27b, a 27b-parameter model based on alibaba's qwen that compresses weights to just 1-2 bits to fit entirely on an iphone 17 pro max. the small variant uses 3.9gb of space and generates roughly 11 tokens per second, allegedly maintaining 90% of the original model's performance and near-perfect accuracy in math and coding tasks. ceo babak hassibi confirmed that apple and other companies are testing the technology for speed and power efficiency. this compression strategy allows complex multi-step agents to run locally without cloud latency or data exposure, potentially enabling a new class of always-on, privacy-first ai applications.