Taalas chips hit 16k tps — but they're locked to one model
amd acquired taalas, whose chips hard-code llm weights to hit 16k tokens/second. this trade-off sacrifices flexibility for extreme speed, signaling a shift to model-specific hardware.
amd acquired toronto-based taalas, which hard-codes ai model architecture and parameters directly into silicon to maximize inference speed. a demo chip running llama 3.1-8b achieved over 16,000 tokens per second per user, significantly outpacing standard gpus. the technology will be integrated into amd's accelerator roadmap, offering a specialized solution alongside its general-purpose instinct cards. this move signals a potential diversification in hardware design, balancing general-purpose compute with model-specific accelerators.