Kog hits 3,000 TPS on standard GPUs — can it scale to big models?

kog released a tech preview demonstrating 3,000 tokens per second (tps) inference speeds on unmodified nvidia h200 and amd mi300x gpus, using its laneformer 2b model and custom kernel optimizations. while initial demos focused on a small 2 billion parameter model to prove the potential of standard hardware, the company generated 200 business leads by promising significant speed improvements for expensive claude code workflows. the startup is now shifting focus to larger models after learning customers will not fine-tune smaller ones, but faces skepticism on whether its low-level assembly approach can scale without consuming excessive engineering time per gpu generation.