French startup Kog targets faster LLM inference on existing GPUs with low-level optimization

Kog, a French startup, argues that conventional GPUs remain underutilized for AI inference and is betting on low-level software optimization to unlock more performance from existing hardware. The company’s approach targets the inference bottleneck that has constrained agentic and professional AI workflows, according to CEO Gaël Delalleau.

Kog’s initial demonstration used the open-source Laneformer 2B model on AMD MI300X and Nvidia H200 GPUs, achieving 3,000 tokens per second per request. The company claims this validates its broader promise of “30x faster LLM inference,” though it has not yet demonstrated the same speedups on larger models.

Delalleau contends that skepticism about GPUs’ suitability for decoding is a misconception, pointing to increasing memory bandwidth in newer GPUs as an untapped resource. He describes Kog’s method as akin to reverse-engineering hardware behavior down to assembly and binary code to maximize efficiency.

The startup’s engineering process is labor-intensive: for each new GPU, Kog dedicates weeks or months to deep hardware research before deploying its optimizations. With a team of 11, this limits the number of supported chips in the near term.

Kog has generated 200 business leads and expects software engineers—particularly users frustrated by slow tool-assisted workflows—to be early adopters. The company also works with design partners in gaming and app generation, where faster inference translates directly to revenue.

Kog’s seed round was co-led by Varsity VC, with support from Scaleway and France’s Bpifrance and French Tech 2030 program. The startup aims to demonstrate a 10x speed improvement on a major model by September 2026 to secure Series A funding and prove broader applicability.

Kog is not alone in pursuing software-driven GPU optimization; competitors like ZML offer hardware-agnostic software that bypasses Nvidia’s CUDA. However, Delalleau positions Kog’s work as deeper and more focused, drawing a parallel to Stanford’s Hazy Research lab.