Researchers propose Kernel Forge, an agentic harness to optimize CUDA kernels for PyTorch models using LLM-driven search

Researchers from the University of Michigan and collaborators introduce Kernel Forge, an open-source, end-to-end agentic harness designed to optimize CUDA kernels for PyTorch models without requiring manual low-level GPU coding. Unlike prior tools that target isolated kernels or randomly generated tensors and emit standalone code requiring manual reintegration, Kernel Forge ingests unmodified PyTorch models and performs in-place kernel optimization.

The system supports vision, diffusion, and large language model workloads and employs Monte Carlo Tree Search (MCTS) to explore multiple optimization paths rather than a single linear refinement chain. It also includes a graphical user interface for monitoring progress, inspecting candidate kernels, and debugging failures.

In evaluations on an NVIDIA DGX Spark with GB10 GPU, Kernel Forge optimized 14 kernels across four PyTorch models. With only 50 optimization iterations per kernel, it achieved speedups over PyTorch eager mode: 1.52× on adaptive_avgpool2d in ResNet-50, 1.70× on group_norm in Stable Diffusion 3.5 Medium, 2.83× on softmax in Gemma 4 E2B, and 1.54× on softmax in Qwen 3.5 35B-A3B.

The authors release Kernel Forge as open-source software and provide a code repository link in the paper.