Anima Anandkumar argues physics modeling requires inductive biases, not just scale

Anima Anandkumar, Bren Professor of Computing at Caltech and former NVIDIA director of machine learning research, argues that the AI field lacks foundation models for physics and must encode physical priors to make progress in modeling the physical world. She states that weather, fusion, and fluid or heat flow are large, chaotic, and multi-scale systems that resist the scaling ideas that have dominated other areas of AI.

Anandkumar's team developed FourCastNet, an open-source weather model that is competitive with the best physics-based simulations and can run on consumer-grade GPUs. The model leverages Fourier Neural Operator and spherical harmonics to stabilize long-term forecasts, addressing the instability of grid-based weather models.

She notes that open-source datasets in many physical domains are limited to tens or hundreds of thousands of examples, far below what token-hungry transformers require, and that the resolution demanded by physics pushes context length into the hundreds of billions, making pure scaling impractical.

Anandkumar pioneered neural operators, a technique that combines data and physical laws to enable multi-scale inputs and outputs, allowing the incorporation of physical priors. She argues that progress in physics modeling comes from building in structure and inductive biases rather than waiting for data that may never exist.

Beyond weather, Anandkumar has applied neural operators to fusion, where a few thousand samples can predict plasma disruptions a million times faster than traditional simulation. She aims to build a foundation model for physics that spans many phenomena and supports both simulation and design.

Anandkumar also discussed integrating neural networks with automated proof techniques, highlighting TorchLean, a framework that lets researchers write PyTorch-style networks inside the proof assistant Lean and formally verify them. This work is aimed at proving bounds on neural networks, which is critical for safety-critical applications like fusion reactor control.

Anandkumar was recently appointed to the United Nations Scientific Advisory Board, where she aims to bring evidence-based viewpoints to policy and improve lives through AI in scientific domains.