New benchmark EarthRecord tests multimodal agents on four decades of satellite imagery for land-change analysis
Benchmark evaluates 12 multimodal agents on raw multi-sensor archives, revealing gaps in mapping from pixels to real-world change.
1 source · cross-referenced
- EarthRecord introduces a benchmark using raw satellite imagery from 56 sites spanning four decades (Landsat, MODIS, VIIRS, Sentinel-2) to test multimodal agents' ability to perform land-change analysis.
A new benchmark called EarthRecord evaluates how well multimodal agents can transform decades of raw satellite imagery into reliable land-change analysis, a task central to Earth observation science. Unlike prior benchmarks that rely on curated or preprocessed inputs, EarthRecord uses the raw multi-sensor archives from 56 sites, including Landsat, MODIS, VIIRS, and Sentinel-2 data, and leaves cloud masking and cross-sensor harmonization to the agent.
The benchmark comprises 238 land-change questions across ten analysis tasks, with answers scored against withheld references from global products, agreement among methods, and visual interpretation. Two control tests are included: a closed-book condition that removes the record entirely, and an instrument-pairing condition that alters one satellite's data while keeping ground conditions unchanged. These controls help isolate whether agents' answers derive from the imagery or prior knowledge.
Evaluations of twelve leading multimodal agents show that while most complete analyses end to end, performance drops when answers require mapping directly from imagery. Further analysis reveals that access to the record shifts agents' reliance from prior knowledge to pixel-level data, but not yet to accurate real-world change: when one satellite's data are altered, half of the previously correct answers follow the alteration.
The authors argue that EarthRecord provides a principled platform for measuring and advancing agents' ability to derive real change from pixels, a capability currently lacking in existing benchmarks.
- Nov 11, 2026 · The Verge — AI
Google DeepMind unveils AlphaGenome 2 to predict genetic variant effects on cell types
Trust79 - Nov 11, 2026 · Google DeepMind — Blog
Google DeepMind unveils AlphaGenome 2 to predict DNA variant effects in specific cell types
Trust84 - Nov 11, 2026 · arXiv cs.AI
Researchers demonstrate auditable human-agent teams for autonomous synchrotron X-ray experiments
Trust79