Verified telemetry cuts agent operational regret by 96.9% in controlled benchmark
Agent Harness Awareness supplies fresh, verified host state to models before consequential actions, reducing observable policy regret and boosting completion rates across three frontier models.
1 source · cross-referenced
- Agent Harness Awareness introduces a trust-separated design where the host supplies a fresh, verified state snapshot to the model before a consequential action, addressing 'operational blindness' in tool-using agents.
- In a 216-decision ablation across GPT-5.6 Sol, Claude Sonnet 5, and Gemini 3.7 Flash, verified tool-health state reduced mean observable policy regret by 96.9% (from 4,252 ms to 131 ms) and raised single-dispatch completion from 69.4% to 95.8%.
- A 27-decision live execution replication with injected faults showed verified state reduced mean regret from 8,956 ms to 0 ms and achieved 9/9 successful completions versus 5/9 for the neutral control.
- The approach retains host authority for measurement, enforcement, evaluation, and final dispatch while improving pre-dispatch operational choice.
Tool descriptions alone do not reveal whether an action is healthy or suitable at the moment of use, creating what the authors term 'operational blindness' in tool-using agents. Execution harnesses, however, often possess this information through monitors, traces, and recent failures, which the model does not have access to.
The proposed Agent Harness Awareness architecture addresses this gap with a trust-separated design: the host supplies a small, fresh, verified state snapshot to the model before a consequential action while retaining full authority for measurement, enforcement, evaluation, and final dispatch.
The authors evaluate the tools-and-network instance of this design using ToolRoute, a controlled isolation benchmark that holds task and candidate semantics constant while varying the operational information available to the model. In a 216-decision ablation across three frontier models—GPT-5.6 Sol, Claude Sonnet 5, and Gemini 3.7 Flash—verified tool-health state reduced mean observable policy regret by 96.9% compared to a same-shape neutral control (from 4,252 ms to 131 ms). Single-dispatch completion rates improved from 69.4% to 95.8%.
A separate 27-decision end-to-end replication with live local TCP/HTTP execution, OpenTelemetry client spans, and Toxiproxy-injected faults showed verified state reduced mean regret from 8,956 ms to 0 ms (zero excess observable operational cost). In this live setting, verified state achieved successful completion in all 9 model-by-fault cases, compared to 5 out of 9 for the neutral control.
The studies establish the tools-and-network instance of Agent Harness Awareness, demonstrating that selected host telemetry materially improves pre-dispatch operational choice. The authors note that runtime state, hardware pressure, economics, retry-aware recovery, and heterogeneous selection remain open empirical extensions of the same architecture.
- Nov 11, 2026 · The Verge — AI
Google DeepMind unveils AlphaGenome 2 to predict genetic variant effects on cell types
Trust79 - Nov 11, 2026 · Google DeepMind — Blog
Google DeepMind unveils AlphaGenome 2 to predict DNA variant effects in specific cell types
Trust84 - Nov 11, 2026 · arXiv cs.AI
Researchers demonstrate auditable human-agent teams for autonomous synchrotron X-ray experiments
Trust79