New verification method improves stateful tool-agent success by 13.2 percentage points in multi-turn tasks

A single plausible but incorrect tool call can derail an otherwise correct agent trajectory. TwinCheck addresses this by introducing an inference-time verification policy that replaces an agent’s proposed action only when the trace meets an evidence condition tied to a hypothesized failure, constructs a counterfactual alternative (a “negative twin”), and validates the replacement via structural checks and a pairwise verifier in both candidate orders.

In paired evaluation using exact replay on 159 multi-turn BFCL V4 tasks, the complete TwinCheck policy increased task success for GPT-5.6 Sol from 45.3% to 58.5%, with a 95% task-bootstrap confidence interval of [8.2, 18.8]. No success-to-failure regressions were observed in the primary analysis.

The method reframes execution-boundary repair as a constrained comparison where the counterfactual action itself becomes the object of verification, separating intervention effects from resampling noise by holding parsed responses and actions fixed until the first accepted replacement.