Preprint quantifies cost and accuracy impact of explicit reasoning-effort terms in model API contracts

Researchers conducted a preregistered, paired-contrast experiment comparing Sonnet 5 under two contract conditions: one with an explicit high-reasoning-effort term and one with the term omitted. The study used 30 AIME 2026 items and five API calls per item, for a total of 300 paid attempts.

Every paid attempt was assigned a frozen terminal category, and the inference protocol resampled items while retaining repeated calls to the same item. The primary outcome was mean delivered cost per call, which was $0.01031 higher under the explicit-high contract than under the omitted contract, with a 95% confidence interval of [$0.00204, $0.01974].

The corresponding accuracy contrast was +0.0133 points, with a 95% interval of [-0.0267, +0.0467]. The authors report that no statistically detectable accuracy difference was observed, and the interval permits a potential gain of up to 4.67 percentage points that the design could not rule out.

Registered point estimates for cost per correct answer were $0.08665 under the high-effort contract and $0.07662 under the omitted contract. The authors also document model-specific omission semantics via a dated contract census, Models-API metadata, and preregistered raw-response probes, noting that claims remained at documentation grade when raw structure was indeterminate.

The request registry, parser, terminal taxonomy, statistical plan, and analysis pipeline were frozen before outcomes were examined, bounding the claims to the specific model, task, and collection date studied.