Hugging Face and Microsoft release ThinkingBox-Training to turn database checks into RL rewards for agent post-training

Microsoft and Hugging Face published a joint blog describing ThinkingBox-Training, a framework that uses executable checks of terminal database state as rewards to fine-tune agentic models outside the original ThinkingBox-Bench dataset.

The authors built ThinkingBox to catch agents that prematurely report completion when backend records remain incorrect; each of the 507 ThinkingBox-Bench tasks runs a real backend, provides MCP tools, and evaluates outcomes by comparing terminal database state to expected records rather than tool-call fidelity.

To curate a training pool, the team estimated offline policy-relative difficulty using the base policy’s success probability on each task and kept the pool fixed for the entire 50-step GRPO fine-tuning run of Qwen3.8-27B.

After training, average success on ThinkingBox-Bench rose from 51.70% to 60.78%, and the number of tasks completed correctly on all twenty attempts increased from 38 to 107, while the share of tasks solved at least once slightly declined from 89.35% to 87.97%.

The authors note that headline pass@1 can mask reliability gaps; they emphasize Observed 20/20—tasks passing every attempt—as a closer proxy for production dependability, where a model that solves a task only once is not necessarily safe to deploy.

They also report domain-level results on ThinkingBox-Bench, showing Qwen3.8-27B RL fine-tuned achieved 60.78% pass@1 overall and 21.10% Observed 20/20, compared with leading proprietary models such as Claude Opus 5.5 at 65.36% pass@1 and 47.53% Observed 20/20.

The implementation and training stack are provided in the accompanying guide, including steps to install, prepare data, configure and inspect the launch, and evaluate checkpoints through OpenEnv.

The post credits Microsoft’s Mirco Milletari, Xiang Jiang, Hugging Face’s Sergio Paniego, and former intern Zhuochun Li (University of Pittsburgh) for co-authoring and reviewing.