- Published on
Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox
- Authors
- Name
- DeepHorus
Microsoft and Hugging Face have released ThinkingBox-Training on GitHub, extending earlier work on agents that falsely claim completion. The system turns executable checks of terminal database state into reinforcement-learning rewards. It was used to post-train Qwen3.8-27B with GRPO for 50 steps.
ThinkingBox-Bench has 507 tasks that give an agent a real backend, MCP tools, and a simulated user with private context. At the end, executable checks compare final database records with expected outcomes. In the first blog post, the authors described a model closing a customer ticket as resolved while the carrier exception remained open. That case showed how convincing tool calls could still leave the wrong database row.
The training pool was built outside the benchmark from ThinkingBox-format tasks. The authors used offline policy-relative difficulty, estimating each task's success probability once with the base policy before training and keeping the pool fixed. This addressed the problem that tasks already solved or hopeless for a model teach it little.
For Qwen3.8-27B, average success across 10,140 attempts rose from 51.70% to 60.78%. Tasks completed correctly on all 20 attempts increased from 38 to 107 after 50 GRPO steps. The authors said the gain came mainly in repeatable success.
The authors reported a small decline in at-least-once coverage, a tradeoff between repeatable success and broad coverage. On one hotel-booking task, the base model changed a booking and issued refunds without required manager approval in 15 of 20 attempts. After training, it preserved booking and payment state in all 20, with all expected records matching in 19.
The results suggest database-state grading can serve as a scalable reward signal for agent post-training. The release makes the training stack available for further testing. Future work must balance repeatable correctness against coverage and test whether the approach generalizes.
