- Published on
Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard
- Authors
- Name
- DeepHorus
Memdoor has shown a five-agent Qwen3.8-27B team matching Fable 5 on a small LiveCodeBench Hard sample. The team uses a planner, two ideators, a solver, a manager and a repair worker. Agents start with fresh context and communicate only through disk files; a script decides success.
The setup is GVS5H (Lee, 2026), a Memdoor workflow with eight YAML files and no code or model name. The authorsā harness was rebuilt on the same Qwen3.8-27B model with their prompts, loop and 128K reply cap. No model is named in the workflow.
The test covered ten LiveCodeBench Hard problems, five passes each, graded on hidden tests with a corrected grader. The 50 runs had a median duration of 40 minutes and cost $0.39 each; total cost was $35.79. Every run was logged.
Results are in bench/results/parity-5pass/, with every request, reply, program, grader output and cost. Comparison columns come from the authorsā regraded files and paper. The claim is parity, not a gain: on ten problems and five passes, 9.8 and 9.6 are effectively the same.
Limits remain: contests date from spring 2025, so contamination cannot be ruled out. Fable 5ās column is one call per problem from the authorsā files and was not run in this harness. The loop sees only public samples while scoring uses hidden tests; the model ran via OpenRouter, not locally.
One problem was lost when the manager invented a wrong counterexample and the team rewrote a correct program. Twice elsewhere a correct program was overwritten and later recovered. Both gaps belong to the authorsā loop and were kept unchanged.
The DAG runs plan, two ideators, solve, test, review, repair and verify. Prompts tell agents to read files, not rely on chat. The paper credits ledger discipline; users can connect OpenRouter or local backends and run team-solve.
