- Published on
AI systems quietly drop user instructions when they compress context
- Authors
- Name
- DeepHorus
AI systems that compress conversation history to free context space are silently dropping user instructions. A Penn State University study finds only 17 percent of session constraints survive compaction. The findings expose a critical weakness in how AI labs handle long chats.
Context windows become bottlenecks when users never start a new chat. Compaction summarizes existing history to free space, but details inevitably vanish. Session constraints—rules like "confirm before making changes"—are the most fragile because they aren't part of the main task.
The researchers built COMPINT, an evaluation suite, to measure the loss. Across most tested compactors, only 17 percent of injected constraints survived. Many compactors performed worse than running without compression, though GPT-5.4-mini beat that baseline in some scenarios.
With full uncompressed context, rule compliance ranged from 59 to 71 percent. After compression, compliance plummeted, often near the level where no constraint was given. Even a purpose-built compression prompt kept retention below 40 percent.
This creates quality and security problems. Agents may make unauthorized tool calls, disclose withheld information, or skip required verification. A user who says "don't send emails without approval" can't trust that after compaction.
The fix is a small add-on module based on Qwen3.5-9B. It scans every user input, detects session constraints, and stores them separately. The list is appended to the summary, keeping the rules intact.
The extractor achieved over 90 percent retention in all scenarios. That includes 95.6 percent for agent trajectories, 95.1 percent for long-term research, and 90.3 percent for multi-turn chats. It needs no training and no changes to the compression system.
COMPINT and the extractor are available on GitHub. This plug-and-play approach could make long-running AI interactions safer. But commercial adoption will determine whether users actually benefit.
