- Published on
ByteDance Seed Team Finds Cause of DeepSeek 'Glitches': Long Context May Lead to Performance Drift
- Authors
- Name
- DeepHorus
IT Home news on October 9: At the end of September this year, the ByteDance Seed team submitted a paper to the preprint platform arXiv, pointing out that phase sensitivity caused by chunked KV cache compression may explain why DeepSeek models perform unstably in long-context tasks. The study focuses on multiple versions of the DeepSeek series, evaluating both base and post-trained models. The paper link is https://arxiv.org/abs/2609.36322.
As demand for long-context reasoning in large models grows, chunked KV cache compression has become a means of reducing memory and attention costs. This technique compresses windows of consecutive tokens into fewer cache entries at a fixed stride. DeepSeek-V4-Flash, DeepSeek-V4-Pro, and the post-trained DeepSeek-V4.1-Flash were all included in the evaluation.
The team found that this compression introduces a new positional coordinate—the phase of a token, that is, its position relative to the boundary of the compression window. In models that use this compression, there is a systematic asymmetry: the same information is easy to retrieve at one phase but difficult to retrieve at another. The team calls this periodic variation in retrieval performance phase sensitivity.
In large open-weight models that adopt such compression, long-context retrieval accuracy can differ by as much as 40 percentage points between different phases. This means average benchmark scores may mask periodic weaknesses. The finding points directly to the cause of DeepSeek's 'glitches,' namely that performance fluctuations are related to phase.
For developers, looking only at average scores may overestimate a model's stability in long contexts. Although chunked KV cache compression saves memory and attention costs, it may introduce retrieval blind spots. For users, the retrieval effect for the same information may be noticeably different at different positions.
The ByteDance Seed team's paper suggests that long-context optimization needs to pay attention to compression window boundaries and phase distribution. Future model design may need to introduce phase-aware retrieval mechanisms and evaluation methods. This research also provides a new perspective for the stability evaluation of open-weight models, and its subsequent impact remains to be seen.
