- Published on
DeepSeek AI Released DeepSeek-V4.1-Flash with 1M Context, FP4 KV Cache, and Cross-Layer Attention Reuse
- Authors
- Name
- DeepHorus
DeepSeek AI has unveiled DeepSeek-V4.1-Flash, a new multimodal Mixture-of-Experts model designed to tackle the growing problem of memory-intensive long-context AI workloads. The model features a 1-million-token context window and innovative compression techniques that dramatically reduce its memory footprint. Its release addresses a critical bottleneck in deploying advanced AI agents, where repeated prefill operations and massive contexts can overwhelm hardware resources.
The new model is a significant technical achievement, boasting a backbone of 552 billion parameters plus 196 billion additional Engram parameters. Crucially, it activates only 8 billion parameters per token during prefill and 16 billion during decode. The headline figure is a global KV cache footprint of just 890 bytes per token, which is roughly a quarter the size of its predecessor and over 400 times smaller than the original DeepSeek-V1.
This efficiency stems from a novel 'Causal Encoder-Decoder' architecture. The model splits its 40 layers into a 20-layer encoder and a 20-layer decoder. The decoder avoids computing its own full key-value cache by deriving it from the encoder, which nearly halves the computational cost of processing long prompts.
Further memory savings are achieved through Compressed Sparse Attention 2 (CSA2) and FP4 quantization for the KV cache. These techniques aggressively compress data storage. Additionally, the deployment system is optimized to keep smaller, transient data in fast memory instead of slower storage, enhancing overall speed.
DeepSeek reports strong performance, stating the base model matches the more powerful DeepSeek-V4-Pro-Base on key benchmarks while using only a third of the parameters. The model is now available under an MIT license, with support for popular frameworks like vLLM and Hugging Face Transformers. A public API is also offered with different reasoning tiers.
The release signals a major step forward for practical, long-horizon AI agents. By solving core memory and efficiency challenges, DeepSeek-V4.1-Flash makes it feasible to deploy more complex AI systems that can maintain context over millions of tokens. This could accelerate development in areas requiring extensive memory, from advanced coding assistants to sophisticated research tools.
