- Published on
Qwen3.6-35B-A3B on a 16 GB M1 Pro with SSD-streamed MoE
- Authors
- Name
- DeepHorus
A new fork of the specialized DwarfStar inference engine is pushing the boundaries of running large language models on consumer hardware. The project, created by developer andreaborio/ds4, demonstrates how models like Qwen3.6-35B-A3B can operate on a 16 GB M1 Pro MacBook by streaming model data from SSD. This approach opens the door for developers with modest Apple Silicon machines to experiment with large MoE (Mixture of Experts) models that would otherwise exceed memory limits.
The DwarfStar engine, originally developed by antirez, is a compact, self-contained inference engine designed for a narrow set of very large models. It includes native model loading, prompt rendering, tool calling, RAM and on-disk KV state, an HTTP server, and a coding agent. The upstream project focuses on high-memory and distributed paths, while this fork explores an alternative question: how far can the same specialized design be pushed on the 16β64 GB Macs many developers already own, when the SSD becomes an active model-memory tier?
Key to this effort is the adaptive memory management system that decides whether to keep a model resident or stream it from SSD. When a model exceeds the available RAM budget, the engine uses a routed-expert cache policy to page in only the necessary parts on demand. This allows a 35-billion-parameter model with 3.6 billion active parameters (Qwen3.6-35B-A3B) to run on a 16 GB machine, albeit as an experimental low-memory tier with no speed guarantees. A 64 GB Mac is considered the practical reference tier for DeepSeek Flash streaming.
The current work concentrates on several technical areas: adaptive Metal residency and routed-expert cache policies across memory tiers; SSD streaming that accounts for page cache, wired memory, swap, I/O, and throughput together; safer model-backed experiments near macOS memory limits; and measured bring-up of GLM 5.2 and Qwen3.6-35B-A3B models. The project also includes GGUF calibration, incremental quantization, and expert-analysis tooling. All changes are kept small enough to validate and potentially send upstream.
The project explicitly positions itself as a complement, not a competitor, to the upstream DwarfStar engine. Every fork change applicable to an upstream-supported path will be opened upstream once its scope, correctness, and performance evidence are ready. Current upstream work includes fixes for quality-score build, GLM streamed-prefill correctness, and indexed-prefill prepare. A DeepSeek regression found on the GLM line is being tracked separately.
Implications for the broader AI development community are significant. By enabling large models on standard consumer hardware, this approach could democratize access to advanced inference capabilities. Developers can now run sophisticated models locally without requiring expensive cloud instances or high-end workstations, potentially accelerating research and experimentation. However, the experimental nature means users should expect variable performance on low-memory systems.
Looking ahead, the project will continue to refine its SSD streaming techniques and explore further model support. The ultimate goal is to contribute general, reproducible improvements back to the upstream DwarfStar project, benefiting the entire ecosystem. As hardware and software optimizations mature, running models of this scale on 16 GB machines may become practical for everyday use, though speed and memory constraints will remain challenges.
