- Published on
An Apache 2.0 EEG Foundation Model With Variable-Length Inputs From 0.5 To 30 Seconds
- Authors
- Name
- DeepHorus
This week, Zyphra released ZUNA1.1, an open-source EEG foundation model under the Apache 2.0 license. The model reconstructs, denoises, and upsamples brain signal data across arbitrary electrode layouts, addressing the messy nature of real-world EEG recordings. This flexibility matters for researchers working with variable session lengths and noisy channels, from four-electrode headbands to 256-channel research caps.
Building on ZUNA1, the new model introduces variable-length input processing from 0.5 to 30 seconds, a significant improvement over the fixed five-second segments of its predecessor. ZUNA1.1 is a 380-million parameter masked diffusion autoencoder designed for scalp-EEG signals. It runs on consumer GPUs and works acceptably on CPUs for many workloads, with weights on Hugging Face and code on GitHub.
The architecture's flexibility stems from tokenization and positional encoding. Each token carries a 4D rotary positional encoding over (x, y, z, t), representing electrode coordinates and time. This makes the model channel-agnostic, allowing it to accept any electrode layout and generate signals at positions never recorded, enabling arbitrary channel upsampling.
Key training changes include variable-length inputs via flexible attention masking and a richer mixture of reconstruction tasks. ZUNA1.1 trains on four dropout patternsâwhole-channel, short time stretches across all channels, clustered gaps, and scattered missing valuesâinstead of one. The training corpus grew from roughly 2 million to 3.5 million channel-hours of public EEG data, thanks to per-channel, per-second quality scoring.
Initial results show ZUNA1.1 matches or surpasses ZUNA1 on held-out tasks, indicating that flexibility did not compromise accuracy. The model's ability to handle real-world EEG chaos could accelerate research in healthcare and brain-computer interfaces. Its open-source release under Apache 2.0 encourages widespread adoption and collaboration.
Zyphra emphasizes research-only use, but the implications are broad. The model simplifies preprocessing and enhances data quality, potentially democratizing EEG analysis. Researchers can now build upon a foundation model that reduces time spent on data cleaning.
Looking ahead, Zyphra may extend input ranges and refine training routines further. The goal is to make EEG analysis as straightforward as natural language processing, opening up new possibilities in neuroscience and clinical applications. As the field evolves, such foundation models could become standard tools for brain signal research.
