- Published on
Retrospectively Reverse-Engineering Apple's Neural Engine
- Authors
- Name
- DeepHorus
Apple's Neural Engine (ANE), once a cornerstone of the company's on-device AI ambitions, is facing an uncertain future as the tech giant shifts its focus toward GPU-based transformer workloads. In a detailed retrospective analysis published on August 10, 2026, independent researcher Eileen Yoon revealed the internal architecture of the M1's ANE and argued that the standalone NPU is nearing its end, citing Apple's M5 chip (2025) as the turning point. Yoon, who previously worked on a reverse-engineered Linux driver for the ANE, stopped that project three years ago after concluding that the hardware's specialized design limited its usefulness for general-purpose acceleration. Now, she has returned to map the ANE's full architecture—compute, datapath, scheduler, memory, and execution model—to understand the design assumptions Apple made when it first introduced the ANE in the A11 Bionic chip in 2017.
The ANE's compute cores, which Yoon describes as the least interesting part, consist of 16 parallel cores, each containing 128 FP16 (or 256 INT8) multiply-accumulate (MAC) lanes, totaling 2,048 parallel MAC lanes across the chip. These lanes perform dense tensor reductions, executing dot products that are fundamental to both convolutional neural networks (CNNs) and transformer attention mechanisms. However, Yoon emphasizes that the MAC units themselves are not what made the ANE specialized; rather, it was the dataflow surrounding them—when and where inputs and outputs enter, stay, and move—that optimized for the predictable reuse patterns of 2017-era CNN models. This dataflow efficiency allowed the ANE to run image-processing workloads on phones, but it also made the architecture too opinionated for broader applications.
The limitations of this design became apparent as AI workloads shifted from CNNs to transformer-based models, particularly with autoregressive decoding, which breaks the predictable reuse patterns the ANE was built to exploit. Yoon notes that even macOS only regularly uses the ANE for generating upsampled preview images in Finder, highlighting its narrow utility. The M5's decision to fold ANE cores into the GPU cores, announced in 2025, officially signals the beginning of the end for the standalone NPU, as Apple pivots to GPUs for LLM performance. Yoon had anticipated this move, but it still marks a significant shift in Apple's hardware strategy.
The implications of this transition are far-reaching for the AI hardware industry. Yoon's reverse-engineering work provides a rare glimpse into Apple's internal design choices, revealing how the company bet on CNN-era workloads and is now adapting to the transformer era. The M5's integration of ANE functionality into the GPU suggests that Apple sees GPUs as more flexible and better suited for modern AI tasks, potentially rendering dedicated NPUs obsolete. This could influence other chipmakers, who may reconsider their own NPU investments in favor of unified architectures.
Reactions from the developer community have been mixed, with some praising Yoon's thorough technical analysis and others lamenting the ANE's untapped potential. The open-source driver project she abandoned remains available on GitHub, serving as a reference for those interested in the ANE's internals. Looking ahead, Yoon's work suggests that the future of on-device AI lies in more adaptable hardware, with GPUs taking on a central role. As Apple continues to refine its M-series chips, the lessons from the ANE's rise and fall will likely inform next-generation designs, though the standalone NPU's days appear numbered.
