A new open-source runtime called Swiftlet claims to run an 80-billion-parameter Qwen model on a Mac with a peak memory footprint of about 4.3GB. The project, published under the Apache 2.0 license, targets the Qwen3-Next and Qwen3.5/3.6 MoE hybrid model family and also supports iPhones.
The key technical idea is expert streaming. Instead of loading the full model into memory, Swiftlet keeps only the small dense core resident and pulls routed mixture-of-experts weights from storage on demand. According to the project’s GitHub repository, the 80B model needs 42GB of disk space but peaks at roughly 4.3GB of RAM during decoding.
What Was Announced
Swiftlet is a Swift + Metal runtime, not a wrapper around an existing inference engine. The repository describes it as a from-scratch implementation of roughly 10,000 lines of Swift and Metal, built against mlx-lm reference implementations.
The headline numbers, as reported by the project:
Read them with some care. These are the project’s own measurements.
| Model | Disk | Peak RAM | Decode speed (M5 Mac) |
|---|---|---|---|
| Qwen3.6-35B-A3B, 4-bit | 18 GB | 2.6 GB | 7 to 11 tok/s |
| Qwen3-Next-80B-A3B, 4-bit | 42 GB | 4.3 GB | 4.5 to 5 tok/s |
The 35B model also runs on an iPhone 17 in about 2.5GB of RAM, at roughly 1 tok/s today. The project credits ANEMLL, which demonstrated a 397B MoE model streaming on an iPhone 17 Pro as a proof of concept earlier in 2026.
Why the Memory Footprint Is Small
The mechanism matters more than the headline number. These MoE models activate only about 3 billion of their parameters per token. Each layer routes every token to 10 of 512 experts on the 80B model, or 8 of 256 on the 35B.
Swiftlet keeps the dense weights resident in memory, including attention, DeltaNet projections, routers, shared experts, and embeddings. That is about 1.3GB for the 35B model and 2.5GB for the 80B at 4-bit quantization. The routed experts are repacked into fixed-stride blobs inside a .qpack container, so fetching one expert is exactly one pread from SSD, with no mmap and no page-cache thrash. A bounded cache with LFU plus recency eviction keeps hot experts in memory.
Another architectural advantage: 75 percent of the layers use Gated DeltaNet linear attention with a fixed-size recurrent state, so there is no growing KV cache for those layers at any context length.
No KV cache growth matters. It keeps memory flat no matter how long the conversation gets.
The Honest Caveats
The project itself is candid about limitations. Only about 3B parameters are active per token, which means these models chat and write like large models but recall facts like small ones. A 4.3GB 80B model is not equivalent to a frontier dense model.
Decode speed is modest. The 80B model produces 4.5 to 5 tokens per second on an M5 Mac, and the iPhone experience is about 1 token per second, which is not practical for interactive chat yet. The repository notes the decode loop is currently dispatch-bound rather than IO-bound, so kernel speed has clear headroom.
The memory savings also come with a disk cost. The 80B container requires 42GB of free SSD space, and the 35B needs 18GB.
What It Means
The significance of Swiftlet is that it attacks the same problem as AirLLM, which we covered separately: running frontier-class models on hardware people already own. AirLLM used layer-by-layer inference on commodity GPUs; Swiftlet uses expert streaming on Apple Silicon. Both directions point to the same conclusion, local inference is no longer limited to small models.
The project is early stage.
It is one repository, built with Claude Code, with a small team behind it. The correctness story is strong on paper, every layer of the forward pass is validated against mlx-lm references, and containers are byte-verifiable against source checkpoints. But real-world adoption will depend on speed improvements and on whether the Apple developer community picks it up.
That said, the trajectory is worth watching. If expert streaming matures, the hardware bar for running large MoE models locally falls from a workstation to a laptop, and eventually to a phone.
Related Reads
- [AirLLM: Running 70B Models on 4GB of VRAM](https://getaibest.com/airllm-70b-models-4gb-vram-local-ai/)
- [Qwen3.8: The 2.4T Parameter Open-Source Release](https://getaibest.com/qwen3-8-open-source-caught-up-to-claude/)
- [DeepSeek V4 Flash Full Release](https://getaibest.com/deepseek-v4-flash-full-release-agent/)