Swiftlet是一款基于Swift和Metal的运行时环境，使Apple设备能够运行Qwen3系列大规模语言模型[1]。该项目实现了80B参数模型在Mac上仅占用4.3GB内存运行的目标，以及35B模型在iPhone上用约2.5GB内存运行的成果[1]，标志着在手机设备上原生运行这一规模模型的首次突破[1]。
项目通过专家流式传输和有界缓存机制优化内存占用，在任何时刻仅保持约3B参数的稀疏激活在内存中，具体而言，80B模型每层的512个专家中每次仅有10个被激活[1]。iPhone上35B模型的推理速度约为每秒1个token[1]。Swiftlet提供OpenAI兼容的API服务器接口[1]，已有Priv AI应用集成该核心技术并上架App Store[1]。该项目采用Apache 2.0许可证开源发布，核心代码约10000行Swift和Metal[1]。
Swiftlet, a Swift and Metal-based runtime, enables users to run the Qwen3 series of large language models natively on Apple devices with dramatically reduced memory requirements.[1] The project achieves a significant milestone: an 80-billion-parameter model runs on Mac with only 4.3 GB of RAM, while a 35-billion-parameter version operates on iPhone using approximately 2.5 GB of RAM at roughly 1 token per second.[1] This marks the first time models of this scale have been deployed natively on mobile devices.
The efficiency breakthrough stems from the runtime's use of expert token streaming and bounded caching mechanisms, which keep only the sparsely activated parameters in memory at any given time.[1] For the 80B model, each token activates only around 3 billion parameters, as the system routes to 10 experts out of 512 available per layer.[1] The project provides OpenAI-compatible API server interfaces, allowing developers to integrate these models into their applications.[1] An application called Priv AI, which incorporates Swiftlet Core, is already available on the App Store.[1] The Swiftlet runtime is open source under the Apache 2.0 license and comprises approximately 10,000 lines of Swift and Metal code.[1]