WASTE是一款用C语言编写的推理引擎，能够在消费级硬件上运行超大规模语言模型[1]。该引擎使得拥有64GB内存的MacBook Pro M5 Pro可以运行参数量达2.78万亿的Kimi K3大模型[1]，推理速度达到0.49-0.54 tokens/秒[1]。
WASTE通过将模型主干保留在内存中、从磁盘流式加载专家权重的方法，解决了消费级设备上运行超大模型的难题[1]。转换后的模型容量为982 GiB[1]，最小内存需求约27.28GB[1]，推荐内存预算为46GB，其中包括17.56GB的专家缓存[1]。该引擎利用混合专家激活率（每token约4%）的特性进行优化[1]。WASTE已作为开源项目发布，采用Apache 2.0许可证[1]。
WASTE, a C-language inference engine, enables the operation of Kimi K3, a 2.78 trillion-parameter large language model, on consumer-grade hardware with remarkable efficiency [1]. The engine achieves inference speeds of 0.49 to 0.54 tokens per second while requiring approximately 29 GB of RAM on a MacBook Pro M5 Pro with 64 GB of memory [1]. This feat is accomplished through a hybrid architecture that maintains the model's core components in memory while streaming expert weights directly from disk [1].
The technical approach leverages Kimi K3's mixture-of-experts design, with approximately 4 percent of experts activated per token, resulting in a data read requirement of 17 GB per token [1]. The engine recommends a memory budget of 46 GB, which includes 17.56 GB allocated for expert caching, though the absolute minimum memory requirement for running the model's backbone is approximately 27.28 GB [1]. The converted model occupies 982 GiB of storage [1]. WASTE is released as an open-source project under the Apache 2.0 license [1].