Apple Research 发布了 MLX 框架的最新版本，展示了在 Mac Studio 上运行大规模语言模型的能力。1 该公司成功在搭载 M5 Ultra 芯片的单台 Mac Studio 上部署了参数规模达 816B 的 Muse Spark Open 模型，实现超过 36 tokens/秒的推理速度。1 相比搭载 M3 Ultra 芯片的设备，M5 Ultra 在首个 token 生成时间上快了 4.6 倍，内存带宽从 819 GB/s 提升至 1.2 TB/s。1
为了支持这一性能提升，Apple 利用了神经加速器和 Thunderbolt RDMA 技术实现多设备扩展。1 量化后的 Muse Spark Open 模型占用约 493GB 内存（NVFP4 格式），可在 Mac Studio 最大 512GB 统一内存配置下运行。1 在四台 Mac Studio 组成的配置中，系统能在 40.6 秒内处理包含 65,536 个 tokens 的上下文，展现了通过管道并行和张量并行技术扩展多设备推理的可行性。1
Apple 计划在 2026 年 4 月 23-27 日举办的 ICLR 大会上（地点为巴西里约热内卢）展示相关研究成果。1
Apple Research has unveiled the latest version of its MLX framework, showcasing the capability to run the Muse Spark Open large language model—featuring 816 billion parameters—on Mac Studio hardware 1. The demonstration highlights significant performance improvements driven by the M5 Ultra chip architecture, which features advanced neural accelerators and Thunderbolt RDMA technology for multi-device scaling 1.
A single Mac Studio with M5 Ultra achieves inference speeds exceeding 36 tokens per second 1. The time to first token (TTFT) metric shows the M5 Ultra performing 4.6 times faster than its M3 Ultra predecessor 1. This performance leap is supported by substantial improvements in memory bandwidth—the M5 Ultra delivers 1.2 terabytes per second compared to 819 gigabytes per second on the M3 Ultra, a 1.6x increase 1. The quantized Muse Spark Open model occupies approximately 493 gigabytes of memory in NVFP4 format, fitting within Mac Studio's maximum unified memory capacity of 512 gigabytes 1.
When scaling across four Mac Studio units, the time to first token extends to 40.6 seconds while handling 65,536 tokens of context, demonstrating pipeline parallelism and tensor parallelism capabilities across multiple devices 1. The M5 Ultra consumes approximately 300 watts compared to roughly 200 watts for the M3 Ultra 1. Apple plans to present this research at the ICLR 2026 conference scheduled for April 23–27 in Rio de Janeiro, Brazil 1.
评论
还没有评论，欢迎留下第一条。