humans& ai团队于2026年发布了名为“GPU加速权重传送”(GPU-Accelerated Weight Teleportation)的技术方案,旨在解决大规模强化学习中训练器与采样器之间权重同步的性能瓶颈 1。该方案通过结合Delta权重更新、双重压缩算法以及NVIDIA Blackwell架构原生的解压引擎(DE),实现了训练侧与推理侧的高效并行处理 1。实验数据显示,在Ampersand生产配置下,该技术将采样器暂停时间降低至0.23秒以下,端到端权重同步时间控制在4秒以内,其性能优于传统的集群内P2P RDMA传输 1。
在具体技术实现上,该方案采用NVFP4 GLM-5.3模型作为负载代理,并利用Triton kernels在GPU HBM中原位执行bitwise XOR更新权重,使硬件解压与SM计算完全重叠 1。为了平衡GPU友好性与压缩比,团队采用了Snappy加Zstd的双重压缩策略;对比显示,LZ4基线产生5.36 GB载荷,CPU Zstd基线为0.876 GB,而最终优化方案进一步缩小了传输体积并消除了磁盘往返开销 1。目前,相关代码已分布在Miles项目(负责编排与分布式压缩)和SGLang项目(提供异构解压管道与API)中开源发布 1。
The humans& ai team, led by Ziang Li, has released a technique named "GPU-accelerated weight teleportation" designed to resolve the bottleneck of weight synchronization between trainers and samplers in large-scale reinforcement learning 1. Published on January 1, 2026, this method combines Delta weight updates, dual compression algorithms (Snappy and Zstd), and the native Decompression Engine (DE) of the NVIDIA Blackwell architecture to achieve efficient parallel processing on both the training and inference sides 1. The implementation is available through open-source projects Miles, which handles orchestration and distributed compression, and SGLang, which manages the heterogeneous decompression pipeline and API 1.
The technology utilizes the NVFP4 GLM-5.3 model as a load proxy and executes bitwise XOR weight updates directly within GPU HBM via Triton kernels 1. This approach allows hardware-based DE decompression to fully overlap with streaming multiprocessor (SM) computations, thereby eliminating disk round-trip overheads 1. Regarding compression strategies, an LZ4 baseline generated a payload of 5.36 GB, while a CPU Zstd baseline resulted in 0.876 GB; the final optimized Snappy plus Zstd dual compression strategy further reduced transmission volume while balancing GPU compatibility and compression ratio 1.
In Ampersand production configurations, the solution reduces sampler pause time to under 0.23 seconds and cuts end-to-end weight synchronization time to less than 4 seconds 1. These metrics demonstrate that the proposed method outperforms traditional intra-cluster P2P RDMA transmissions 1.
评论
还没有评论,欢迎留下第一条。