突破三值大语言模型1.58比特存储瓶颈

研究人员提出BITCOS算法，有效降低三值大语言模型的存储成本。1该算法分析了29个三值LLM模型中的权重分布特性，发现零权重占比最高达51.5%。1利用这一特征，BITCOS采用自适应分布式布局方案，在26个测试模型上的存储效率超过了传统五进制打包方法。1

传统五进制打包每权重需要1.625比特的存储空间，而BITCOS算法通过充分利用零权重分布实现了2-z比特每权重的成本（z为零密度），最稀疏的模型存储效率达到1.485比特每权重。1除了存储优化外，该算法还带来了显著的推理性能提升，在矩阵-向量乘法中最高实现1.28倍加速，CPU解码吞吐量提升至1.18倍，GPU解码提升至1.27倍。1该方案支持AVX-512、AVX2和Intel Xe2 GPU等多个计算平台。1

Researchers have introduced BITCOS, a novel algorithm designed to overcome storage efficiency limitations in ternary large language models.1 The algorithm addresses a critical bottleneck by analyzing the weight distribution across 29 ternary LLM models, revealing that zero weights account for as much as 51.5% of parameters.1 Traditional quinary packing achieves 1.625 bits per weight, but BITCOS employs an adaptive distributed layout strategy that achieves as low as 1.485 bits per weight on the sparsest models tested.1

The performance improvements extend beyond storage efficiency.1 Testing across 26 of the evaluated models showed BITCOS outperforming conventional packing methods, with matrix-vector multiplication accelerating by up to 1.28 times on certain platforms.1 End-to-end inference benchmarks demonstrated CPU decoding throughput improvements reaching 1.18 times and GPU decoding reaching 1.27 times.1 The algorithm supports multiple hardware architectures, including AVX-512, AVX2, and Intel Xe2 GPUs.1 The research was submitted on September 14, 2026.1