NeuraCryption Infrastructure Research Lab发布并开源了名为SPECTRA的大语言模型压缩架构,旨在解决消费级GPU上的内存墙瓶颈 1。该框架通过免校准的拓扑显著性提取和Group-32微缩放技术,实现约3倍的无损压缩比 1。具体而言,其压缩比率在2.80x至3.31x之间,接近无损 1。
在性能表现方面,SPECTRA可在低于50 MB RAM的主机缓冲及4GB VRAM GPU上运行部分模型 1。测试数据显示,在NVIDIA GeForce RTX 3050 Ti Laptop GPU (4GB VRAM)上,70亿参数模型的生成速度达到82.53 tokens/s 1。此外,该架构支持Transformer、ConvNet及Mamba-2架构,兼容NVIDIA CUDA和AMD ROCm平台 1。在精度保留方面,ARC-Challenge准确率从FP16基线的34.73%提升至35.58%,语义相似度>0.996 1。核心算法包括Group-32微缩放、免校准AST通道敏感性分析以及寄存器融合Triton内核 1。相关资源已发布,GitHub仓库地址为 https://github.com/NeuraCryption/SPECTRA,Zenodo DOI为 10.5281/zenodo.24058202 1。
NeuraCryption Infrastructure Research Lab has released SPECTRA, an open-source architecture designed to compress large language models by approximately three times with near-lossless accuracy. 1 The framework targets the memory bottleneck commonly faced by consumer-grade graphics processing units, enabling efficient model deployment on hardware with limited resources. 1 By utilizing a calibration-free approach combined with Group-32 micro-scaling technology, SPECTRA achieves compression ratios ranging from 2.80x to 3.31x while maintaining high inference precision. 1
The system demonstrates significant performance gains on constrained devices, such as the NVIDIA GeForce RTX 3050 Ti Laptop GPU equipped with only 4GB of VRAM. 1 On this hardware, SPECTRA enables 7-billion parameter models to generate text at a speed of 82.53 tokens per second. 1 Furthermore, the architecture can operate using host buffers of less than 50 MB of RAM for certain model configurations. 1 In terms of quality preservation, tests on the ARC-Challenge dataset showed an accuracy improvement from a 34.73% baseline in FP16 to 35.58%, alongside a semantic similarity score exceeding 0.996. 1
SPECTRA is engineered for broad compatibility, supporting Transformer, ConvNet, and Mamba-2 architectures across both NVIDIA CUDA and AMD ROCm environments. 1 Core algorithmic innovations include AST channel sensitivity analysis without calibration requirements and register-fused Triton kernels. 1 The project’s source code is available via GitHub at https://github.com/NeuraCryption/SPECTRA, and its associated documentation includes a Zenodo DOI of 10.5281/zenodo.24058202. 1 Although cited with a 2026 publication year in the paper’s references, the release is presented as a recent technological advancement. 1
评论
还没有评论,欢迎留下第一条。