独立系统研究者Aman Sachan发布了名为MAPLE的剪枝技术,成功将Gemma 4 26B-A4B混合专家(MoE)大语言模型的体积从9.92 GB大幅压缩至4.75 GB 1。这一优化使得该模型的有效参数量降至1.48 bits/param,并在仅配备6GB显存的消费级GPU上实现了100%内存执行 1。在此配置下,模型推理吞吐量达到45.4 Tokens/Second 1。
MAPLE的核心策略在于针对每层切除80个休眠专家,保留48个活跃专家流形,从而捕获了79.88%的激活质量 1。该技术专门针对Python编程、算法合成、数据结构以及正式STEM推理领域进行了优化,旨在保持上述场景下的推理能力 1。开发过程中解决了包括“Ghost Router”(代理嵌入失败)、“Cascaded Normalization Suppression”(级联归一化抑制)以及“DRY Sampler Interference”(干采样干扰)在内的多项关键技术挑战 1。目前,预构建的GGUF容器已托管于Hugging Face平台,文件名为AmSach/Gemma-4-26B-A4B-MAPLE-Sub5GB-GGUF 1。
Independent systems researcher Aman Sachan has released MAPLE, a pruning technique that significantly reduces the memory footprint of the Gemma 4 26B-A4B Mixture-of-Experts (MoE) large language model 1. The method compresses the model from its original size of 9.92 GB down to 4.75 GB, achieving an effective rate of 1.48 bits per parameter 1. This reduction allows the model to run entirely within VRAM on consumer-grade GPUs with only 6 GB of memory, delivering a throughput of 45.4 tokens per second 1.
The pruning strategy operates by removing 80 dormant experts in each layer while retaining a manifold of 48 active experts, which preserves 79.88% of the activation quality 1. Although the compression is aggressive, the resulting model maintains performance specifically optimized for Python programming, algorithm synthesis, data structures, and formal STEM reasoning tasks 1. The development process involved overcoming several technical hurdles, including issues identified as "Ghost Router" failures where proxy embeddings failed, cascaded normalization suppression, and interference from DRY samplers 1. A pre-built GGUF container for this compressed model is available for download on Hugging Face under the repository AmSach/Gemma-4-26B-A4B-MAPLE-Sub5GB-GGUF 1.
评论
还没有评论,欢迎留下第一条。