扩散语言模型的构建与推理加速突破

扩散语言模型正在成为推理阶段的关键技术进展。1掩码扩散语言模型（MDLM）本质上是具有随机掩码率的生成式BERT，通过平行生成能够实现显著的推理加速。1截至2026年，已有多个产业级扩散模型发布，在质量上与自回归模型相当。1

当前开源与商业模型均展示了扩散方法的优势。Mercury 2在标准GPU上达到超过1,000令牌/秒/用户的性能，速度为Claude Haiku和Gemini Flash的5-10倍。1LLaDA规模为8B参数，采用MDLM骨干网络、块扩散和重掩码技术。1Nemotron Diffusion报告在保留99%质量的同时实现2-8倍的吞吐量提升。1Google的Gemma Diffusion也已推出。1

扩散模型支持迭代优化进行错误纠正，可实现5-10倍的平行生成加速。1业界认为扩散模型可能对推理阶段产生类似Transformer对预训练阶段的深远影响。1

Diffusion language models represent an emerging approach to generative AI that diverges from traditional autoregressive architectures. Masked diffusion language models (MDLM) function as generative BERT variants that employ random masking rates during training 1. By 2024, these models have achieved quality parity with autoregressive alternatives, with multiple production-grade implementations released by 2026 1.

The key advantage of diffusion models lies in their inference efficiency through parallel generation. Recent open-source and commercial releases—including LLaDA (8 billion parameters using MDLM backbone with block diffusion and re-masking techniques), Mercury 2, Gemma Diffusion, and Nemotron Diffusion—demonstrate substantial speedup gains 1. Mercury 2 achieves throughput exceeding 1,000 tokens per second per user on standard GPUs, delivering 5-10 times faster inference compared to Claude Haiku and Gemini Flash 1. Nemotron Diffusion similarly reports 2-8 times throughput improvements while preserving 99% of quality 1.

Beyond acceleration, diffusion models enable iterative refinement for error correction and support variable-length generation, suggesting they may become a breakthrough approach for the inference stage, analogous to the transformative impact of Transformer architecture on pretraining 1.