DiffusionGemma技术报告发布 离散扩散模型实现文本生成加速

研究人员推出了DiffusionGemma，一个采用离散扩散技术的开源语言模型1。该模型通过并行处理256个token块而非逐一解码的方式，实现了文本生成的显著加速1。

DiffusionGemma基于Gemma 4微调而得1，具有3.8B激活参数和25.2B总参数1。在单块NVIDIA H100 GPU上，该模型可达到约1,500个输出token/秒的生成速度1，远超同类自回归模型。每个forward pass可生成约20个token1。

该模型采用两阶段训练方式，结合监督微调与强化学习采样器蒸馏1。值得注意的是，训练耗用的token预算少于起始模型总训练token预算的10%1。研究团队在改进中保留了原模型对思考模式、多模态输入和长上下文的支持1。

Researchers have introduced DiffusionGemma, an open-source language model that employs discrete diffusion techniques to generate text significantly faster than conventional approaches 1. Rather than producing tokens sequentially through autoregressive decoding, the model processes 256-token blocks in parallel, enabling substantial acceleration in generation speed 1.

Built on fine-tuned Gemma 4, DiffusionGemma achieves approximately 1,500 output tokens per second on a single NVIDIA H100 GPU 1. The model maintains 3.8 billion active parameters within a total parameter count of 25.2 billion, and generates roughly 20 tokens per forward pass 1. Its training consumed less than 10% of the starting model's total training token budget 1. The development process involved two stages: supervised fine-tuning followed by reinforcement learning integrated with sampler distillation 1. The model preserves original capabilities for reasoning patterns, multimodal inputs, and extended context handling 1.