研究团队于2026年10月9日发布论文,提出了一种名为Next-Concept Modeling(NCM)的新框架,旨在解决扩散语言模型(DLMs)在减少采样步骤时性能下降及计算效率低的问题 1。该框架采用两阶段由粗到细的策略,通过多尺度残差VQ-VAE将文本编码为层次化离散代码,并利用Transformer逐层并行预测这些代码 1。
实验结果显示,NCM在保持或提升生成质量的同时显著提高了速度,在多个基准测试中均优于现有最强DLM基线 1。具体而言,在LM1B数据集上,NCM实现了更好的生成困惑度且速度快21倍;在OpenWebText数据集上,其表现与最强DLM相当但速度快10倍;而在XSum摘要任务中,NCM匹配了最强DLM的ROUGE-L分数,速度却快31倍 1。此外,在数独推理任务的所有难度级别上,NCM不仅优于自回归和扩散基线,还减少了函数评估次数 1。
A research team has proposed a new framework called Next-Concept Modeling (NCM) to address the performance degradation and low computational efficiency of diffusion language models (DLMs) when sampling steps are reduced 1. The NCM framework employs a two-stage coarse-to-fine strategy that utilizes multi-scale residual VQ-VAE encoding to convert text into hierarchical discrete codes 1. These codes are then predicted in parallel by a Transformer model 1.
Experimental results indicate that NCM maintains or improves generation quality while achieving significant speedups compared to the strongest DLM baselines 1. On the LM1B benchmark, NCM achieved better generation perplexity and was 21 times faster than the top DLM baseline 1. For the OpenWebText dataset, NCM performed comparably to the strongest DLM but operated 10 times faster 1. In XSum tasks, NCM matched the ROUGE-L scores of the best DLM baseline while being 31 times faster 1.
In addition to standard generation benchmarks, NCM demonstrated superior reasoning capabilities in Sudoku tasks 1. The framework outperformed both autoregressive and diffusion baselines across all difficulty levels, requiring fewer function evaluations 1. The study was submitted on October 9, 2026 1.
评论
还没有评论,欢迎留下第一条。