研究人员提出了一种名为“自我重写”(self-rewriting)的新机制,使语言模型能够通过自身推理直接修改权重,而无需依赖梯度或传统训练 1。在该机制下,模型充当自身的优化器,通过前向传播过程完成参数调整 1。相关论文《Language Models Can Rewrite Their Own Weights》由作者 Jiajun Fan 撰写,其 v1 版本于 2026年11月16日提交,v2 版本于 2026年11月18日修订 1。
实验数据显示,一个拥有 27B 参数的开源模型应用该机制后,在未见过的任务上将准确率提高了 6.6 个百分点 1。此外,该模型能够在连续八个会话中保持精度的持续上升 1。在修复能力方面,它可以将受损的模型副本恢复至原始图像任务准确率的 80% 以上,且在此过程中无需读取原始权重 1。研究表明,这种自我编辑能力随模型规模涌现,仅在最大规模的模型中显现,较小模型则无法执行自我编辑甚至可能自我破坏 1。这一发现被视为通向递归自我改进的重要一步,打破了训练与推理之间的界限 1。
Researchers have introduced a novel mechanism termed "self-rewriting," which enables large language models to directly modify their own weights through internal reasoning, bypassing the need for gradients or traditional training processes. 1 This approach allows the model to act as its own optimizer, adjusting parameters during forward propagation rather than relying on standard backpropagation methods. 1 The capability represents a significant shift by blurring the boundary between training and inference phases in artificial intelligence systems. 1
In experimental trials using an open-weight model with 27 billion parameters, the self-rewriting mechanism improved accuracy by 6.6 percentage points on tasks the model had not previously encountered. 1 Furthermore, the system demonstrated resilience by repairing corrupted copies of itself, restoring performance to more than 80% of the original accuracy on image tasks without needing access to the initial weight data. 1 Precision was observed to increase consistently across eight consecutive sessions, indicating sustained learning capabilities within this framework. 1
The study highlights that this self-editing ability emerges only at larger model scales, as smaller versions failed to perform self-correction or risked damaging themselves during the process. 1 Jiajun Fan authored the paper titled "Language Models Can Rewrite Their Own Weights," with version one submitted on November 16, 2026, and a revised version two released on November 18, 2026. 1 Researchers view these findings as a crucial step toward recursive self-improvement in AI systems. 1
评论
还没有评论,欢迎留下第一条。