论文《Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights》提出了“权重空间编程”(Weight-Space Programming)这一核心概念,指大语言模型无需执行代码或训练循环,即可直接分析并修改神经网络的原始浮点数权重 1。该研究指出,前沿模型如 GPT-6 Astra 具备从零基础编写超越 PPO 训练一亿步的游戏网络的能力;具体数据显示,Astra 通过仅编辑 0.016% 的权重,将 MATH-500 基准测试分数从 7.9% 提升至 48.1%,并将 IFEval 分数提高三倍 1。
在此背景下,文章基于 Simon Willison 的实验,验证了开源模型 Qwen 3.8 27B(大小约 17GB,通过 LM Studio 在本地笔记本运行)仅读取原始浮点数权重来推断神经网络训练目标的能力 1。实验结果显示,该模型正确识别出多款经典电子游戏的策略网络:Breakout 游戏用时 52分25秒,置信度达 88%;Pong 游戏用时 1小时4分54秒,置信度为 75%;Pac-Man 游戏用时 46分12秒,置信度为 72% 1。然而,在尝试识别较为冷门的 Freeway 游戏时,模型未能给出正确答案,而是将其误判为简化版的 Pac-Man 1。
针对上述发现,社区中存在争议 1。Hacker News 用户 Matthew Honnibal 批评该论文混淆了“无工具理解权重”与“有代码辅助更新权重”这两个话题 1。另一用户 Sean Goedecke 则指出,Qwen3.5-0.8B 模型数学能力的提升过程,其操作步骤与人类研究员的行为具有相似性 1。
A recent experiment verified that the open-source model Qwen 3.8 27B, running locally on a laptop, can infer neural network training objectives by reading raw floating-point weights 1. This capability aligns with "Weight-Space Programming," defined as large language models analyzing and modifying model weights without executing code or traditional training loops 1. The findings are based on the paper "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights" 1.
The 17GB Qwen 3.8 27B model was executed via LM Studio and successfully identified strategy networks for three classic games 1. It correctly recognized Breakout with 88% confidence in 52 minutes and 25 seconds, Pong with 75% confidence in 1 hour and 4 minutes and 54 seconds, and Pac-Man with 72% confidence in 46 minutes and 12 seconds 1. However, the system failed to accurately identify the less common game Freeway, mistakenly classifying it as a simplified version of Pac-Man 1.
The research discusses controversial findings regarding frontier models like GPT-6 Astra, which reportedly can write game networks from scratch that surpass those trained via Proximal Policy Optimization (PPO) over 100 million steps 1. Specific data indicates Astra improved its MATH-500 score from 7.9% to 48.1% and tripled its IFEval score by editing only 0.016% of its weights 1. Matthew Honnibal argued that the paper conflates understanding weights without tools with updating weights assisted by code 1. Additionally, Sean Goedecke noted that the mathematical capability improvements in Qwen3.5-0.8B mirror the operational steps taken by human researchers 1.
评论
还没有评论,欢迎留下第一条。