Goodfire研究员Nick Cammarata发表26000字博文，主张大语言模型和神经网络比此前认为的更容易理解和控制1。他声称神经网络内部具有可解释的逻辑结构而非完全混乱的纠缠，挑战了学界长期以来对这一领域的基本认知1。通过重新审视谷歌2014年开发的Inception v1模型，Cammarata发现看似无关的概念（如猫和汽车）实际上因共同的视觉特征被存储在同一神经元中1。
Cammarata表示自己投入了大量精力从事这项研究，称"花费大约10小时一天，一周七天，去年大部分时间只看神经元"1。他对理解大型语言模型的前景持乐观态度，声称"如果我有100,000年去逐个电路查看前沿语言模型，我有相当把握能得出很好的理解"1。这一主张在AI学界引发热烈讨论，一些研究人员表示受到启发1。
然而，该观点也遭到质疑。斯坦福大学等研究机构的学者提出批评，其中Jacob Hilton指出研究者对顶级大语言模型的理解不足0.01%1，暗示Cammarata的主张过于乐观，人类对这些系统的真正理解程度仍然极其有限。
Nick Cammarata, a researcher at Goodfire, has published a 26,000-word essay asserting that large language models may be far more interpretable and controllable than previously believed, challenging the prevailing view that neural networks are fundamentally chaotic and entangled systems.1 The claim has sparked intense debate across the artificial intelligence community, though it has also drawn criticism from leading research institutions.
Cammarata contends that neural networks possess comprehensible logical structures rather than incomprehensible complexity. In re-examining the Inception v1 model, developed by Google in 2014, he discovered that seemingly unrelated concepts—such as cats and automobiles—are actually stored in the same neuron due to shared visual features like V-shapes.1 To reach these conclusions, Cammarata states he "spent like, I don't know, 10 hours a day, seven days a week, for most of the last year just looking at neurons."1 He expressed cautious optimism about the possibility of understanding large language models, suggesting that "if I had 100,000 years to go through individual circuits of frontier language models, I have considerable confidence I could derive a very good understanding."1
The research has generated significant response within the AI research community. One researcher commented on X that "I feel like my understanding of what a neural net is has been catapulted into the future by five years or something."1 However, skeptics have emerged, with Jacob Hilton noting that researchers understand less than 0.01% of the inner workings of top-tier language models.1 These competing assessments underscore the ongoing uncertainty about the true interpretability of advanced neural systems.
评论
还没有评论，欢迎留下第一条。