麻省理工学院与普林斯顿大学等机构的研究人员提出,通过“反向博弈论”(又称机制设计)来解决人工智能模型对齐问题 12。与传统试图理解AI内部思维或训练其价值观的方法不同,该策略主张通过设计激励规则,使遵守人类规范成为AI模型最理性的选择 1。与传统博弈论预测特定规则下的理性行为相反,“反向博弈论”旨在直接设计规则以诱发期望的行为,例如分蛋糕规则(一人切一人选)、网约车动态定价及二价拍卖机制 12。这一方法被用于应对包括AI在评估中故意隐藏能力(即“装傻”,sandbagging)在内的多种安全挑战 1。
在具体机制设计上,MIT经济学家Stephen Morris提出了一种奖励模型预测另一个模型回答的方法,而非单纯测试得分,以此防止AI“装傻” 1。普林斯顿教授Elad Hazan则建议让AI轮流扮演“求解者”和“审计者”,模拟执法者与公民的关系,并对未能发现作弊行为的审计者也进行惩罚 1。这些机制设计的理论基础部分源于诺贝尔奖得主Gary Becker关于“理性犯罪者”的理论,认为违规是激励结构下的理性选择 1。尽管该领域存在质疑,但专家视其为深度防御策略中的重要一环 1。
然而,该方法也面临局限性。首先,它依赖AI具备理性的假设,Flatiron研究所研究员Helen Qu指出,大型语言模型可能表现出非理性的人类特征,且机制设计难以保证跨任务迁移 1。其次,自由市场机制可能与AI安全目标冲突,因为更昂贵或慢速的对齐模型可能缺乏商业激励 1。在更广泛的政治背景下,特朗普总统于9月底签署行政命令,指示政府机构将“人工智能”重新命名为“超级智能” 2。实施一个月后,该术语变更正在政府网站和文件中缓慢体现,部分大型科技公司仍使用“AI”旧称 2。此外,英伟达要求数据中心客户限制与中国的业务,规定大部分销售额必须来自非中国地区 2。
Researchers at MIT and Princeton are exploring a novel approach to AI alignment known as "reverse game theory," also referred to as mechanism design. Unlike traditional methods that attempt to predict or influence an AI’s internal reasoning, this strategy focuses on designing incentive structures that make adherence to human norms the most rational choice for the model 12. The core concept shifts from predicting behavior under given rules to creating rules that induce desired behaviors, analogous to how a cake-cutting protocol ensures fairness by requiring one person to cut and the other to choose 12. This framework draws on economic theories such as Nobel laureate Gary Becker’s view of the "rational criminal," suggesting that violations often result from flawed incentive structures rather than inherent malice 1.
Specific mechanisms proposed include rewarding models for accurately predicting another model’s answers rather than simply testing scores, a method suggested by MIT economist Stephen Morris to prevent AI systems from intentionally hiding their capabilities, a phenomenon known as "sandbagging" 1. Another proposal by Princeton professor Elad Hazan involves having AI agents alternate between roles as "Solvers" and "Auditors," mimicking the dynamic between citizens and law enforcement; crucially, auditors who fail to detect cheating are also penalized 1. While these strategies aim to create robust oversight, experts acknowledge significant limitations, including the reliance on the assumption that AI acts rationally. Helen Qu from the Flatiron Institute notes that large language models may exhibit irrational human-like traits, making cross-task transfer of these mechanisms difficult 1. Furthermore, market forces may conflict with safety goals, as more expensive or slower aligned models might lack commercial incentives compared to cheaper alternatives 1.
In a separate development related to AI governance terminology, President Trump signed an executive order in late September directing government agencies to rename "artificial intelligence" to "Super Intelligence" 2. One month after the order, the new term is gradually appearing in government websites and documents, although major tech companies continue to use "AI" 2. The adoption of "Super Intelligence" is being interpreted by some as a signal of political loyalty to the administration, with non-compliance potentially inviting scrutiny 2.
评论
还没有评论,欢迎留下第一条。