人工智能实验室Anthropic在周一发布的修订版S-1A文件中警告,公司可能即将触及“递归自我改进”(RSI)的能力临界点 1。这一概念指代AI系统通过自我训练实现能力的急剧加速提升,被视为潜在超越人类控制的风险节点 1。尽管Anthropic明确指出目前尚无模型被认定跨过该门槛,但承认内部估算显示,未来不久可能满足其《负责任扩展政策》中关于“由AI归因的进展大幅加速”的定义 1。
针对利用AI辅助构建更强大模型所带来的巨大安全风险,Anthropic承诺在研发过程中保持人类监督 1。该公司考虑更新《负责任扩展政策》,以确保对前沿模型开发过程的控制权 1。回顾过往经验,Anthropic提到曾未能及时识别发展速度的显著加速,直到事后才察觉这一现象 1。
Artificial intelligence laboratory Anthropic issued a warning in its revised S-1A filing, released on Monday, that the company may be close to reaching the capability threshold for "Recursive Self-Improvement" (RSI) 1. RSI refers to the ability of AI systems to become proficient at self-training, leading to an abrupt acceleration in their capabilities 1. This scenario is viewed as a critical risk point where AI could surpass human control 1. The filing highlights the substantial safety risks associated with using AI to help build more powerful AI models 1.
Anthropic emphasized that no current model has been designated as exceeding the RSI threshold and pledged to maintain human oversight in its research and development 1. However, internal estimates indicate that the conditions defined by the company's Responsible Scaling Policy regarding "significant acceleration of progress attributed to AI" could be met in the near future 1. The firm acknowledged that it has already utilized AI to accelerate its own research 1. It also noted that past developments were not identified as significant accelerations until after they occurred 1. Consequently, Anthropic is considering updates to its Responsible Scaling Policy to maintain control over the development of frontier models 1.
评论
还没有评论,欢迎留下第一条。