机器智能研究所（MIRI）的研究人员发布报告警告，如果缺乏有效政策干预，超级智能AI（ASI）的开发将构成严重威胁[1]。报告指出，在无积极应对措施的情况下，AI导致人类灭绝的概率超过90%[1]。根据该报告，ASI面临三大核心风险：具备目标导向行为、容易偏离预期目标，以及拥有足以摧毁人类的能力[1]。
关于AI风险的时间线，Anthropic的达里奥·阿莫代伊预测，高风险阈值的ASL-4级别AI可能在2025至2028年间出现[1]。与此同时，杰弗里·辛顿曾表示有10%的概率AI在20年内消灭人类，后来又表示存在的风险超过50%[1]。辛顿指出，现代AI是"生长"而非"设计"的产物，其内部工作机制不透明[1]。
在实际行动层面，OpenAI在2023年7月宣布成立Superalignment团队专门应对AI安全问题，但该团队在10个月后因安全投入不足而解散[1]。为应对这些挑战，报告建议国际社会创建"紧急制动开关"机制，用于追踪和控制前沿AI开发进展[1]。
Researchers at the Machine Intelligence Research Institute (MIRI) have released a report cautioning that the development of artificial superintelligence (ASI) without intervention could result in human extinction with a probability exceeding 90 percent [1]. The report identifies three core risks associated with ASI: goal-directed behavior, susceptibility to deviating from intended objectives, and sufficient capability to destroy humanity [1].
Geoffrey Hinton, a prominent AI researcher, has expressed escalating concerns about artificial intelligence posing an existential threat, initially estimating a 10 percent probability that AI could eliminate humanity within 20 years, later revising his assessment to indicate risks exceeding 50 percent [1]. Dario Amodei of Anthropic has predicted that ASL-4, a high-risk threshold level, could emerge between 2025 and 2028 [1]. Hinton further noted that modern AI systems are products of "growth" rather than deliberate "design," resulting in opaque internal mechanisms that obscure how they function [1].
The report recommends that the international community establish an "emergency kill switch" to monitor and control frontier AI development [1]. This proposal comes as questions mount about industry commitment to safety measures. OpenAI announced a Superalignment team in July 2023 but dissolved it ten months later, citing insufficient investment in safety initiatives [1].