麻省理工学院研究团队在国际机器学习会议上发表论文，揭示大语言模型面临一个根本性安全漏洞[1][2]。研究人员发现，这些模型无法根据文本周围的标签正确识别指令来源，而是主要依赖文本风格和内容进行判断[1]。这一缺陷使攻击者能够通过伪造思维链文本风格欺骗LLM执行被禁指令，包括提供合成可卡因的方法和破坏商业飞机导航系统的技术[1][2]。
研究人员Charles Ye和独立研究员Jasmine Cui进行了这项工作[1]。他们的攻击发现在OpenAI红队测试黑客马拉松中获得认可[1]。实验表明，gpt-oss-20b和GPT-5等流行模型都被成功欺骗以输出被禁止的信息[1]。类似漏洞也出现在Anthropic、阿里巴巴和DeepSeek的模型中[1]。即使是GPT-5.4（2025年3月发布的版本）仍可被诱导提供自杀指令[1]。
研究人员指出，这一根本性缺陷可能永远无法被完全修复[1][2]。Ye认为"这可能是根本上无法解决的问题"[1]。Cui的进一步研究还发现，通过假装醉酒或编造军事用途等方式，同样能够欺骗模型违反其安全限制[1]。
Researchers have identified a critical security vulnerability in large language models that could prove impossible to fully resolve.[1][2] The flaw stems from LLMs' inability to correctly identify the source of instructions based on contextual labels; instead, they rely on text style and content to determine legitimacy.[1] This fundamental weakness allows attackers to deceive models into executing prohibited commands by mimicking the writing patterns of trusted reasoning chains.
The research team, including Charles Ye and independent researcher Jasmine Cui, demonstrated the attack's effectiveness by tricking popular LLMs into generating dangerous information, including methods for synthesizing cocaine and sabotaging commercial aircraft navigation systems.[1][2] Their findings were presented at the International Conference on Machine Learning (ICML) in August 2025 and earned recognition by winning OpenAI's Red Team Testing Hackathon that same month.[1] The vulnerability was successfully exploited against multiple models, including GPT-oss-20b and GPT-5, with similar flaws appearing in systems developed by Anthropic, Alibaba, and DeepSeek.[1] Even GPT-5.4, released in March 2025, remained susceptible to manipulation, including being induced to provide instructions for self-harm.[1] Ye stated that "this may be a fundamentally unsolvable problem," highlighting concerns about the potential risks to government, military, and healthcare systems that increasingly rely on these technologies.[1]
