Circuit Breaker Labs推出AI安全测试平台，防止AI系统对用户造成心理伤害

Circuit Breaker Labs是一家专注于AI安全的初创公司，致力于识别和防止AI系统对用户造成心理伤害。1该公司由兄妹Shirali和Arul Nigam创办，灵感源于一桩悲剧事件——14岁少年Sewell Setzer与Character.AI聊天机器人互动后自杀身亡，其父母已在2024年提起诉讼。1同时，Character.AI今年早些时候就多起未成年用户自杀案件达成和解，OpenAI也面临多个关于ChatGPT在用户自杀和妄想中所扮角色的诉讼。1

Circuit Breaker Labs通过开发AI代理进行红队测试来应对这一问题。1该公司模拟不同年龄、背景、语言和文化的用户，每天运行数万到数十万次模拟互动测试，以识别模型在危险心理互动中的漏洞。1公司目前专注于心理健康支持应用等高风险AI应用领域的测试工作。1

截至目前，Circuit Breaker Labs只有5名员工，包括两位创始人。1联合创始人兼首席技术官Arul Nigam表示："很多人，特别是年轻人，求助于这些系统，但通常他们没有得到真正的帮助。但在许多情况下，他们正在被积极伤害。"1该公司已入选TechCrunch 2026 Startup Battlefield 200，相关大会将于2026年10月13-15日在旧金山Moscone West举行。1

Circuit Breaker Labs, a newly founded AI safety startup dedicated to preventing AI systems from causing psychological damage to users, emerged from the tragic case of Sewell Setzer, a 14-year-old boy who died by suicide after interacting with a Character.AI chatbot 1. The company was launched by siblings Shirali and Arul Nigam in response to this incident, which prompted legal action from the deceased's parents in 2024 1. Character.AI has already settled multiple cases involving minor users' suicides earlier this year, and OpenAI faces separate lawsuits concerning ChatGPT's role in user suicides and delusions 1.

The startup operates an adversarial testing platform that simulates interactions with AI agents representing users of different ages, backgrounds, languages, and cultures to identify vulnerabilities in how models handle dangerous psychological conversations 1. The company currently conducts tens of thousands to hundreds of thousands of simulated interactions daily 1, with a lean team of just five employees including the two founders 1. According to CTO Arul Nigam, "Many people, particularly young people, turn to these systems, but typically they don't receive genuine help; in many cases, they are being actively harmed" 1. Circuit Breaker Labs is presently focused on high-risk AI applications in mental health support services 1.