Hugging Face首席政策官Irene Solaiman在TechCrunch Disrupt大会上指出，当前AI模型的评估体系存在根本性缺陷1。她表示，AI基准测试方法"正在不断演进，但我们还不知道如何有意义地将它们相互比较"1。Solaiman强调，虽然业界广泛推崇嵌入式第三方评估者作为替代政府监管的方案，但这一做法的有效性和中立性仍存重大疑问1。
Solaiman指出，现有评估体系缺乏可复现性和标准化1。她表示"代理评估实在太难了，代价也很大"1，并指出"信任是一个权重很大的词"1。为了使第三方评估者真正发挥作用，行业需要建立更多的标准化措施来确定有效评估者的资格1。此外，Anthropic聘请Accenture作为首个嵌入式评估者的决定也引发了业界批评1。
Irene Solaiman, chief policy officer at Hugging Face, cautioned that embedded third-party evaluators of AI models are not a straightforward solution to regulatory oversight, speaking at the TechCrunch Disrupt conference.1 Current benchmarking methodologies for AI systems remain underdeveloped, lacking reproducibility and standardization across the industry.1 Solaiman noted that while embedded evaluators have gained traction as an alternative to government regulation, questions persist about their effectiveness and neutrality.1
The limitations of existing assessment approaches are substantial.1 "They are an evolving science — where we don't know how to meaningfully compare them to each other," Solaiman stated, describing the current state of evaluation methods.1 She emphasized that agentic evaluations—assessments of autonomous AI agents—present particular challenges, as "agentic evals are really hard and really expensive."1 Trust in evaluation outcomes remains a critical concern; as Solaiman put it, "Trust is such a weighted term."1
Solaiman stressed that third-party evaluators require substantially more standardization to establish clear qualifications for what constitutes an effective evaluator.1 The approach drew scrutiny following Anthropic's engagement of Accenture as its first evaluator, raising questions about the feasibility and impartiality of embedded assessment frameworks.1
评论
还没有评论，欢迎留下第一条。