Artificial Analysis 于2026年10月27日发布了 Intelligence Index v5.0，这是自v4版本以来规模最大的一次升级 2。新版索引整合了知识工作、编码、科学和通用能力四大类别的9项具体评测任务，包括 AA-Briefcase v1.1, AutomationBench-AA, GDP.pdf, FrontierCode 2.0, Terminal-Bench 4.0, AA-Omniscience v1.1, HLE-Diamond, CritPt v2 (Hard Subset) 以及 Terminal-Bench-Science 0.1.0 12。该版本首次将60%的评估权重建立在私有数据集或保留答案上，旨在更真实地反映模型在现实世界中的使用情况 2。
在方法论与基准构成方面，v5.0 对原有体系进行了显著调整。索引由四个类别加权平均计算得出，每个类别各占25%的权重 1。主要变更包括新增 HLE-Diamond（占General类别10%）、Terminal-Bench-Science（占Science类别15%）和 FrontierCode 2.0（占Coding类别15%），同时移除了 GDPval-AA, SciCode, AA-LCR 和 Humanity's Last Exam 等旧基准 1。此外，GDP.pdf 的权重从 v4.3.2 版本的10%下调至5% 1。评分主要采用 pass@1 scoring（首次尝试正确率），非推理模型的设置温度为0，而推理模型为0.6，除非实验室另有建议；Agentic 任务则在 E2B sandbox 环境中运行 1。部分评测如 AA-Briefcase 采用 Elo 评分体系，其锚点设定涉及 GPT-5.5 (medium) at 1000 1。
最新的排名结果显示，Claude Opus 5.5 (max) 以58分位居榜首，领先于得分为57分的 GPT-6 Astra (max) 和得分为55分的 GPT-6.1 Sol (max) 2。开源模型与闭源模型之间的性能差距正在扩大，最佳开源模型 Kimi K3.1 Pro 得分为46分，落后榜首12分，而在 v4.3 时代这一差距仅为3分 2。成本差异也极为显著，GPT-6.1 Terra (max) 每任务成本为0.96美元，而 Claude Fable 5.1 (max) 高达13.04美元 2。各领域表现各有侧重，Claude Opus 5.5 在代码和知识领域领跑，而 GPT-6 Astra 则在科学和研究物理领域占据优势 2。伴随此次发布，Artificial Analysis 还推出了 Progress Index v1，用于追踪模型在统一尺度上的能力演变 2。
Artificial Analysis released the fifth major version of its Intelligence Index on October 27, 2026 12. This comprehensive benchmark evaluates language model capabilities across four categories: Knowledge Work, Coding, Science, and General aptitude 1. The update represents the most significant revision since version 4, integrating nine specific evaluation tasks and shifting 60% of the assessment weight to private datasets or questions with retained answers to better reflect real-world usage 2. The index is calculated as an equal-weighted average of the four categories, each contributing 25% 1.
The methodology for v5.0 incorporates several structural changes, including the addition of new benchmarks such as FrontierCode 2.0 (weighted at 15%) and Terminal-Bench-Science (weighted at 15%), alongside upgrades to existing tests like HLE-Diamond (weighted at 10%), AA-Omniscience v1.1, and CritPt v2 12. Conversely, the update removed previous components including GDPval-AA, SciCode, AA-LCR, and Humanity's Last Exam 1. Notably, the weight assigned to GDP.pdf decreased from 10% in the prior version to 5% in v5.0 1. Scoring primarily utilizes pass@1 metrics, with temperature settings adjusted to 0 for non-reasoning models and 0.6 for reasoning models unless laboratory specifications dictate otherwise 1. Agentic tasks are executed within an E2B sandbox environment, while certain evaluations like AA-Briefcase employ Elo rating systems anchored to specific reference models 1.
In the resulting performance rankings, Claude Opus 5.5 (max) secured the top position with a score of 58, followed closely by GPT-6 Astra (max) at 57 and GPT-6.1 Sol (max) at 55 2. A distinct disparity has emerged between proprietary and open-weight models; the highest-scoring open-source model, Kimi K3.1 Pro, recorded a score of 46, lagging behind the leader by 12 points compared to a 3-point gap in the v4.3 era 2. Performance strengths vary by domain, with Claude Opus 5.5 leading in code and knowledge tasks, while GPT-6 Astra excels in scientific and research physics evaluations 2. Additionally, operational costs differ significantly among top performers, with GPT-6.1 Terra (max) costing $0.96 per task versus $13.04 for Claude Fable 5.1 (max) 2. To complement the static ranking, Artificial Analysis simultaneously launched Progress Index v1, a tool intended to track the evolution of model capabilities over time on a unified scale 2.
评论
还没有评论，欢迎留下第一条。