Surge AI发布了名为DAYJOB: Tax的新基准测试，旨在评估AI代理在处理50项专业税务任务时的能力 1。该测试涵盖个人/家庭、小企业、合伙企业、房地产/投资、遗产/信托、国际税、公司交易及税务审查/争议共8个税务实践领域 1。与传统基准如TaxBench不同，DAYJOB: Tax强调“发现客户未提出的问题”和“处理混乱记录”的能力，重点考察AI在面对不完整指令和冲突证据时，能否展现出类似人类税务专家的专业判断力 1。
该基准包含1,789个工作区文件，格式包括PDF、Excel和Word等；每个任务的文件数量中位数为30个，最多为123个 1。初步结果显示，表现最佳的模型Claude Sonnet 5.5的通过率为10.8%（基于95%评分标准阈值），其他模型如Claude Fable 5.1和Claude Opus 5.5的通过率分别为9.6%和9.2% 1。这一数据表明，AI在承担端到端专业责任方面仍面临巨大挑战 1。
以Anna的公寓出售案例为例，多数AI未能识别出2027年8月后可能失去税收豁免资格的截止日期，这体现了其在识别客户未提及关键信息方面的局限性 1。
Surge AI has released a new benchmark named DAYJOB: Tax, designed to evaluate the capabilities of AI agents on 50 professional tax assignments 1. Unlike traditional benchmarks such as TaxBench, DAYJOB emphasizes the ability to discover issues not raised by clients and to handle messy records, rather than focusing solely on calculation and knowledge retrieval 1. The assessment tests whether AI can demonstrate professional judgment similar to human tax experts when facing incomplete instructions and conflicting evidence 1.
The benchmark consists of 50 environments covering eight areas of tax practice: personal/family, small business, partnerships, real estate/investments, estates/trusts, international tax, corporate transactions, and tax review/disputes 1. These tasks include 1,789 workspace files, with a median of 30 files per task and a maximum of 123, in formats such as PDF, Excel, and Word 1.
Initial results show that even frontier models struggle to assume end-to-end professional responsibility. Claude Sonnet 5.5 achieved the highest pass rate of 10.8% based on a 95% scoring threshold 1. Other models scored lower, with Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2% 1. In a key case involving a client named Anna selling an apartment, most AI agents failed to identify a critical deadline after August 2027, which would result in the loss of tax exemption eligibility 1.
评论
还没有评论，欢迎留下第一条。