澳大利亚沃伦大学研究人员进行的最新研究表明，生成式AI在法律考试中的表现相比2023年已显著改善。1研究团队使用来自五家AI供应商的九个模型测试了两门法律课程的考试卷，结果显示AI的能力已足以帮助学生作弊。1
在刑法考试中，AI平均得分达到76.3%，超过了82.5%的学生成绩；在侵权法中，AI得分为66%，高于61%的学生水平。1这与2023年的研究形成鲜明对比——当年AI在刑法论文中的平均得分仅为52.5%，排在第22百分位，而2024年已提升至第82.5百分位。1在测试的18篇AI生成论文中，有7篇的成绩位于学生的90百分位及以上。1值得注意的是，2023年研究中AI在假设法律情景的批判性分析上表现出明显弱点，但这一弱点在2024年已大幅消失。1
面对AI能力的快速进步，研究建议大学采取三种互补的应对措施：保留不涉及AI的监督考试、设计需要与AI协作的评估任务，以及采用"接力"模式来平衡学生的独立思考能力与AI协作能力。1
Researchers at an Australian university have found that generative artificial intelligence systems are performing substantially better on law school examinations than they did just one year ago.1 Testing nine AI models from five different providers on two law courses, researchers discovered performance gains that raise concerns about academic integrity.1
In criminal law exams, AI achieved an average score of 76.3%, placing it in the 82.5th percentile of student performance, compared to an average student score of 59.6%.1 In tort law, AI scored 66%, surpassing approximately 61% of students.1 This represents a dramatic shift from 2023 testing, when the same criminal law assessment yielded an average AI score of just 52.5%, ranking only in the 22nd percentile.1 Of the 18 AI-generated legal papers evaluated in the 2024 study, seven ranked at or above the 90th percentile of student grades.1 Notably, AI has also closed a significant analytical gap: while models struggled with critical analysis of hypothetical legal scenarios in 2023, this weakness has largely disappeared in the updated testing.1
To address these capabilities, the research team recommends universities adopt three complementary approaches.1 These include maintaining some examinations without AI access, designing assessments that require students to collaborate with AI systems, and implementing a "relay" model that evaluates both independent reasoning and AI-assisted skills.1
评论
还没有评论，欢迎留下第一条。