Google的Gemini 4 Argon在StarSkirmish Bench v0.2基准测试中获得37-39分,与Claude Opus 5.5和Claude Fable 5.1并列第三1。该测试评估大语言模型在1小时内用C++编写星际争霸Protoss机器人的能力1。OpenAI的GPT-6 Astra模型排名第一,GPT-6.1 Sol排名第二1。
尽管这些模型展现了代码生成能力,但它们仍远低于顶级人类编写机器人的水平1。排名第一的GPT-6 Astra与顶级人类机器人(Stardust、PurpleWave、BananaBrain)的对战中仅赢得7%的游戏1。研究发现,模型得分的最强预测因素是在一小时内对7个最强机器人的练习次数1。
Gemini 4 Argon在不同运行间表现出最大的差异性,最佳和最差运行之间相差18分,而GPT-6 Astra为9分,Claude Opus 5.5为8分1。
Google's Gemini 4 Argon has been evaluated in StarSkirmish Bench v0.2, a benchmark that measures large language models' ability to write StarCraft bots within a one-hour timeframe 1. The test gave each model the same constraint: one hour to generate C++ code for a Protoss bot 1. Gemini 4 Argon placed third overall with a score of 37-39 points, tied with Claude Opus 5.5 and Claude Fable 5.1 1. OpenAI's GPT-6 Astra claimed first place, followed by GPT-6.1 Sol in second 1.
Despite their high rankings, all tested models remain substantially below the performance level of top-tier human-written bots 1. The leading model, GPT-6 Astra, won only 7 percent of games against elite human competitors including Stardust, PurpleWave, and BananaBrain 1. Analysis revealed that the number of practice matches against the seven strongest bots completed within the one-hour window emerged as the strongest predictor of model performance 1. Notably, Gemini 4 Argon exhibited the highest variance across different runs, with an 18-point gap between its best and worst performances, compared to 9 points for GPT-6 Astra and 8 points for Claude Opus 5.5 1.
评论
还没有评论,欢迎留下第一条。