近期研究对开源模型Kimi K3与闭源模型Fable 5进行了全面对标测试1。此项评估涵盖约1030个智能体任务1，旨在比较两款模型在性能与成本方面的差异。
在SWE基准测试中，K3得分达92.4%，Fable为92.6%1，两者性能接近。但在成本效益方面，K3展现出明显优势。在Oracle路由测试中，有72-96%的任务由K3处理达到最优结果1。
具体来看，在SWE任务上，K3平均消耗55轮和130万tokens完成，而Fable消耗21轮和130K tokens1。在长终端任务中，Fable消耗64轮和150万tokens1。研究指出，通过提示词缓存技术，即使K3读取10倍tokens，仍然保持成本优势1。
研究表明，两个模型在不同任务类型上各有专长，通过实施任务级路由机制，可实现比单一模型更优的综合性能1。
A comparative benchmark analysis evaluated the open-source model Kimi K3 against the closed-source Fable 5 across approximately 1,030 agentic tasks, revealing significant cost-efficiency advantages for K3 1.
On the SWE benchmark, K3 achieved a score of 92.4%, closely matching Fable's 92.6% 1. However, performance efficiency differed substantially between the two models. For SWE tasks, K3 consumed an average of 55 rounds and 1.3 million tokens, while Fable required only 21 rounds and 130,000 tokens 1. On longer-horizon tasks, Fable consumed 64 rounds and 1.5 million tokens 1.
In an Oracle routing evaluation, K3 was selected as the optimal choice for 72–96% of tasks tested 1. The research found that despite K3's higher token consumption per task, it maintains a cost advantage through prompt caching, which allows the model to remain economical even when processing 10 times more tokens 1.
The analysis demonstrated that task-level routing—dynamically selecting between K3 and Fable depending on task requirements—produces superior overall performance compared to relying on any single model alone 1. The findings suggest that the two models exhibit complementary strengths across different task categories 1.
评论
还没有评论，欢迎留下第一条。