RTK（Rust Token Killer）是一款广受欢迎的工具，曾声称能将Claude Code的token成本削减高达60%1。其中一条社交媒体帖子因这一说法获得了31.3万次浏览1。RTK官方文档更是宣称可削减高达90%的bash输出1。然而，JetBrains研究团队通过大规模基准测试发现，RTK的实际效果与这些承诺存在显著偏差。
研究团队在Terminal-Bench 2.1上进行了1740次测试，耗资超过1500美元，覆盖85个Fable任务和89个DeepSeek任务1。测试结果显示，RTK对不同模型的影响差异巨大。按成本与通过率的综合计算，使用RTK后Fable模型成本下降3%，而DeepSeek模型成本反而上升7%1。若按任务平均水平衡量，Fable成本增加1%，DeepSeek成本平均增加17%1。值得注意的是，Fable所有的成本节省主要来自单个任务1，这进一步削弱了RTK效果的普遍性。
数据进一步暴露了RTK报告与实际成本的偏离。RTK声称节省了349.2百万token（减少89%），但这一庞大数字并未转化为相应的账单节省1。在DeepSeek的测试中，虽然终端输出字符减少了9%，但提示token反而增加了9%1。在58个增加turns数量的DeepSeek任务中，其中44个的成本反而更高1。此外，两个模型在使用RTK后的通过率都出现下滑，分别下降1%和2%1。
A popular tool called RTK (Rust Token Killer) has generated significant attention with claims of reducing token costs for Claude Code by up to 60%, yet independent testing reveals substantially more modest results.1 An X post asserting RTK could slash Claude Code tokens by as much as 60% accumulated 313,000 views, while RTK's official documentation states the tool can reduce bash output by up to 90%, though it cautions this does not equate to an equivalent reduction in billing costs.1
Researchers from JetBrains conducted an extensive evaluation using Terminal-Bench 2.1, spending over $1,500 to run 1,740 tests across 85 Fable tasks and 89 DeepSeek tasks.1 The findings diverged sharply from RTK's promotional claims. When measured by cost per successful completion, RTK reduced expenses for Fable by only 3%, with the majority of savings concentrated in a single task, while increasing costs for DeepSeek by 7%.1 By average task-level analysis, Fable costs rose 1% with RTK enabled, and DeepSeek costs climbed an average of 17%.1 Pass rates declined by 1% for Fable and 2% for DeepSeek after implementing RTK.1
The discrepancy between RTK's reported efficiency gains and actual cost savings proved particularly stark for DeepSeek.1 While terminal output characters decreased by 9% in RTK-enabled attempts, prompt tokens actually increased by 9%, indicating the tool's optimization strategy did not uniformly reduce overall token consumption.1 RTK reported 349.2 million tokens saved—an 89% reduction—yet this figure did not correspond to proportional cost savings in practice.1 Among 58 DeepSeek tasks where RTK increased the number of turns, 44 experienced higher costs.1
评论
还没有评论，欢迎留下第一条。