一个GitHub议题指控pgrust数据库在ClickBench基准测试中采用不正当手段获得高分1。根据调查，pgrust v0.4-preview二进制文件针对基准测试的43个查询分别嵌入了专用手写内核，并通过使用缓存结果而非实际计算来伪造性能数据1。该引擎通过计划匹配器和环境变量来控制查询执行行为，对不在预定范围内的查询变体会直接拒绝，报错提示为"sqe: statement over columnar relation is not served"1。
测试者在本地验证中发现了缓存机制的证据1。以Q20查询为例，热运行耗时从11.2秒大幅下降至3毫秒，表明系统在重放前一次运行的缓存答案而非进行真实扫描1。当进行实际执行时，耗时达到数百毫秒，与ClickHouse在同一硬件上的性能相当1。在67个相似查询的探测中，20个被拒绝执行；在49个查询变体中，3个被拒绝1。尽管虚假的基准成绩不可信，但该项目的43个基准查询答案已被验证为正确，其中30个完全匹配，其余为LIMIT查询的有效结果1。此外，pgrust不支持INSERT、UPDATE、DELETE等数据修改操作1。
A GitHub issue (#2165) has alleged that pgrust employed unfair practices to achieve inflated performance scores on the ClickBench database benchmark 1. The accusation centers on pgrust v0.4-preview, which reportedly embedded hand-written kernels for each of the 43 ClickBench test queries (labeled q00.rs through q36.rs) rather than executing genuine database operations 1.
The investigation revealed multiple deceptive mechanisms within pgrust's query execution layer 1. A query planner utilized environment variables and rejection messages to control which queries would be executed, denying service to variant queries outside the predefined 43-query set with error messages stating "sqe: statement over columnar relation is not served" 1. In testing, approximately 67 similar query probes resulted in 20 rejections, while 49 query variants saw 3 failures 1. Most critically, tester verification of the Q20 query demonstrated that hot runs completed in just 3 milliseconds—down from an initial 11.2 seconds—indicating result caching rather than actual computation 1. When genuinely executed, queries required hundreds of milliseconds, performance metrics comparable to ClickHouse running on identical hardware 1.
While pgrust's answers for the 43 benchmark queries were verified as correct—30 matching exactly and the remainder being acceptable tie-breaking results from LIMIT queries—the underlying operations did not reflect legitimate database functionality 1. The project additionally lacks support for fundamental SQL operations including INSERT, UPDATE, and DELETE 1.
评论
还没有评论，欢迎留下第一条。