研究团队开发了一套合约级验证器，用于审计大语言模型生成的GPU内核代码质量[1]。该验证器对2,638个机器生成的内核进行了系统检查，结果显示39.5%的内核存在根本性缺陷，62.1%至少包含一项违规问题[1]。
验证器采用了12个对抗性检验门进行多维度评估，其中多个检验设置了无容差限制[1]。值得注意的是，业界现有的标准测试框架会接受1,487个被验证器判定为错误的内核，而仅有14个反向情况发生，这表明验证器的检测能力明显强于现有测试标准[1]。此外，研究还提出了首个原生支持Blackwell架构的门控线性递归家族训练反向传播内核[1]。
Researchers have developed a contract-grade verifier designed to assess the correctness of GPU kernels generated by large language models.[1] The verifier was used to audit 2,638 machine-generated kernels, revealing significant quality issues: 39.5% contained fundamental defects, and 62.1% exhibited at least one violation of correctness standards.[1] In a striking contrast, industry-standard testing accepted 1,487 kernels that the verifier identified as incorrect, while rejecting only 14 that the verifier deemed faulty.[1]
The verification system employs twelve adversarial test gates, many of which operate without tolerance margins, to catch errors across multiple dimensions of kernel correctness.[1] The research also introduces the first natively optimized training backpropagation kernel for Blackwell architecture in the gated linear recurrence family.[1] These findings underscore both the prevalence of subtle bugs in LLM-generated compute code and the limitations of conventional testing methodologies in detecting them.