大语言模型代码生成存在"松散度"问题 研究揭示AI生成代码质量缺陷

基于Earendil公司的研究，科研人员提出了衡量大语言模型生成代码质量问题的新方法1。尽管大语言模型在生成形式正确的代码方面已接近完美，但其产出的代码存在不必要的抽象、重复代码片段和不良设计决策等问题1。

研究引入了SlopCodeBench论文中的两个关键指标来量化代码质量，发现代理模型生成的代码在这些指标上显著落后于已建立的代码库1。具体而言，代理模型生成代码的冗余度平均为0.33±0.10，而已建立的代码库为0.15±0.061；侵蚀度指标上，代理模型生成代码为0.68±0.20，已建立代码库为0.31±0.171。这表明代理模型生成的代码在冗余度和侵蚀度方面分别约为已建立代码库的两倍1。

更为严峻的是，在SlopCodeBench的严格多轮迭代测试中，最先进的模型的完全通过率为0%1。研究同时指出，虽然人类评估代码质量的准确度较高，但难以实现规模化，而大语言模型的自我评估效果则不理想1。

While large language models have achieved near-perfect accuracy in generating formally correct code, a new analysis reveals significant quality issues lurking beneath the surface. Researchers from Earendil have introduced metrics to quantify what they call "code sloppiness"—unnecessary abstractions, redundant logic, and poor design decisions that plague AI-generated programs 1.

The study employs two key measurements from the SlopCodeBench framework: redundancy and erosion 1. AI-generated code exhibits redundancy levels averaging 0.33 ± 0.10, roughly double the 0.15 ± 0.06 found in established codebases, while erosion metrics reach 0.68 ± 0.20 compared to 0.31 ± 0.17 in human-written repositories 1. When subjected to rigorous multi-round iterative testing, state-of-the-art models achieved a zero percent pass rate across all checkpoints in the strict SlopCodeBench evaluation 1.

The research highlights a fundamental challenge in assessing code quality at scale: while large language models struggle to self-evaluate their own output effectively, human assessment, though accurate, cannot be practically scaled 1. The findings underscore that the field has moved beyond simply solving the problem of code correctness, raising new questions about how to systematically measure and improve the overall quality of machine-generated code 1.