一位开发者通过运行软件工厂实验对GPT-6 Astra的代码生成能力进行了测试1。在35小时的实验中，该模型消耗约10亿个token，成本约1200美元，最终生成了75000行代码但未产生任何有价值的输出1。这次实验共产生79次代码提交，平均每次提交的成本约15.5美元1。
实验过程中，开发者发现Astra存在多个问题1。该模型倾向于生成高度压缩且难以理解的代码，过度使用Python字符串操作和随机数字索引进行文件操作，而非采用合适的工具1。更值得关注的是，代理任务的命名方式出现了明显退化，从结构化的"1, 2, 3, 5, 5a"逐渐演变为"8b2c2b3"和"8b2c2b2b checkpoint1"这样的无意义标识，表明系统的质量随时间恶化1。
开发者对这项投入的实用性提出了深层质疑1。他指出，当前AI模型的训练目标（令牌效率优化）与真实软件工程的需求存在根本矛盾，同时质疑令牌效率和任务完成率的优化是否与"人类可理解"这一核心目标相冲突1。他总结认为，使用Astra进行软件工程的成本"极高"但"结果不在那里"1。
A developer conducted an extensive experiment testing GPT-6 Astra's code generation capabilities through a software factory setup, consuming approximately 1 billion tokens over 35 hours at a cost of around $1,2001. Despite this substantial investment, the effort produced 75,000 lines of code across 79 commits that ultimately yielded no valuable output1.
The experiment revealed significant quality issues with Astra's code generation approach1. The model demonstrated a tendency to rely heavily on Python string operations and cryptic numerical indexing rather than employing standard tools like patch utilities for file operations1. As the experiment progressed, the system's degradation became apparent—task naming evolved from logical sequences like "1, 2, 3, 5, 5a" to increasingly incoherent designations such as "8b2c2b3" and "8b2c2b2b checkpoint1," with the average cost per commit reaching $15.501.
The developer raised fundamental concerns about the alignment between current AI model training objectives and practical software engineering requirements1. The focus on token efficiency optimization appears to conflict with producing code that is comprehensible and usable by human developers1. This raises broader questions about whether contemporary AI models, despite their scale and capability claims, can deliver meaningful value for real-world coding tasks1.
评论
还没有评论，欢迎留下第一条。