OpenAI发布的GPT-6 Astra模型采用了循环变换器(looped transformers)架构,在多项基准测试中取得显著成效。1该模型在约100,000个Grace Blackwell GPU上进行训练,1并在ARC-AGI-3基准测试中达到99.9%的准确率,远超前代GPT-5.6 Sol的7.8%。1在固定精度条件下,Astra使用的token数相比GPT 5.6 Sol更少。1
循环变换器架构通过让多个变换器块进行重复应用来提升效率。1例如Nanbeige 4.2-3B模型使用22个变换器块循环2次,相当于44次块应用但参数数量减半。1相关研究表明,这种Mixture-of-Recursions方法在较大模型和较小训练预算条件下的性能优于标准变换器。1关于该架构是否刻意隐藏推理过程的说法被文章驳斥,OpenAI首席科学家Jakub Pachocki声明:"我们现有的前沿模型(包括Astra)的计算图深度在GPT-4的两倍以内。"1
在应用能力方面,Astra在数学、编码和3D渲染等任务上相比前代模型有显著提升,特别是在计算机交互和图形处理方面表现突出。1为了训练计算机使用能力,OpenAI购买了数万台Mac Minis和Mac Studios用于强化学习。1
OpenAI has unveiled GPT-6 Astra, a new large language model incorporating innovative architectural advances and demonstrating substantial performance gains across multiple domains.1 The model was trained on approximately 100,000 Grace Blackwell GPUs, according to OpenAI's CEO.1
GPT-6 Astra exhibits dramatic improvements in benchmark performance, achieving 99.9% accuracy on the ARC-AGI-3 test compared to just 7.8% for the preceding GPT-5.6 Sol model.1 The model shows particular strength in mathematical problem-solving, code generation, and three-dimensional rendering tasks, as well as notable capabilities in computer interaction and graphics processing.1 OpenAI has invested in thousands of Mac Minis and Mac Studios to support reinforcement learning training focused on developing computer usage abilities.1
The model employs looped transformer architecture—a technique that cycles through transformer blocks multiple times rather than stacking them linearly, thereby reducing parameter count while maintaining computational depth.1 Research on Mixture-of-Recursions demonstrates that this approach outperforms standard transformer configurations in larger models with constrained training budgets.1 OpenAI's Chief Scientist Jakub Pachocki stated: "Our current frontier models, including Astra, have computational graph depth within twice that of GPT-4,"1 refuting claims that the architecture deliberately obscures reasoning processes. Additionally, Astra requires fewer tokens than GPT-5.6 Sol when operating at fixed precision levels.1
评论
还没有评论,欢迎留下第一条。