研究人员构建了一个基于物理模拟器的特斯拉Roadster飞行强化学习环境，旨在测试大语言模型在强风干扰下的实时控制能力 1。该实验模拟了从坡道翻转并飞跃Cybertruck的动作，涉及对四个冷气推进器（每个推力12 kN）的控制 1。结果显示，经过强化学习后训练的4B参数Qwen模型在实时任务中表现优于包括GPT-6 Astra在内的主流大模型 1。
在优化方案上，团队使用SkyRL框架对Qwen3-4B-Instruct进行了LoRA微调及GRPO强化学习训练，总成本约为13美元（耗时4 GPU小时） 1。性能对比数据显示，训练后的4B模型平均响应时间仅为0.09秒，在33 m/s的风速下Kickflip动作成功率达到90% 1。相比之下，拥有无限思考时间的GPT-6 Astra成功率为74%，这意味着小模型在实时场景中的性能高出21个百分点 1。
基准测试揭示了“实时性”是大模型在物理控制任务中的关键瓶颈。在实时模拟环境中，DeepSeek V4.1 Flash、Muse Spark Open等模型因每次决策耗时1至6秒，导致Kickflip成功率为0%；GPT-6 Astra同样为0% 1。然而，当模拟器暂停以允许无限思考时，DeepSeek在风暴环境中的表现最佳，成功率达87%，但一旦恢复实时时钟，其性能立即归零 1。目前，相关的RL环境、提示词及SkyPilot配置文件已在GitHub开源，支持用户自定义模型接入 1。
Researchers have developed a reinforcement learning environment based on a physics simulator to evaluate the real-time control capabilities of large language models (LLMs) under storm conditions. 1 The simulation replicates a demonstration involving a Tesla Roadster performing a kickflip over a Cybertruck, requiring precise control of four cold gas thrusters, each generating 12 kN of thrust. 1 Major LLMs, including GPT-6 Astra and DeepSeek V4.1 Flash, exhibited extremely high failure rates in this task due to significant decision-making delays during live simulation runs. 1
In contrast, a post-trained 4-billion parameter Qwen model achieved a 90% landing success rate with an average response time of just 0.09 seconds in windspeeds reaching 33 m/s. 1 This performance was 21 percentage points higher than that of GPT-6 Astra, which managed a 74% success rate even when allowed unlimited thinking time by pausing the simulation. 1 The optimization involved applying LoRA fine-tuning and Group Relative Policy Optimization (GRPO) reinforcement learning training to the Qwen3-4B-Instruct model using SkyRL, a process that cost approximately $13 and took four GPU hours. 1
The experiment highlighted that real-time responsiveness is the critical bottleneck for LLM-based physical control. 1 When the simulator was paused to allow unlimited computation time, DeepSeek V4.1 Flash performed best with an 87% success rate in storm conditions, but its performance dropped to zero once the real-time clock resumed. 1 Similarly, other models such as Muse Spark Open failed entirely, with a 0% kickflip success rate, because their decision processes took between one and six seconds per step. 1 All code, data, RL environments, prompts, and SkyPilot configuration files are open-sourced on GitHub, enabling users to integrate custom models into the framework. 1
评论
还没有评论，欢迎留下第一条。