儿童掌握语言的效率远远超越现代人工智能模型，这一发现正引发科学家的深入思考。1研究表明，儿童在满一岁时仅通过听到约1000万单词就已开始掌握语言，而Meta的Llama 3.1等现代大型语言模型在预训练时需要15万亿个token才能获得类似能力。1斯坦福大学认知科学家Michael C. Frank指出这一巨大差异的荒谬性："我们仍然需要砍倒一片森林、汇集人类所有知识的总和，才能重现这个在我们客厅里花一年时间就发生的里程碑。"1
为了破解儿童学习的秘密，研究人员正通过多个项目进行反向工程。12024年的BabyLM竞赛吸引科研团队尝试用更少数据训练AI模型，其冠军模型GPT-BERT仅在约1亿单词上预训练，却在某项基准测试中超越了用约15000倍训练数据量预训练的Meta Llama 2 70B模型。1为了更深入理解儿童的学习过程，SAYCam项目记录了3名儿童从6个月到2.5岁期间每周2小时的生活，共提供61小时视频数据供研究之用；1另一项由Uri Hasson主导的项目则记录了17名儿童前1000天的生活，每天12小时。1这些研究努力不仅有助于开发更高效的AI模型，也将揭示人类语言获得和认知的基础机制。
Children master language with extraordinary efficiency compared to large language models, a gap that persists despite dramatic advances in AI over the past four years.1 By age one, children typically grasp language after hearing approximately 10 million words, yet modern LLMs require vastly more data—Meta's Llama 3.1, for instance, demands 150 trillion tokens during pretraining.1 According to Michael C. Frank, a cognitive scientist at Stanford University, the disparity remains stark: "We still have to burn down a forest and scrape the entire sum of all human knowledge to re-create this milestone that happens in our living rooms over the course of a year."1 When GPT-2 is trained on just 30 million words, Frank notes, "you get a nonsense generator; you don't get a kid."1
Researchers are now systematically reverse-engineering how children learn in hopes of building more efficient AI models. The 2024 BabyLM competition produced a striking result: GPT-BERT, trained on approximately 100 million words, surpassed Meta's Llama 2 70B on certain benchmark tests despite using roughly 15,000 times less training data.1 To understand the cognitive mechanisms underlying efficient language acquisition, scientists have launched ambitious data-collection projects. The SAYCam initiative recorded three children for two hours per week between ages six months and 2.5 years, yielding 61 hours of video data for analysis.1 Uri Hasson's project extended this approach further, documenting the first 1,000 days of life for 17 children at 12 hours daily.1
评论
还没有评论，欢迎留下第一条。