Randall Balestriero于2026年10月29日发布题为《Proving and Improving the Platonic Representation Hypothesis with JEPAs》的研究论文,成功将“柏拉图表征假说”(PRH)从理论猜想转化为数学定理1。该研究核心发现指出,在“良好视图”条件下——即同一场景的两个观测值——使用联合嵌入预测架构(JEPA)训练的不同模态模型能够收敛到相同的世界表征,两者之间仅存在一次旋转差异1。
实验数据表明,相较于传统大语言模型中常见的基于文档片段的“不良视图”,JEPA采用的“良好视图”能更高效地实现模态对齐1。具体而言,达到相同的对齐效果时,“良好视图”所需的训练样本量比传统的“下一个token预测”方法减少42倍1。在性能对比中,一个参数量仅为1.09亿的文本JEPA模型,在与DINOv2和CLIP等图像编码器的对齐表现上,优于所有评估过的、参数量高达720亿的大语言模型以及CLIP文本编码器1。例如,在ImageNet-100零样本分类任务中,该JEPA模型准确率达到64%,而参数量在8B以下的语言模型最高准确率仅为56%1。此外,研究还展示了通过单次旋转即可将独立训练的编码器组合成图像描述生成器,其在COCO数据集上的CIDEr得分为0.8381。
A new study by Randall Balestriero, submitted on October 29, 2026, transforms the "Platonic Representation Hypothesis" (PRH) from a conjecture into a theorem by demonstrating that models trained using Joint Embedding Predictive Architectures (JEPA) converge to identical world representations across different modalities 1. The research establishes that under specific conditions known as "good views"—defined as two observations of the same scene—different JEPA models learn representations that differ only by a single rotation 1. This finding suggests that good views are significantly more efficient for aligning modalities than the "bad views," such as document fragments, commonly used in traditional large language models 1.
Experimental results highlight substantial efficiency gains, showing that achieving the same level of alignment requires 42 times fewer training samples when using good views compared to next-token prediction methods 1. In terms of performance, a text JEPA model with 109 million parameters outperformed all evaluated large language models up to 72 billion parameters, as well as CLIP text encoders, in alignment tasks with visual encoders like DINOv2 and CLIP 1. Specifically, in zero-shot classification on ImageNet-100, this JEPA model achieved an accuracy of 64%, surpassing the maximum 56% accuracy recorded by language models with fewer than 8 billion parameters 1. Furthermore, the study demonstrated practical application by combining independently trained encoders through a simple rotation to create an image captioning generator, which scored 0.838 CIDEr on the COCO dataset 1.
评论
还没有评论,欢迎留下第一条。