一个新提出的基于Grassmann流的神经网络架构质疑了显式自注意力机制对实现强性能的必要性。1这项研究于2025年12月22日提交，其核心创新在于用低秩子空间的几何变形替代传统的非结构张量空间。1
该架构在多个基准测试上与同规模Transformer模型相当或略优。在Wikitext-2语言建模任务上，Grassmann架构的困惑度与Transformer相差10-15%。1在SNLI自然语言推理任务上，Grassmann-Plücker头的验证集准确率达0.8550、测试集0.8538，而对应的Transformer分别为0.8545和0.8511。1该模型的参数规模为13-18百万参数。1
这项工作通过在Grassmann流形上用Plücker坐标编码局部令牌对，为序列建模提供了更清晰的数学解释基础。1
Researchers have proposed a novel neural network architecture that challenges the necessity of explicit self-attention mechanisms for achieving strong performance in sequence modeling tasks.1 The approach, detailed in a paper submitted on December 22, 2025, introduces a Grassmann flow-based architecture that encodes local token pairs on Grassmann manifolds using Plücker coordinates, offering an alternative geometric framework to traditional attention mechanisms.1
The proposed method demonstrates competitive performance across standard benchmarks.1 On the Wikitext-2 dataset, the Grassmann architecture achieves perplexity within 10-15% of transformer models of comparable scale.1 For the SNLI natural language inference task, the Grassmann-Plücker variant achieves validation set accuracy of 0.8550 and test set accuracy of 0.8538, compared to transformer performance of 0.8545 and 0.8511 respectively.1 The models operate with 13-18 million parameters.1 Rather than relying on non-structured tensor spaces, the architecture replaces conventional attention with geometric transformations of low-rank subspaces, providing a more mathematically interpretable alternative for sequence processing.1
评论
还没有评论，欢迎留下第一条。