Meta FAIR团队开源了名为e-means的新型k-means聚类算法,旨在作为传统Lloyd算法的现代化、稳健替代方案1。该算法通过引入指数移动平均(EMA)更新规则和自动超参数调整机制,在保持更低损失值的同时,实现了约10倍的训练速度提升1。具体而言,相较于Flash-KMeans,e-means在GPU上的墙钟时间缩减幅度达到10至45倍;而在CPU上对比Faiss,其性能提升约为8至15倍1。
在技术实现层面,e-means采用迷你批次更新策略,并结合缩放定律自动设定批次大小与步长,从而免除了用户手动调整超参数的需求1。针对传统聚类中常见的局部极小值和“死簇”问题,该算法引入了改进的初始化方法以及余弦调度衰减率,并创新性地通过分割大簇而非随机采样的方式来复活死簇1。这些技术优势已在包括SIFT和SPACEV在内的24种不同嵌入及合成数据集上得到实验验证1。目前,e-means的代码已托管于faiss/contrib/e_means仓库,相关论文引用编号为arXiv:2611.095011。
Meta FAIR has open-sourced e-means, a modern and robust k-means clustering algorithm designed as an alternative to the traditional Lloyd’s algorithm. 1 The new method achieves approximately 10 times faster training in terms of both epochs and actual wall-clock time compared to Lloyd’s algorithm, while simultaneously yielding lower loss values. 1 Specifically, on GPUs, e-means demonstrates a 10–45 times reduction in wall-clock time relative to Flash-KMeans, and on CPUs, it offers an 8–15 times speedup over Faiss. 1
The performance gains are driven by technical innovations including mini-batch updates combined with exponential moving average (EMA) rules. 1 This approach eliminates the need for manual hyperparameter tuning by automatically setting batch sizes and step lengths through scaling laws. 1 Furthermore, the algorithm incorporates improved initialization strategies, a cosine schedule decay rate, and a novel "dead cluster" revival mechanism that splits large clusters instead of relying on random sampling. 1 These features collectively help avoid local minima and enhance stability. 1
Validation experiments were conducted across 24 different embedding types and synthetic datasets, such as SIFT and SPACEV. 1 The source code is available in the faiss/contrib/e_means repository, and the underlying research paper is indexed under arXiv:2611.09501. 1
评论
还没有评论,欢迎留下第一条。