机器学习中无需矩阵微积分

一篇发布在Hacker News上的文章论证，在机器学习领域采用传统微分方法即可满足需求，无必要使用矩阵微积分1。作者建议改用物理学家的做法：将向量和矩阵公式展开为元素级别的表达式，随后应用普通微分法则进行求导1。

文章通过具体示例阐述了这一方法的应用。以神经元激活函数为例，作者展示了如何用索引记号表示激活函数为 activation(x)_i = max(0, (Ax)_i + b_i)1，进而通过设定对应微分变量为1、其余为0的方式，逐元素地求取不同的偏导数，包括 ∂activation(x)_i/∂A_kl 和 ∂activation(x)_i/∂x_k 等1。作者声称这种逐元素求偏导的索引记号法相比矩阵微积分更为简洁高效1。

An author argues that matrix calculus is unnecessarily complex for machine learning applications and proposes a simpler alternative approach inspired by physics methodology 1. Rather than relying on matrix calculus, the proposed method involves expanding vector and matrix formulas down to element-level expressions and applying standard differentiation rules 1.

The approach uses index notation to work with individual elements, such as representing an activation function as activation(x)_i = max(0, (Ax)_i + b_i) 1. By systematically setting the differential variable corresponding to a specific element to 1 and all others to 0, practitioners can derive partial derivatives for different components 1. The author provides concrete derivations for partial derivatives such as ∂activation(x)_i/∂A_kl and ∂activation(x)_i/∂x_k, claiming that this element-by-element differentiation method is more concise and efficient than traditional matrix calculus 1.