一篇发表在Hacker News的文章阐述了如何从系统微观态数量出发定义马尔可夫链的熵[1]。作者以玻尔兹曼的经典公式S = k·ln(Ω)为基础，其中Ω代表系统可能的微观态数，说明熵与系统可能状态数的关系[1]。
文章通过具体模型演示了这一方法的应用[1]。在库里磁体模型中，当5个原子的能量为E=1时，存在10种微观配置（对应5选3的组合），相应的熵为log₂10 = 3.32比特[1]。随后作者将该方法应用于戴森细胞玩具模型的马尔可夫链，在平衡态下假设空态概率为1/2、活跃态和非活跃态各为1/4，对8个位点的配置进行计数，其中4个空态、2个活跃态、2个非活跃态的微观配置数为8!/(4!·2!·2!) = 420[1]。文章还阐述了第二热力学定律，指出在不可逆过程中，熵总是增加，除非向系统施加能量[1]。
An article explores how entropy can be defined for Markov chains by starting from the number of microscopic states in a system.[1] The work draws on Boltzmann's foundational approach, which relates entropy to the number of possible states a system can occupy through the equation S = k·ln(Ω), where Ω represents the count of possible microstates.[1]
The author illustrates this methodology through concrete examples. In the Curie magnetic model with five atoms and energy E=1, there are 10 possible microscopic configurations (calculated as 5 choose 3), corresponding to an entropy of log₂10 = 3.32 bits.[1] The framework is then applied to the Dyson cellular toy model's Markov chain, where the entropy is defined by counting microscopic configurations in the equilibrium state.[1] When the probability of empty cells is 1/2 and both active and inactive states each have probability 1/4, eight sites with 4 empty, 2 active, and 2 inactive cells yield 8!/(4!·2!·2!) = 420 distinct microscopic configurations.[1]
The analysis emphasizes that according to the second law of thermodynamics, entropy always increases in irreversible processes unless energy is applied to the system.[1]