PyTorch被称为现代深度学习的"通用语言"，其独特之处在于兼具研究灵活性和生产可扩展性的双重角色[1]。有观点认为，深度学习框架应采用参考实现与生产实现分离的架构，即拥有用于研究的实现、用于规模化的实现，以及用于验证两者等价性的验证工具[1]。
虽然内核DSL已改变了算子实现的方式，但维持PyTorch参考实现用于数值验证仍然必要[1]。为了优化大规模训练性能，可通过大语言模型生成显式的前向-反向计算代码，从而避免隐式自动求导图带来的性能问题[1]。同时，可通过比特级等价性测试或图捕获等手段来验证参考实现与优化实现之间的等价性[1]。
PyTorch has emerged as the universal language of modern deep learning, serving a dual role that balances research flexibility with production scalability [1]. The framework functions simultaneously as both a reference implementation for algorithmic verification and a production system for large-scale training, a relationship conceptualized through the principle: "One implementation to research in, one implementation to scale with, and one verifier to, in the darkness, bind them" [1].
The architecture separates concerns across multiple layers of implementation [1]. While kernel domain-specific languages have transformed how operators are implemented, PyTorch's reference implementation is retained as a numerical validation tool to ensure correctness across optimized variants [1]. This separation enables researchers to work with accessible, understandable code while production systems leverage specialized, high-performance implementations without compromising reliability.
To address performance challenges in large-scale training, the approach leverages large language models to generate explicit forward and backward computation graphs, circumventing the performance limitations inherent in implicit automatic differentiation [1]. Equivalence between reference and optimized implementations is verified through bit-level equivalence testing or graph capture mechanisms, creating a verification framework that binds the separate implementations together [1].