如何在PyTorch中查找和了解autograd源代码

Question

I have a good understanding of the autograd algorithm, and I think I should learn about the source code in PyTorch. However, when I see the project on GitHub, I am confused by the structure, cuz so many files include autograd. So which part is the most important core code of autograd?

Answer 1

1. Try to understand the autograd variable is probably the first thing, what you can do. From my understanding is autograd only a naming for the modules, which containing classes with enhancement of gradients and backward functions.

2. Be aware a lot of the algorithm, eg back-prop through the graph, is hidden in compiled code.

3. If you look into the __init__.py , you can get a glimpse about all important functions (backward & grad)

Answer 2

I recommend you make associations between your understanding of autograd and the PyTorch datastructures involved by making a simple graph and printing/visualizing the structure as below:

• Print Autograd Graph
• Resnet example viz (jupyter notebook)

Answer 3

Reading the PyTorch code is doable, but you may be overwhelmed with details. To get a basic idea of autograd, you may want to refer to some simple autograd implementations, such as https://evcu.github.io/ml/autograd/ and https://medium.com/@ralphmao95/simple-autograd-implementation-understand-automatic-differentiation-hand-by-hand-9e86f6d703ab