这个Python变量的数据类型是什么：node（抽象语法树）

Question

I'm building my own interpreter in C# using Ruslan Pavik's Guide . I am at part 7 where we are creating an Abstract Syntax Tree. I was able to get by the first parts and was able to translate the Python code in the guide into C#. But in Part 7, it became entirely difficult for a guy with no Python experience.

I am having difficulty figuring out what the datatypes and the return types are, with Python being a dynamic programming language.

I am confused on this part:

 def visit(self, node):
    method_name = 'visit_' + type(node).__name__
    visitor = getattr(self, method_name, self.generic_visit)
    return visitor(node)

1. I don't know the datatype for visitor but I think it's a string.
2. visitor gets assigned getattr although I don't know what object it tries to get the attribute from.
3. I don't know what datatype does node have.

Later on in the guide, he declares this class:

class AST(object):
pass

But I know that in C# it's just a class with nothing inside the braces: public class AST { } .

The next class after AST is the BinOp that inherits AST :

class BinOp(AST):
def __init__(self, left, op, right):
    self.left = left
    self.token = self.op = op
    self.right = right

Basically I'm really confused right now. But my main confusion is the node since I don't know what datatype it has. Since a lot of classes in the parser and interpreter use node , I really can't go forward in my interpreter without knowing what datatype it has.

Answer 1

node is an instance of a subclass of AST here. visitor is a method on a subclass (which you are supposed to write yourself) of the NodeVisitor class. It is looked up dynamically because the passed in node is going to be any one of the possible subclasses of AST , and you are expected to implement specific visit_ methods for specific nodes on your custom subclass. The fallback is to use the self.generic_visit method.

AST is the base class for nodes. BinOp is a specific node class; the different classes are documented in the Abstract Grammar section ; each camel-cased name is also an AST subclass.

By looking up methods dynamically by the name of the specific node, the developers avoided having to create specific concrete methods for every node type that the grammar defines.

To make this a little more concrete: say you are interested in specific BinOp operator nodes; perhaps because you want to analyse how + is used in a piece of code.

You can then implement a NodeVisitor subclass, and add a visit_BinOp() method on that subclass, and it'll be called automatically when you pass in a tree of nodes to NodeVisitorSubclass().visit(toplevel_node) ; for any nodes for which there is no specific visit_* method present, the NodeVisitor.generic_visit() method will make sure the child nodes of the tree are visited (through the iter_fields() function).

Demo:

>>> import ast
>>> class DemoVisitor(ast.NodeVisitor):
...     def visit_BinOp(self, binop_node):
...         if isinstance(binop_node.op, ast.Add):
...             print('Found addition of {} and {}'.format(
...                 ast.dump(binop_node.left), ast.dump(binop_node.right)))
...
>>> tree = ast.parse('function(foo + bar)', '', 'eval')
>>> ast.dump(tree)
"Expression(body=Call(func=Name(id='function', ctx=Load()), args=[BinOp(left=Name(id='foo', ctx=Load()), op=Add(), right=Name(id='bar', ctx=Load()))], keywords=[]))"
>>> DemoVisitor().visit(tree)
Found addition of Name(id='foo', ctx=Load()) and Name(id='bar', ctx=Load())

In the above demo, the NodeWalker.visit() method found the visit_BinOp method and called it for us, but since there are no visit_Expression or visit_Call , etc. methods, those nodes were passed to the NodeWalker.generic_visit() method instead, which processes each field and calls self.visit() for any further nodes.

Answer 2

Regarding your question about getattr : This is just a function which returns an attribute with a given name (in the 2nd parameter) of an object (in the 1st parameter) and with a fall back (in the 3rd parameter) if there is no attribute.

So in your case getattr(self, method_name, self.generic_visit) it tries to get an attribute with the name method_name from itself. And if it doesn't exist it uses self.generic_visit . Now if method_name is "visit_BinOp" , it'll try to get self.visit_BinOp , and if that doesn't exist it'll instead get self.generic_visit .

In the end, without running the code, it cannot be determined what it'll return or what type visitor is going to be. It could be anything in theory, but usually it is a method. But that is a difference from Python and C#, that the type of a variables in Python depends on what is assigned to it. You can do the following without an issue:

i = 42
# i is now an int, but you can easily overwrite it with a string
i = "Hello World"

And the same is with node , in theory it can be anything, but if used correctly it'll be a subclass of AST .