[英]Python reconstruct function from AST, default parameters
I am attempting to implement a decorator that receives a function, parses it into an AST, eventually will do something to the AST, then reconstruct the original (or modified) function from the AST and return it.我正在尝试实现一个装饰器,它接收 function,将其解析为 AST,最终将对 AST 做一些事情,然后从 AST 重建原始(或修改后的)function 并返回它。 My current approach is, once I have the AST,
compile
it to a code <module>
object, then get the constant in it with the name of the function, convert it to FunctionType
, and return it.我目前的方法是,一旦我有了 AST,将其
compile
为code <module>
object,然后在其中获取名称为 function 的常量,将其转换为FunctionType
并返回它。 I have the following:我有以下内容:
import ast, inspect, types
def as_ast(f):
source = inspect.getsource(f)
source = '\n'.join(source.splitlines()[1:]) # Remove as_ast decoration, pretend there can be no other decorations for now
tree = ast.parse(source)
print(ast.dump(tree, indent=4)) # Debugging log
# I would modify the AST somehow here
filename = f.__code__.co_filename
code = compile(tree, filename, 'exec')
func_code = next(
filter(
lambda x: isinstance(x, types.CodeType) and x.co_name == f.__name__,
code.co_consts)) # Get function object
func = types.FunctionType(func_code, {})
return func
@as_ast
def test(arg: int=4):
print(f'{arg=}')
Now, I would expect that calling test
later in this source code will simply have the effect of calling test
if the decorator were absent, which is what I observe, so long as I pass an argument for arg
.现在,我希望在此源代码中稍后调用
test
将只是在没有装饰器的情况下调用test
的效果,这是我观察到的,只要我为arg
传递一个参数。 However, if I pass no argument, instead of using the default I gave ( 4
), it throws a TypeError
for the missing argument.但是,如果我不传递任何参数,而不是使用我给出的默认值 (
4
),它会为缺少的参数引发TypeError
。 This makes it pretty clear that my approach for getting a callable function from the AST is not quite correct, as the default argument is not applied, and there may be other details that would slip through as it is now.这很清楚地表明,我从 AST 获取可调用 function 的方法并不完全正确,因为未应用默认参数,并且可能还有其他细节会像现在一样漏掉。 How might I be able to correctly recreate the function from the AST?
我怎样才能从 AST 正确地重新创建 function? The way I currently go from the code module object to the function code object also seems... off intuitively, but I do not know how else one might achieve this.
The way I currently go from the code module object to the function code object also seems... off intuitively, but I do not know how else one might achieve this.
The root node of the AST is a Module. AST 的根节点是一个模块。 Calling
compile()
on the AST, results in a code object for a module.在 AST 上调用
compile()
会导致模块的代码为 object。 Looking at the compiled code object returned using dis.dis()
, from the standard library, shows the module level code builds the function and stores it in the global name space.查看使用
dis.dis()
从标准库返回的编译代码 object,显示模块级代码构建 function 并将其存储在全局名称空间中。 So the easiest thing to do is exec
the compiled code and then get the function from the 'global' environment of the exec
call.所以最简单的做法是执行编译后的代码,然后从
exec
调用的“全局”环境中获取exec
。
The AST node for the function includes a list of the decorators to be applied to the function. function 的 AST 节点包括要应用于 function 的装饰器列表。 Any decorators that haven't been applied yet should be deleted from the list so they don't get applied twice (once when this decorator compiles the code, and once after this decorator returns).
任何尚未应用的装饰器都应该从列表中删除,这样它们就不会被应用两次(一次在此装饰器编译代码时,一次在此装饰器返回后)。 And delete this decorator from the list or you'll get an infinite recursion.
并从列表中删除此装饰器,否则您将获得无限递归。 The question is what to do with any decorators that came before this one.
问题是如何处理在此之前出现的任何装饰器。 They have already run, but their result is tossed out because this decorator (
as_ast
) goes back to the source code.他们已经运行了,但是他们的结果被抛弃了,因为这个装饰器(
as_ast
)回到了源代码。 You can leave them in the list so they get rerun, or delete them if they don't matter.您可以将它们保留在列表中,以便它们重新运行,或者如果它们无关紧要则删除它们。
In the code below, all the decorators are deleted from the parse tree, under the assumption that the as_ast
decorator is applied first.在下面的代码中,假设首先应用
as_ast
装饰器,所有装饰器都从解析树中删除。 The call to exec()
uses a copy of globals()
so the decorator has access to any other globally visible names (variables, functions, etc).对
exec()
的调用使用globals()
的副本,因此装饰器可以访问任何其他全局可见的名称(变量、函数等)。 See the docs for exec()
for other considerations.有关其他注意事项,请参阅
exec()
的文档。 Uncommented the print statements to see what is going on.取消注释打印语句以查看发生了什么。
import ast
import dis
import inspect
import types
def as_ast(f):
source = inspect.getsource(f)
#print(f"=== source ===\n{source}")
tree = ast.parse(source)
#print(f"\n=== original ===\n{ast.dump(tree, indent=4)}")
# Remove the decorators from the AST, because the modified function will
# be passed to them anyway and we don't want them to be called twice.
for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef):
node.decorator_list.clear()
# Make modifications to the AST here
#print(f"\n=== revised ===\n{ast.dump(tree, indent=4)}")
name = f.__code__.co_name
code = compile(tree, name, 'exec')
#print("\n=== byte code ===")
#dis.dis(code)
#print()
temp_globals = dict(globals())
exec(code, temp_globals)
return temp_globals[name]
Note: this decorator has not been tested much and has not been tested at all on methods or nested functions.注意:这个装饰器没有经过太多测试,也没有在方法或嵌套函数上进行过测试。
An interesting idea would be to for as_ast
to return the AST.一个有趣的想法是让
as_ast
返回 AST。 Then subsequent decorators could manipulate the AST.然后后续的装饰器可以操纵 AST。 Lastly, a
from_ast
decorator could compile the modified AST into a function.最后,
from_ast
装饰器可以将修改后的 AST 编译成 function。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.