将函数递归应用于字符串值的最pythonic方法

Question

Question: I was wondering the most pythonic way to recursively apply a function to string values in an object without knowing it's schema in advance? Preferably in a generic enough manner to make it a reusable component for other types of operations.

Context: I am processing a json string as input from an api request, using json.loads() to load it, before applying validations I'd like to strip off any whitespace before or after any of the strings in the object. I would prefer this code to be adaptive so that changes in the schema do not break it.

Current Solution:

def strip(obj):
    return obj.strip()

def recurse_into(obj, baseaction, basetype=str):
    if isinstance(obj, basetype):
        return baseaction(obj)
    elif isinstance(obj, list):
        return [recurse_into(o, baseaction, basetype) for o in obj]
    elif isinstance(obj, tuple):
        return tuple(recurse_into(o, baseaction, basetype) for o in obj)
    elif isinstance(obj, dict):
        return dict((k, recurse_into(v, baseaction, basetype)) 
                    for (k, v) in obj.items())
    else:
        return obj

def generate_recurse(baseaction, basetype=str):
    def f(obj):
        return recurse_into(obj, baseaction, basetype)
    return f

def recursive_strip_whitespace(obj):
    clean_whitespace = generate_recurse(strip)
    return clean_whitespace(obj)

Problem with Current Solution: It looks awfully terse, it's hard to understand what is going on for people who didn't write it, and I really hope there is more readable way to do this. Or is this honestly the best it's going to get?

Answer 1

My suggestion is to cut down on some redundant code:

def strip(obj):
    return obj.strip()

def recurse_into(obj, baseaction, basetype=str):
    if isinstance(obj, basetype):
        return baseaction(obj)
    elif isinstance(obj, list):
        return [recurse_into(o, baseaction, basetype) for o in obj]
    elif isinstance(obj, tuple):
        return tuple(recurse_into(o, baseaction, basetype) for o in obj)
    elif isinstance(obj, dict):
        return dict((k, recurse_into(v, baseaction, basetype)) 
                    for (k, v) in obj.items())
    return obj

def recursive_strip_whitespace(obj):
    return recurse_into(obj, strip)

Answer 2

The same approach "reversed" would be to split cases in separate functions and map them out. It's less metamagical, but might look more readable.

def strip_obj(obj):
    return obj.strip()
def strip_tuple(tuple):
    return tuple(recurse(obj) for obj in tupl)
...
def recurse(root):
    actions = {basetype: strip_obj,
              tuple: strip_tuple,
              ...}
    return actions[type(root)](root)

Note that for iterables you could "go functional" with map , but I personally find it too dense. Similarly, you could get your meta-juju back by using lambda in those actions values, but again it wouldn't be great for readability.

Answer 3

Here is a simpler and more elegant solution, that preserves the inner type structure of the nested list:

def isiter(x):
    return hasattr(x, '__iter__') and not isinstance(x, (str, bytes))

def apply_recursively(x, func, args=[], kwargs={}):
    if not isiter(x):
        return func(x, *args, **kwargs)
    else:
        ls_type = type(x)
        ls = [apply_recursively(item, func, args, kwargs) for item in x]
        return ls_type(ls)

For example:

>>> a = ['1', '2', '34', ('5', '67', ['8','10'])]
>>> apply_recursively(a, int)
[1, 2, 34, (5, 67, [8, 10])]

>>> b = ['forest', 'mountain', ('grass', 'cow', ['leaf','spider'])]
>>> apply_recursively(b, (lambda s, x: s+x), ['_thing'])
['forest_thing', 'mountain_thing', 
 ('grass_thing', 'cow_thing', 'leaf_thing', 'spider_thing'])]