在python中获取类的路径或名称空间，即使它是嵌套的

Question

I'm currently writing a serialization module in Python that can serialize user defined classes. in order to do this I need to get the full name space of the object and write it to a file. I can then use that string to recreate the object.

for example assume that we have the following class structure in a file named A.py

class B:
    class C:
        pass

now with the assumption that my_klass_string is the string "A::B::C"

klasses = my_klass_string.split("::")
if globals().has_key(klasses[0]):   
    klass = globals()[klasses[0]]
else:
    raise TypeError, "No class defined: %s} " % klasses[0]
if len(klasses) > 1:
    for klass_string in klasses:
        if klass.__dict__.has_key(klass_string):
            klass = klass.__dict__[klass_string]
        else:
            raise TypeError, "No class defined: %s} " % klass_string            
klass_obj = klass.__new__(klass)

I can create an instance of the class C even though it lies under class B in the module A . the above code is equivalent to calling eval(klass_obj = ABC__new__(ABC))

note: I'm using __new__() here because I'm reconstituting a serialized object and I don't want to init the object as I don't know what parameters the class's __init__ methods takes. I want to create the object with out calling init and then assign attributes to it later.

any way I can create an object of class ABC from a string. bout how do I go the other way? how to I get a string that describes the full path to the class from an instance of that class even if the class is nested?

Answer 1

You can't, in any reasonable non-crazy way. I guess you could find the class name and the module, and then for each class name verify that it exist in the module, and if not, go through all classes that does exist in the module in a hierarchical way until you find it.

But since there is no reason to ever have class hierarchy like that, it's a non-problem. :-)

Also, I know you don't want to hear this at this point in your work, but:

Cross-platform serialization is an interesting subject, but doing it with objects like this is unlikely to be very useful, as the target system must have the exact same object hierarchy installed. You must therefore have two systems written in two different languages that are exactly equivalent. That's almost impossible and likely to not be worth the trouble.

You would for example not be able to use any object from Pythons standard library, as those don't exist in Ruby. The end result is that you must make your own object hierarchy that in the end use only basic types like strings and numbers. And in that case, your objects have just become containment for basic primitives, and then you can just as well serialize everything with JSON or XML anyway.

Answer 2

You cannot get the "full path to the class given an instance of the class", for the reason that there is no such thing in Python. For instance, building on your example:

>>> class B(object):
...     class C(object):
...             pass
... 
>>> D = B.C
>>> x = D()
>>> isinstance(x, B.C)
True

What should the "class path" of x be? D or BC ? Both are equally valid, and thus Python does not give you any means of telling one from the other.

Indeed, even Python's pickle module has troubles pickling the object x :

>>> import pickle
>>> t = open('/tmp/x.pickle', 'w+b')
>>> pickle.dump(x, t)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/pickle.py", line 1362, in dump
    Pickler(file, protocol).dump(obj)
  ...
  File "/usr/lib/python2.6/pickle.py", line 748, in save_global
   (obj, module, name))
  pickle.PicklingError: Can't pickle <class '__main__.C'>: it's not found as __main__.C

So, in general, I see no other option than adding an attribute to all your classes (say, _class_path ), and your serialization code would look it up for recording the class name into the serialized format:

class A(object):
  _class_path = 'mymodule.A'
  class B(object):
    _class_path = 'mymodule.A.B'
    ...

You can even do this automatically with some metaclass magic (but also read the other comments in the same SO post for caveats that may apply if you do the D=BC above).

That said, if you can limit your serialization code to (1) instances of new-style classes, and (2) these classes are defined at the top-level of a module, then you can just copy what pickle does (function save_global at lines 730--768 in pickle.py from Python 2.6).

The idea is that every new-style class defines attributes __name__ and __module__ , which are strings that expand to the class name (as found in the sources) and the module name (as found in sys.modules ); by saving these you can later import the module and get an instance of the class:

__import__(module_name)
class_obj = getattr(sys.modules[module_name], class_name)

Answer 3

I'm currently writing a serialization module in Python that can serialize user defined classes.

Don't . The standard library already includes one. Depending on how you count, actually, it includes at least two ( pickle and shelve ).

Answer 4

There are two ways of doing this.

Solution 1

The first one goes via the garbage-collector.

B -> __dict__ -> C

this is the code:

>>> class B(object):
    class C(object):
        pass

>>> gc.get_referrers(B.C) # last element in the list
[<attribute '__dict__' of 'C' objects>, <attribute '__weakref__' of 'C' objects>, (<class '__main__.C'>, <type 'object'>), {'__dict__': <attribute '__dict__' of 'B' objects>, '__module__': '__main__', '__weakref__': <attribute '__weakref__' of 'B' objects>, 'C': <class '__main__.C'>, '__doc__': None}] 

>>> gc.get_referrers(gc.get_referrers(B.C)[-1]) # first element in this list
[<class '__main__.B'>, [<attribute '__dict__' of 'C' objects>, <attribute '__weakref__' of 'C' objects>, (<class '__main__.C'>, <type 'object'>), {'__dict__': <attribute '__dict__' of 'B' objects>, '__module__': '__main__', '__weakref__': <attribute '__weakref__' of 'B' objects>, 'C': <class '__main__.C'>, '__doc__': None}]]

>>> gc.get_referrers(gc.get_referrers(B.C)[-1])[0]
<class '__main__.B'>

Algorithm:

1. search for a class dictionairy with the same __module__ as C
2. take the class, use the 'C' attribute
3. if this class is nested. do 1. recurively

Solution 2

use the source file. use inspect to get the lines of the class and scan upwards for new classes that nest it.

Note: I know no clean way in python 2, but python 3 provides one.