Create objects of custom class from YAML in Python

Question

I have Class definition:

Class definition

class Task(object):
    def __init__(self, name, *depends):
        self.__name    = name
        self.__depends = set(depends)

    @property
    def name(self):
        return self.__name

    @property
    def depends(self):
        return self.__depends

The rest of the code is using sample definition like this:

a = Task("a", "b")
b = Task("b")

nodes = (a, b)

I am trying to create this objects dynamically by parsing YAML in separate function which returns dictionary data and then I create all the objects:

Yaml File

a:
  foo:
   - "bar"
  graph_dep:
  - "b"
b:
  foo:
   - "bar"

I believe I was able to achieve creation of a and b objects of a Task class via function

def format_input(data):
    services = data.keys()
    holder = {Task(name=name) for name in services}
    return holder

q1: How do I combine them to get nodes object and have same result as using sample definitions? q2: How do I get values from graph_dep and add them in holder = {Task(name=name) for name in services} string?

Sorry newbie question in Python:)

Answer 1

Generally, if you don't need the YAML's original structure, it is unnecessary to load it into native Python types. Instead, you should only load them into a YAML node graph and then load the native structure you want to have from that:

import yaml

input = """
a:
  foo:
   - "bar"
  graph_dep:
  - "b"
b:
  foo:
   - "bar"
"""

class Task(object):
  def __init__(self, name, *depends):
    self.__name    = name
    self.__depends = set(depends)

  @property
  def name(self):
    return self.__name

  @property
  def depends(self):
    return self.__depends

  def __str__(self):
    return "Task({}, {})".format(self.__name, self.__depends)

def load_dependency_list(loader, node):
  for item in node.value:
    if item[0].value == "graph_dep":
      return loader.construct_sequence(item[1])
  return []

def load_tasks(loader, node):
  ret = ()
  for item in node.value:
    name = loader.construct_scalar(item[0])
    deplist = load_dependency_list(loader, item[1])
    ret += (Task(name, *deplist),)
  return ret

loader = yaml.SafeLoader(input)
nodes = load_tasks(loader, loader.get_single_node())
for node in nodes:
  print(node)

What this code does is:

• getting the node graph of the single document in the YAML file via get_single_node()
• on the returned root node, execute your loading code

load_tasks and load_dependency_list handle the levels in the YAML. PyYAML does provide facilities to register these functions as constructors , however that only makes sense if you have tags in your YAML input (like eg --- !nodes as first line). Without tags, PyYAML cannot automatically map constructors so you need to implement the construction process yourself.

In this case, load_dependency_list constructs the list contained in graph_dep or returns the empty list, ignoring other things in the YAML node. load_tasks iterates over the top level items, constructing Task object from each of them, and concatenates them into a tuple.

I added a __str__ method to Task for output. The output is

Task(a, {'b'})
Task(b, set())

This code has no safeguards whatsoever and I advise to check for proper node types whereever you access a YAML node so that the user gets a good error message if the structure is wrong.