如何将Newick树格式转换为树状分层对象？

Question

I want to convert a Newick file to a hierarchical object (similar to what has been posted in this post ) in Python.

My input is a Newick file like this:

(A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5)F:0.9

The original post parses the string character by character. To store the branch lengths also, I have modified the JavaScript file (from here ) as follows:

 var newick = '// (A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5)F:0.9', stack = [], child, root = [], node = root; var na = ""; newick.split('').reverse().forEach(function(n) { switch(n) { case ')': // ')' => begin child node if (na != "") { node.push(child = { name: na }); na = ""; } stack.push(node); child.children = []; node = child.children; break; case '(': // '(' => end of child node if (na != "") { node.push(child = { name: na }); na = ""; } node = stack.pop(); // console.log(node); break; case ',': // ',' => separator (ignored) if (na != "") { node.push(child = { name: na }); na = ""; } break; default: // assume all other characters are node names // node.push(child = { name: n }); na += n; break; } }); console.log(node); 

Now, I want to translate this code to Python.

Here's my attempt (I know it's incorrect):

class Node:

  def __init__(self):
    self.Name = ""
    self.Value = 0
    self.Children = []

newick = "(A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5,G:0.8)F:0.9"
stack = []
# root = []
# node = []

for i in list(reversed(newick)):
  if i == ')':
    if na != "":
      node = Node()
      node.Name = na
      child.append(node)
      na = ""
    stack.append(node)
    # insert logic
    child = node.Children
    # child.append(child)

  elif i == '(':
    if (na != ""):
      child = Node()
      child.Name = na
      node.append(child)
      na = ""
    node = stack.pop()
  elif i == ',':
    if (na != ""):
      node = Node()
      node.Name = na
      node.append(child)
      na = ""
  else:
    na += n

Since I am totally new to JavaScript, I am having trouble 'translating' the code to Python. In particular, I didn't understand the following lines:

child.children = [];
node = child.children;

How can I correctly write this in Python, to also extract the lengths?

Answer 1

The following code might not be an exact translation of the javascript code, but it works as expected. There were some issues like "n" to be not defined. I also added the parsing of the node name into name and value, and a parent field.

You should consider using already existent parsers like https://biopython.org/wiki/Phylo as they already give you the infrastructure and algorithms to work with the trees.

class Node:
    # Added parsing of the "na" variable to name and value.
    # Added a parent field
    def __init__(self, name_val):
        name, val_str = name_val[::-1].split(":")
        self.name = name
        self.value = float(val_str)
        self.children = []
        self.parent = None

    # Method to get the depth of the node (for printing)
    def get_depth(self):
        current_node = self
        depth = 0
        while current_node.parent:
            current_node = current_node.parent
            depth += 1
        return depth

    # String representation
    def __str__(self):
        return "{}:{}".format(self.name, self.value)

newick = "(A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5,G:0.8)F:0.9"

root = None
# na was not defined before.
na = ""
stack = []
for i in list(reversed(newick)):
    if i == ')':
        if na != "":
            node = Node(na)
            na = ""
            if len(stack):
                stack[-1].children.append(node)
                node.parent = stack[-1]
            else:
                root = node
            stack.append(node)

    elif i == '(':
        if (na != ""):
            node = Node(na)
            na = ""
            stack[-1].children.append(node)
            node.parent = stack[-1]
        stack.pop()
    elif i == ',':
        if (na != ""):
            node = Node(na)
            na = ""
            stack[-1].children.append(node)
            node.parent = stack[-1]
    else:
        # n was not defined before, changed to i.
        na += i

# Just to print the parsed tree.
print_stack = [root]
while len(print_stack):
    node = print_stack.pop()
    print(" " * node.get_depth(), node)
    print_stack.extend(node.children)

The output of the print bit at the end is the following:

 F:0.9
  A:0.1
  B:0.2
  E:0.5
   C:0.3
   D:0.4
  G:0.8

Answer 2

Some comments on the JavaScript version:

• It has some code repetition ( if (na != '') ... ) which is easy to avoid.
• It uses node as variable name for an array. Readability is improved when you use a plural word for arrays (or lists in Python).
• It does not output what you want to have: it outputs nodes with names like "9.0:F", not isolating the length from the name.

Because of the last point, the code needs first to be corrected before making the translation into Python. It should support splitting the name/length attributes, allowing either of them to be optional. Additionally, it could assign id values to each created node and add a parentid property to refer to a node's parent.

I personally prefer coding with recursion instead of using a stack variable. Also, with a regular expression API you can easily tokenise the input to facilitate the parsing:

JavaScript version of Newick format parser

 function parse(newick) { let nextid = 0; const regex = /([^:;,()\\s]*)(?:\\s*:\\s*([\\d.]+)\\s*)?([,);])|(\\S)/g; newick += ";" return (function recurse(parentid = -1) { const children = []; let name, length, delim, ch, all, id = nextid++;; [all, name, length, delim, ch] = regex.exec(newick); if (ch == "(") { while ("(,".includes(ch)) { [node, ch] = recurse(id); children.push(node); } [all, name, length, delim, ch] = regex.exec(newick); } return [{id, name, length: +length, parentid, children}, delim]; })()[0]; } // Example use: console.log(parse("(A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5,G:0.8)F:0.9")); 

 .as-console-wrapper { max-height: 100% !important; top: 0; } 

Python version of Newick format parser

import re

def parse(newick):
    tokens = re.findall(r"([^:;,()\s]*)(?:\s*:\s*([\d.]+)\s*)?([,);])|(\S)", newick+";")

    def recurse(nextid = 0, parentid = -1): # one node
        thisid = nextid;
        children = []

        name, length, delim, ch = tokens.pop(0)
        if ch == "(":
            while ch in "(,":
                node, ch, nextid = recurse(nextid+1, thisid)
                children.append(node)
            name, length, delim, ch = tokens.pop(0)
        return {"id": thisid, "name": name, "length": float(length) if length else None, 
                "parentid": parentid, "children": children}, delim, nextid

    return recurse()[0]

# Example use:
print(parse("(A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5,G:0.8)F:0.9"))

About the assignment node = child.children in your JavaScript code: this moves the "pointer" (ie node ) one level deeper in the tree that is being created so that in the next iteration of the algorithm any new nodes are appended at that level. With node = stack.pop() that pointer tracks back one level up in the tree.

Answer 3

Here is a pyparsing parser for this input string. It uses pyparsing's nestedExpr parser builder, with a defined content argument so that the results are parsed key-value pairs, not just simple strings (which is the default).

import pyparsing as pp
# suppress punctuation literals from parsed output
pp.ParserElement.inlineLiteralsUsing(pp.Suppress)

ident = pp.Word(pp.alphas)
value = pp.pyparsing_common.real

element = pp.Group(ident + ':' + value)
parser = pp.OneOrMore(pp.nestedExpr(content=pp.delimitedList(element) + pp.Optional(','))
                      | pp.delimitedList(element))

tests = """
    (A:0.1,B:0.2,(C:0.3,D:0.4)E:0.5)F:0.9
"""
parsed_results = parser.parseString(tests)
import pprint
pprint.pprint(parsed_results.asList(), width=20)

Gives:

[[['A', 0.1],
  ['B', 0.2],
  [['C', 0.3],
   ['D', 0.4]],
  ['E', 0.5]],
 ['F', 0.9]]

Note that the pyparsing expression for parsing reals also does parse-time conversion to Python floats.