Convert string representation of list of lists to list of list python without eval

Question

I have this:

x = "[['ATRM', 'SIF', 'NWPX'], ['NAV','SENEA'], ['HES','AGYS', 'CBST', 'GTIM', 'XRSC']]"

x is a string. and I want this:

x = [['ATRM', 'SIF', 'NWPX'], ['NAV','SENEA'], ['HES','AGYS', 'CBST', 'GTIM', 'XRSC']]

where x is a list.

I would normally use eval or ast.literal_eval but those functions are unavailable. Any ideas? Maybe I can use re , but I don't know how.

Answer 1

This is an odd workaround, but if you replace the single quotes with double quotes, could always use a json parser.

>>> import json
>>> json.loads(x.replace("'", '"'))
[['ATRM', 'SIF', 'NWPX'], ['NAV', 'SENEA'], ['HES', 'AGYS', 'CBST', 'GTIM', 'XRSC']]

Answer 2

Imo, you need to write your own little parser here, eg:

def tokenizer(string):
    buffer = ""
    quote = False
    for c in string:
        if quote:
            if c == "'":
                yield ("VALUE", buffer)
                buffer = ""
                quote = not quote
            else:
                buffer += c
        else:
            if c == "[":
                yield ("LIST_OPEN", None)
            elif c == "]":
                yield ("LIST_CLOSE", None)
            elif c == "'":
                quote = not quote
            else:
                pass


def parser(tokens):
    lst = []
    for token in tokens:
        x, y = token
        if x == "LIST_OPEN":
            lst.append(parser(tokens))
        elif x == "LIST_CLOSE":
            return lst
        elif x == "VALUE":
            lst.append(y)
    return lst[0]

With some test assertions:

assert parser(tokenizer("['HES', ['ATRM', 'SIF', 'NAV']]")) == ['HES', ['ATRM', 'SIF', 'NAV']]
assert parser(tokenizer("[['ATRM', 'SIF', 'NWPX'], ['NAV','SENEA'], ['HES','AGYS', 'CBST', 'GTIM', 'XRSC']]")) == [['ATRM', 'SIF', 'NWPX'], ['NAV','SENEA'], ['HES','AGYS', 'CBST', 'GTIM', 'XRSC']]

---

The idea is to first tokenize your string into values and commands and then convert this to an actual list.

Answer 3

I acknowledge this is a very janky and limiting answer because it would only work with the given information based on the example text:

def list_list_str_to_list(data_str):
    final_word_list_list = []
    for temp_list_as_str in data_str.split("],"):
        final_word_list = []
        for raw_word in temp_list_as_str.split(","):
            new_word = raw_word
            for letter in "[],'\"":
                new_word = new_word.replace(letter, "")
            final_word_list.append(new_word)
        final_word_list_list.append(final_word_list)
    return final_word_list_list


def main():
    data_str = "[['ATRM', 'SIF', 'NWPX'], ['NAV','SENEA'], ['HES','AGYS', 'CBST', 'GTIM', 'XRSC']]"

    for final_word_list in list_list_str_to_list(data_str):
        print(final_word_list)


main()

The main idea it works off of is that you can tell a list ends by splitting the string when there is an instance of "],". The bulk of the code is just cleaning the words by removing unwanted trailing characters like brackets, quotes, and spaces. To reiterate, this would only work if:

1. The string is a string representation of ONLY a 2d list and
2. There are no brackets or single/double quotes within the individual strings

Answer 4

Ok, I think found the answer using re.

x = "[['ATRM', 'SIF', 'NWPX'], ['NAV','SENEA'], ['HES','AGYS', 'CBST', 'GTIM', 'XRSC']]"

y = [re.findall(r"\[(.+?)\]", x[1:])[i] for i in range(x.count('[')-1)]

answer = [re.findall(r"'(.+?)'", y[i]) for i in range(len(y))]