Read txt file data as dictionary

Question

I am new to python. I have a txt file which contains input data for a 3D model:

...
# comments
rfk = [1e-5, 1e-7, 1e-1]
rsig = [0.0005]
# comments
lengths = [[13,13,1.6],[13,13,1.6],[13,13,1.6]]
...

I guess the best way to extract the data for further processing is to save them into dictionaries - like this:

data1 = {rfk:[1e-5, 1e-7, 1e-1], rsig:[0.0005], lengths:[[13,13,1.6],[13,13,1.6],[13,13,1.6]]}

I now struggle a little bit with reading the file and saving the data (by ignoring the comments) like in my example as dictionaries. My approach for reading the data from the file was like this:

for line in open("myfile.txt"):
    li=line.strip()
    if not li.startswith("#"):        
        # what to do here?

Answer 1

We can use ast.literal_eval to evaluate the right hand side into Python objects

from ast import literal_eval

with open('myfile.txt') as file:
    d = {}
    for line in file:
        line = line.strip()
        if not line.startswith('#'):
            key, value = map(str.strip, line.split('='))
            d[key] = literal_eval(value)

For your example data, d is then

{'rfk': [1e-05, 1e-07, 0.1], 'rsig': [0.0005], 'lengths': [[13, 13, 1.6], [13, 13, 1.6], [13, 13, 1.6]]}

Answer 2

You can use the string method partition * to seperate your data to key and value for your dictionary,
The string method strip to clean up your key's string from leading and tailing spaces
And then the built-in function eval to extract the pythonic object from the value's string, as such:

with open('myfile.txt') as file:
    d = {}
    for line in file:
        line = line.strip()
        if not line.startswith('#'):
            key, _ ,value = line.partition('=')
            d[key.strip()] = eval(value)

* partition splits your string in the first occurance of your delimiter (which is '=' in this case)
and returns a list of [leading string, delimiter, following string]

Answer 3

You can treat the at as json and add a few safety checks:

import json

data = {}
with open('thefile.txt') as fobj:
    for raw_line in fobj:
        line = raw_line.strip()
        if line and not line.startswith('#'):
            try:
                key, values = line.split('=', 1)
            except ValueError:
                print('skipping invalid line:')
                print(line)
            try:
                data[key.strip()] = json.loads(values)
            except json.JSONDecodeError
                print('skipping invalid line (no JSON):')
                print(line)

for your example data contains:

{'lengths': [[13, 13, 1.6], [13, 13, 1.6], [13, 13, 1.6]],
 'rfk': [1e-05, 1e-07, 0.1],
 'rsig': [0.0005]}