Python: parsing a config file to dictionary

Question

I'm trying to parse a config file for a fortran model to a python dictionary. The config file is basically a number of lines containing arrays or strings eg:

0, 400, 700, 0.02, 488, 0.00026, 1, 5.3
rootname
filename
0,1,0,0,0,1
2,1,0,2,3
4,4
0.0, 0.0, 2980.9579870417047, 0.01,
...
...
...

The line number and array index tell me to which variable the value belongs. So I decided to make a dictionary where the keys describe the variable and the values the indices of where the variable can be found in the config file eg:

parameters = {"var1": [3,2],
              "var2": [4,1],
              "var3": [4,2]}

So if I would read the config file as a list with .read() I could create a dictionary with the parameter values as follows:

def get_parameters(config_list):
    dict_out = {}
    for key, value in parameters.items():
        dict_out[key] = config_list[value[0]][value[1]]
    return dict_out

The problem however is that the config file has a dynamic number of lines depending on the number of components in the model. Fortunately, the number of components (another variable) is also specified in the config file. Let assume the first 7 lines are static; after the 7th line the number of lines are depended on the number of components. The number of components ncomp is 3 (specified on the 5th line and the 5th entry in the array ([4][4]). Now i want to retrieve variable var4 on line 7+ ncomp +1 and array index 2 . How do I go about this in an elegant way?

I thought about adding lambda expressions to my parameter dictionary:

parameters = {"var1": [3,2],
              "var2": [4,1],
              "var3": [4,2],
              "ncomp": [4,4],
              "var4": [lambda ncomp: ncomp+7,2]}

But this would mean that I first have to retrieve ncomp and then evaluate the lambda function to get the indices. With the indices I could then retrieve the values of var4 . It sounds doable but I feel like there might me a more elegant way to solve this problem. Suggestions?

Answer 1

You can do it in two steps.

First all parameters but var4 :

parameters = {"var1": [3,2],
              "var2": [4,1],
              "var3": [4,2],
              "ncomp": [4,4]}

out = get_parameters(config_list)

Now var4 :

out["var4"] = config_list[int(out['ncomp']) + 7, 2]

Answer 2

Update :
Since you have well defined dependencies, you will have to set up the loop accordingly while fetching parameters in the dictionary.There's no alternative as I see it.
You do not have to bother about defining your dependent parameters in the 'parameters' dictionary since many params are dynamically read and obtained.
So, only define the independent ones in 'parameters'
Suppose there are 2 dependency defining params like "ncomp" and "ncomp2" . And there are 2 dependent params for each, say var10, var11, var12 and var13.
While var14 and var15 are dependent on both "ncomp" and "ncomp2" .
You'll now group them into the respective batches as follows.

def get_parameters(config_list):
    dict_out = {}
    for key, value in parameters.items():
        if key == "ncomp":  # for all the params dependent on number of components.
            dict_out[key] = config_list[value[0]][value[1]]
            ncomp_val = dict_out[key] # we now have the value of ncomp in out dictionary available for all its dependent parameters.

            dict_out["var10"] = config_lit[ncomp_val*2 +1][4]
            dict_out["var11"] = config_lit[ncomp_val*3 +1][2]

         elif key == "ncomp2":  # for all the params dependent on number of components.
            dict_out[key] = config_list[value[0]][value[1]]
            ncomp1_val = dict_out[key]

            dict_out["var12"] = config_lit[ncomp1_val*2 -1][3]
            dict_out["var13"] = config_lit[ncomp1_val*3 +3][4]

         elif "ncomp" in dict_out and "ncomp2" in dict_out and "var14" not in dict_out:  # for the multi-dependency params, dependent on ncomp and ncomp2.
            ncomp_val = dict_out["ncomp"]
            ncomp1_val = dict_out["ncomp1"]

            dict_out["var14"] = config_lit[ncomp_val*2 + ncomp1_val -1][3]
            dict_out["var15"] = config_lit[ncomp1_val*3 + ncomp1_val*2 +3][4]

        else:  # for all other independent params.
            dict_out[key] = config_list[value[0]][value[1]]

    return dict_out

Original Response :
I agree with @Mike Muller's answer, however, you can avoid breaking it into 2 steps.
While retrieving the parameters into your dictionary, you can easily check for the "ncomp" key and proceed.
Let's work with your assumption of 7 static lines, and by then, you will have gotten your "ncomp" value into your dictionary.
You can now capture all your dependent parameters based on this value as follows.

def get_parameters(config_list):
    dict_out = {}
    for key, value in parameters.items():
        if "ncomp" in dict_out:  # for all the params dependent on number of components.
            dict_out[key] = [dict_out["ncomp"]+7+1 , 2]
        else:  # for all other independent params.
            dict_out[key] = config_list[value[0]][value[1]]
    return dict_out

Answer 3

@murphy1310 thanks for your thorough answer. However I am not a fan of hardcoding all the indices in the function itself. I would rather keep all the indices of the variables in a (single) dictionary so i can keep the function "clean". I came up with the following:

# config_list is a 2D list of parameter values and strings
config_list = [[...],
               [...],
               [...]]

# dictionary containing indices of where variables can be found in config_list
parameters = {
              # independent variables
              "var1": [3,2],
              "var2": [4,1],
              "var3": [4,2],
              "ncomp": [4,4],
              # dependent variables
              "var4": lambda ncomp: [ncomp+1, 1],
              "var5": lambda ncomp: [ncomp*2+1, 1]}

def get_parameters(dictin, dictout = {}):
    dictin_copy = dictin.copy()
    for key, value in dictin_copy.items():
        if not callable(value) and not dictout.get(key):
            dictout[key] = config_list[value[0]][value[1]]

    ncomp = dictout["ncomp"]
    for key, value in dictin_copy.items():
        if callable(value):
            dictin_copy[key] = value(ncomp)
            get_parameters(dictin_copy, dictout)
    return dictout

# now get parameters from config_list
new_dict = get_parameters(parameters)

I'm wondering what you guys think of this approach...