ConfigParser 慢吗？ ConfigParser 与丑陋的算法

Question

I was trying to improve a function I created to parse a bunch of INI files into a single JSON file. When I wrote this function I was a newbie and I didn't know configparser module, so now I want to exploit it.

INPUT : INI file (parent) containing a long set of references to other INI files (children).

GOAL : to convert the INI parent file to JSON and include some info taken from all the children files into it. In other words: take some info from a set of INI files and export them into a single JSON file.

Question : I expected my new code to be at least as fast as the old one, but it isn't: it is 2 times slower. Why is that? Is it ConfigParser or is it me? Performance are really important and my function takes one second to parse around 900 INI files, while the old one takes half a second.

Parent Example

(it can have from hundreds of lines to tens of thousands ):

[General]
Name = parent
...
   
[Item 000001]
Name = first item
path = "path/to/child_1.ini"
...

[Item 000002]
Name = second item
...

[...]

[Item 001000]
Name = thousandth item
...   

Child Example

(it can have from less than 100 lines to about 200):

[General]
Name = name
ID = 12345
...

[Options]
...

JSON Output Example

{
    "Parent": {
        "Name": "parent",
        "Count": "1000",
        [...]
        "child1": {
            "Name": "name",
            "ID": "12345",
            "Option1": "...",
            "Option2": "...",
            "Option3": "..." 
        },
        "child2": {
            "Name": "name2",
            "ID": "22222",
            "Option1": "...",
            "Option2": "...",
            "Option3": "..." 
        },
        [...]
        "child1000": {
            "Name": "name1000",
            "ID": "12332",
            "Option1": "...",
            "Option2": "...",
            "Option3": "..." 
        }
    }
}

OLD CODE

def split_string_by_equal(string):
    str_operands = string.split(' = ')
    first_part = (str_operands[0]).strip()
    second_part = (' '.join(str_operands[1:])).strip()
    return [first_part, second_part]

def parse_ini_to_json(path):
    parent_dict = {}
    child_dict = {}
    num_child = 1
    parent_directory = os.path.dirname(testflow)
    with open(path, 'r') as parent_file:
        for line in tfl_file:
            left_part = split_string_by_equal(line)[0]
            right_part = split_string_by_equal(line)[1]
            if left_part in SOME_WORDS:
                parent_dict.update({left_part: do_something(parent_directory, right_part)})
            elif left_part == 'Count':
                parent_dict.update({'Count': right_part})
            elif left_part == 'JohnDoe':
                parent_dict['JohnDoe'] = right_part
            elif 'Item' in line:
                if child_dict:
                    parent_dict.update({'test{}'.format(num_child): child_dict})
                    child_dict = {}
                    num_child += 1
            elif left_part in SOME_OTHER_WORDS:
                child_dict.update({left_part: right_part})
            if left_part == 'path':
                child_dict.update(extract_data_from_child(right_part))
    if child_dict:
        parent_dict.update({'child{}'.format(num_test): child_dict})
    return parent_dict

def extract_data_from_child(path):
    """ same methodology used in above function """
    [...]
    return child_dict

NEW CODE

def get_config_parser(path):
    config = configparser.ConfigParser()
    config.optionxform = str
    config.read(path)
    return config

def parse_ini_to_json(path):
    config = get_config_parser(path)
    parent_directory = os.path.dirname(testflow)
    parent_dict = {}
    for key in config['Folders'].keys():
        parent_dict[key] = do_something( parent_directory, config['Folders'][key])
    parent_dict['Count'] = config['General']['Count']
    parent_dict['JohnDoe'] = config['General']['JohnDoe']
    counter = 1
    for key in config.keys():
        if 'Item' in key:
            child_dict = {}
            for child_prop in config[key].keys():
                if child_prop in SOME_WORDS:
                    child_dict[child_prop] = config[key][child_prop]
            child_path = config[key]['path']
            child_dict.update(extract_data_from_child(child_path))
            child_dict[f'child{counter}'] = child_dict
            counter += 1
    return parent_dict


def extract_data_from_child(path):
    config = sysfunc.get_config_parser(path)
    child_dict = {}
    for key in config['General'].keys():
        if key in SOME_KEYWORDS:
            child_dict[key] = config['General'][key]
    for key in config['Levels'].keys():
        if key in SOME_OTHER_KEYWORDS:
            child_dict[key] = config['Options'][key]
    try:
        some_value = config['Levels']['SomeKey']
    except KeyError:
        pass
    for key in config['Options'].keys():
        value = config['Options'][key]
        key_enabled = key.strip() + 'enabled'
        try:
            if config['Options'][key_enabled] == '0':
                continue
        except KeyError:
            continue
        if 'false' in value:
            value = '0'
        elif 'true' in value:
            value = '1'
        child_dict[key] = value
    return child_dict 

Answer 1

I expected my new code to be at least as fast as the old one, but it isn't: it is 2 times slower. Why is that? Is it ConfigParser or is it me?

It's both. To understand where time is being spent in your code and in ConfigParser, you should look at using a Profiler . https://docs.python.org/3/library/profile.html is a good starting point for learning how to profile code.