How to increment value in dictionary using python

Question

I have these lines inside text file:

2018-11-06 16:52:01.901| on thread[140447603222272 c0s0]| IP[192.168.0.244:5000]| master| 192.168.0.244| omer| (stmt : 0) | admin|  Success
2018-11-06 16:52:15.160| on thread[140447603222272 c0s1]| IP[192.168.0.244:5001]| master| 192.168.0.244| userAdmin| (stmt : 1) | admin|  Success
2018-11-06 16:52:27.351| on thread[140447603222272 c0s3]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 3) | admin|  Success
2018-11-06 16:52:28.159| on thread[140447603222272 c0s5]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 5) | admin|  Success
2018-11-06 16:52:30.485| on thread[140447603222272 c0s7]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 7) | admin|  Success
2018-11-06 16:52:54.217| on thread[140447603222272 c0s11]| IP[192.168.0.244:5000]| master| 192.168.0.244| Gilc| (stmt : 11) | admin|  Success
2018-11-06 16:53:04.600| on thread[140447603222272 c10s12]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 12) | admin|  Success
2018-11-06 16:53:05.617| on thread[140447603222272 c10s13]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 13) | admin|  Success
2018-11-06 16:53:18.958| on thread[140447603222272 c10s14]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 14) | admin|  Success
2018-11-06 16:53:26.25| on thread[140447603222272 c10s15]| IP[192.168.0.244:5000]| master| 192.168.0.244| Guy| (stmt : 15) | admin|  Success
2018-11-06 16:53:36.816| on thread[140447603222272 c10s17]| IP[192.168.0.244:5005]| master| 192.168.0.244| userAdmin| (stmt : 17) | admin|  Success
2018-11-06 16:53:41.116| on thread[140447603222272 c13s18]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 18) | admin|  Success
2018-11-06 16:53:51.517| on thread[140447603222272 c13s19]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 19) | admin|  Success
2018-11-06 16:53:55.500| on thread[140447603222272 c13s21]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 21) | admin|  Success
2018-11-06 16:53:56.333| on thread[140447603222272 c13s23]| IP[192.168.0.244:5000]| master_backup| 192.168.0.244| userAdmin| (stmt : 23) | admin|  Success
2018-11-06 16:54:03.758| on thread[140447603222272 c13s25]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 25) | admin|  Success
2018-11-06 16:54:07.891| on thread[140447603222272 c13s26]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 26) | admin|  Success
2018-11-06 16:54:22.66| on thread[140447603222272 c20s27]| IP[192.168.0.244:5000]| master| 192.168.0.244| ben| (stmt : 27) | admin|  Success
2018-11-06 16:54:27.849| on thread[140447603222272 c20s28]| IP[192.168.0.216:5000]| master_tati| 192.168.0.244| userAdmin| (stmt : 28) | admin|  Success
2018-11-06 16:54:36.533| on thread[140447603222272 c22s29]| IP[192.168.0.244:5000]| master| 192.168.0.244| userAdmin| (stmt : 29) | admin|  Success

I want to count for each user how many lines he got for example I want to get this dictionary for user 'userAdmin' -> dict = {'userAdmin: 16} because there are 16 lines with 'userAdmin'

I have implemented the following code:

def parse_log_file(log_file):
    print(len(""))
    my_path = os.path.abspath(os.path.dirname(__file__))
    path = os.path.join(my_path, log_file)
    with open(path, 'r') as f:
        lines = f.readlines()[1:]
        map = {}
        for line in lines:
            elements = line.strip().lstrip().split('|')
            if elements[8].lstrip() == 'Success':
                map[elements[5].lstrip()] = 1
                if map[elements[5].lstrip()] is not None:
                    map[elements[5].lstrip()] += 1
                # map[elements[5]] = elements[8]

        for k, v in map.items():
            print(k, ':', v)

But this is what I got which is wrong:

omer : 2
userAdmin: 2
Gilc : 2
Guy : 2
ben : 2

i would expect userAdmin to be 16 and not 2

Answer 1

You're overwriting the entry inside of your dictionary inside if your for-loop.

Try this instead:

def parse_log_file(log_file):
    my_path = os.path.abspath(os.path.dirname(__file__))
    path = os.path.join(my_path, log_file)
    with open(path, 'r') as f:
        lines = f.readlines()[1:]
        d = {}
        for line in lines:
            elements = line.strip().lstrip().split('|')
            if elements[8].lstrip() == 'Success':
                try:
                    d[elements[5].lstrip()] += 1
                except KeyError:
                    d[elements[5].lstrip()] = 1

        for k, v in d.items():
            print(k, ':', v)

I've changed your dictionary from map to d because map is a built-in function, and you should avoid overwriting those, because they can create bugs that are hard to track down.

The output of the above code is:

userAdmin : 16
Gilc : 1
Guy : 1
ben : 1

What's happening here is that we're using EAFP logic to try to do +1 to the variable inside of the dictionary. If that fails, we catch the KeyError and then just create a new entry inside of the dictionary.

Answer 2

For every success line, you change the count back to 1 in the line map[elements[5].lstrip()] = 1 .

Instead you would like to initialize the counter to 1 for the first line the user appears, and for the next times just increment the counter by 1.

You can change the logic to the following:

name = elements[5].lstrip()
if name in d.keys():
    d[name] += 1
else:
    d[name] = 1

Answer 3

Improvements that can be made:

• Check for existence of key in dict before assigning, if the key exists, increment the value

• Reduce over-use of lstrip and rstrip , strip does both these at once

• You are advised to change the name of our dictionary from map . That is a python Keyword.

Revised solution:

def parse_log_file(log_file):
    print(len(""))
    my_path = os.path.abspath(os.path.dirname(__file__))
    path = os.path.join(my_path, log_file)
    with open(path, 'r') as f:
        lines = f.readlines()[1:]
        mapping = {}
        for line in lines:
            elements = line.split('|')
            # strip the lines of surrounding spaces
            elements = [t.strip() for t in elements]
            if elements[8].lstrip() == 'Success':
                # check if the key already exists in the dict
                # create key if not exists
                if not mapping.get(elements[5]):
                    mapping[elements[5]] = 1
                else:
                    # increment the value if key exists
                    mapping[elements[5]] += 1

        for k, v in mapping.items():
            print(k, ':', v)

This produces:

userAdmin : 16
Gilc : 1
Guy : 1
ben : 1