How to check if an data already present in a CSV file using Python

Question

Good day all,

I am trying to write a python script that picks up IP address(es) from a textfile, check if the IP addresses already present in CSV file, if IP address is already present then do nothing (pass), if the IP is not present in the CSV file then append the IP address in the CSV file.

I can add the IP addresses to the CSV file but I am not able to get my head around with how to check if the IP address is already present in the file.

if os.path.isfile('IP_file.txt'):
    logfile = open('IP_file.txt', 'r')
    csv_file = open('csv_file.csv', 'ab')
    for line in logfile:
        if line in csv_file: # This is not working
            pass
        else:
            csv_file.write(line)
    csv_file.close()
    logfile.close()
else:
    print('No file with filename ' logfile 'created')

I also tried to use the CSV module but no luck. Any help is much appreciated.

Answer 1

CSV is just a text format. Read it line by line, test every line against your IP. Like

ip="192.168.1.1"
for line in csv_file:
    if ip in line:
        found = True
        break

I am a bit concerned against your cycling over file on the disk. Probably it is better to read files into memories (find all IPs in CSV file first and put in a list) and then check against the list instead of opening and iterating over whole CSV file for every line of IP file.

Answer 2

Keep a list or dict that holds the unique fetched values:

found_address = []
with open('IP_file.txt', 'r') as logfile:
    with open('csv_file.csv', 'ab') as csv_file:
        for line in logfile:
            if not line in found_address:
                csv_file.write(line)
                found_address.append(line)

Answer 3

You are opening csv_file.csv in append mode ab . And then you are trying to check line in csv_file . In append-mode file-pointer points to the end-of-file.

Answer 4

little crazy functional-style python:

ip_file = set(map(str.strip, open('IP_file.txt').readlines()))

if CSV file is: 192.168.1.1,192.168.1.2,192.168.1.3,...

csv_file = set(map(str.strip, open('csv_file.csv').read().split(',')))
diff_ip = ip_file - csv_file
open('csv_file.csv','a').write(''.join(map(lambda x: ',{}'.format(x), list(diff_ip))))