how to map a row of a csv file to specific rows in python

Question

I have a strange looking csv file. It has sub sections and each sub section has csv formatted data.

Data eg:

Device Name: Test-0001

Sub section 1
header 1, header 2, header 3,header 4,.......header n
field 1, field 2, field 3, field 4, .......field n

Sub section 2
header 1, header 2, header 3,header 4,.......header n
field 1, field 2, field 3, field 4, .......field n

.....

Sub section n

header 1, header 2, header 3,header 4,.......header n
field 1, field 2, field 3, field 4, .......field n

Device Name: Test2-0002
and same format afterwards....

I am looking to map each "Device Name: " to all the sub sections until next "Device Name :" comes

How can I do it?

Answer 1

I would try to define a dictionary taking your devices as keys and a list of lines as values. Assuming that lines is a list of the lines of your file:

with open(your_file_name, "r") as f:
    lines = f.readlines()

devices = {}
key = None
for line in lines:
    if not line.strip():
        continue
    if line.startswith("Device Name"):
        key = get_key(line)
        devices[key] = []
    else:
        devices[key].append(line)

where get_key is a function you would define, that would transform a line like Device Name: Test2-0002 into something you want.

For example, if you want your key like Device Name: Test2-0002 , you could define get_key as get_key = lambda line: line : it's a function that when given a line will return exactly the same line without any processing.

If you want your key to be like Test2-0002 , you could write your get_key function to remove the Device Name: string, or take the part of the string after the ':' ...

It's rather basic a solution but should work if you don't mind storing the lines of your file at once. If this is a problem, just use a file.readline() instead of the lines list.

A slightly more advanced solution could be based on the mmap module: find the positions of your Device Name lines and store them. You can then read section by section.