read every 25 line of a csv file and pass to list using python

Question

I want to read a file and convert every 25 lines of that file into a list, that is, it should have 4 lists with 25 items in each (for 100 lines of a file). I am not being able to get the code for this problem. The input file looks like this , in actual it has 100 lines:

{'PutRequest': {'Item': {'id': {'S': 'E1DBEAE3'}, 'value': {'M': {'result': {'N': u'0.0015'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': '31C6C'}, 'value': {'M': {'result': {'N': u'0.1129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': '59D40'}, 'value': {'M': {'result': {'N': u'0.00129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': 'A2A9'}, 'value': {'M': {'result': {'N': u'0.05129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}

Also, I want to prepend and append a string to the very first element and the last element of every list respectively, prepend string like :

'{"test":[' and append string like: ']}'

After prepend and append it should look like for a list size of 3 for example:

{"test":[{'PutRequest': {'Item': {'id': {'S': 'E1DBEAE3'}, 'value': {'M': {'result': {'N': u'0.0015'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': '31C6C'}, 'value': {'M': {'result': {'N': u'0.1129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': '59D40'}, 'value': {'M': {'result': {'N': u'0.00129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': 'A2A9'}, 'value': {'M': {'result': {'N': u'0.05129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}]}

I have tried this code:

from itertools import islice
list =[]
with open('output_of_json.json', 'r') as infile:
    lines_gen = islice(infile, 25)
    for line in lines_gen:
        list.append(line)

Unable to go past the first 25 lines of the file

Answer 1

This can be made into a function. It's logically what you want. You can use enumerate to clean it up.

with open(filename,'r') as f:

    counter = 25
    iteration = -1
    out_dict = {}            

    for i in f.readlines():

        if counter == 25:

            if out_dict[iteration]:

                # append your list
                out_dict[iteration].append('string here')

            counter = 0
            iteration += 1

            # create new instance and pre-pend
            out_dict[iteration] = ['string here']

        out_dict[iteration].append(i)

Answer 2

Look at the grouper function in itertools recipes .

import itertools

def grouper(iterable, n, fillvalue=None):
    "Collect data into fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return itertools.zip_longest(*args, fillvalue=fillvalue)

from itertools import islice
list =[]
with open('output_of_json.json', 'r') as infile:
    lines_gen = grouper(infile, 25, fillvalue='')
    for line in lines_gen:
        # whatever you want to do

Note that if the final block of lines has fewer than 25 lines, that code will fill out the 25 with blank lines.