How can I process data after a specific line in python 2.6?

Question

I have a script that basically reads a text file and creates 8 lists. It works perfectly if it reads the file from line 1. I need it to start reading the text file from line 177 to line 352 (that is the last line).

This is my script and the change. I'm not getting any error but not any result either. The program hangs there without response:

f = open("Output1.txt", "r")

lines = [line.rstrip() for line in f if line != "\n"] #Get all lines, strip
newline chars, and remove lines that are just newlines.


NUM_LISTS = 8

groups = [[] for i in range(NUM_LISTS)]



listIndex = 0


for line in lines:


    while line > 177: #here is the problem


        if "Transactions/Sec for Group" not in line:
            groups[listIndex].append(float(line))
            listIndex += 1
            if listIndex == NUM_LISTS:
                listIndex = 0
                value0 = groups[0]
                value1 = groups[1]
                value2 = groups[2]
                value3 = groups[3]
                value4 = groups[4]
                value5 = groups[5]
                value6 = groups[6]
                value7 = groups[7]



json_file = 'json_global.json'

json_data = open(json_file)

data = json.load(json_data)

for var1 in range(0, 11):

    a = value0[var1]
    b = value1[var1]
    c = value2[var1]
    d = value3[var1]
    e = value4[var1]
    f = value5[var1]
    g = value6[var1]
    h = value7[var1]

    var2 = var1 + 57

    item = data[var2]['item']

    cmd = data[var2]['command']


    var1+= 1

    print item, cmd,  a, b, c, d, e, f, g, h)

Answer 1

line contains the contents of each line, not the line number. Even if it did, this would fail, because you'd skip over the loop since the first line's number is not greater than 177. Here's a way to do what you want:

for linenumber, line in enumerate(lines, 1):
    if linenumber > 177:
        do_stuff(line)

enumerate() takes an iterable and returns an iterable of (index, item) tuples. The 1 argument tells it what index to start at; it defaults to 0 . Adjust that and the number in if linenumber > 177: according to what you want to do.

Another way of doing it is using itertools.islice() , as also mentioned by Anand S Kumar in his answer . Here's a version using islice() that doesn't read the entire file into memory beforehand:

from itertools import islice

with open('Output1.txt', 'r') as f:
    lines = (line.rstrip() for line in f if line != '\n')
    for line in islice(lines, 177, None):
        do_stuff(line)

That'll effectively slice the lines as if you had done lines[177:] (which is another solution).

Note that you're not including lines that are only a newline, so line 177 in the file is not the same line 177 in your program.

Answer 2

The issue is that lines is the list of lines in the , so when you do -

for line in lines:

The line is a string, not the index of the line, and so the next while loop is true and hence it should go into an infinite loop there , since you never change line inside the while loop and the condition is always true.

Instead of doing all that , I would recommend you to use itertools.islice , for iterating from line 177 till the end. Example -

import itertools
for line in itertools.islice(lines,177,None):
    if "Transactions/Sec for Group" not in line:
        groups[listIndex].append(float(line))
        listIndex += 1
        if listIndex == NUM_LISTS:
            listIndex = 0
            value0 = groups[0]
            value1 = groups[1]
            value2 = groups[2]
            value3 = groups[3]
            value4 = groups[4]
            value5 = groups[5]
            value6 = groups[6]
            value7 = groups[7]