Why does this give me an IndexError?

Question

I have the following code that opens a csv , and appends all the values to a list. I then remove all the values that do not start with '2' . However, on the line if lst[k][0] != '2': , it raises an error:

Traceback (most recent call last):
  File "historical_tempo1.py", line 23, in <module>
    if lst[k][0] != '2':
IndexError: list index out of range

Here is the code:

y = open('today.csv')
lst = []
for k in y:
        lst.append(k)

lst = ' '.join(lst).split()

for k in range(0, len(lst)-1):
        if lst[k][0] != '2':
                lst[k:k+1] = ''

Here is the first bit of content from the csv file:

Date,Time,PM2.5 Mass concentration(ug/m3),Status

3/15/2014,4:49:13 PM,START
2014/03/15,16:49,0.5,0
3/15/2014,4:49:45 PM,START
2014/03/15,16:50,5.3,0
2014/03/15,16:51,5.1,0
2014/03/15,16:52,5.0,0
2014/03/15,16:53,5.0,0
2014/03/15,16:54,5.4,0
2014/03/15,16:55,6.4,0
2014/03/15,16:56,6.4,0
2014/03/15,16:57,5.0,0
2014/03/15,16:58,5.2,0
2014/03/15,16:59,5.2,0
3/15/2014,5:03:48 PM,START
2014/03/15,17:04,4.8,0
2014/03/15,17:05,4.9,0
2014/03/15,17:06,4.9,0
2014/03/15,17:07,5.1,0
2014/03/15,17:08,4.6,0
2014/03/15,17:09,4.9,0
2014/03/15,17:10,4.4,0
2014/03/15,17:11,5.7,0
2014/03/15,17:12,4.4,0
2014/03/15,17:13,4.0,0
2014/03/15,17:14,4.6,0
2014/03/15,17:15,4.7,0
2014/03/15,17:16,4.8,0
2014/03/15,17:17,4.5,0
2014/03/15,17:18,4.4,0
2014/03/15,17:19,4.5,0
2014/03/15,17:20,4.8,0
2014/03/15,17:21,4.6,0
2014/03/15,17:22,5.1,0
2014/03/15,17:23,4.2,0
2014/03/15,17:24,4.6,0
2014/03/15,17:25,4.5,0
2014/03/15,17:26,4.4,0

Answer 1

1. Why do you get an IndexError ? Because when you write lst[k:k+1] = '' , you have just removed the k+1 element from your list, which means your list is shorter by 1 element, and your loop is still going up to the old len(lst) , so the index variable k is guaranteed to go over.

2. How can you fix this? Loop over a copy and delete from the original using list.remove() .

The following code loops over the copy.

for s in lst[:]:
     if k[0] != '2':
         list.remove(k)

Answer 2

The expressions lst[k][0] raises an IndexError , which means that either:

# (1) this expressions raises it
x = lst[k]
# or (2) this expression raises it
x[0]

If (1) raises it, it means len(lst) <= k , ie there are fewer items than you expect.

If (2) raises it, it means x is an empty string, which means you can't access its item at index 0.

---

Either way, instead of guessing, use pdb . Run your program using pdb , and at the point your script aborts, examine the values of lst , k , lst[k] , and lst[k][0] .

Answer 3

Basically, your list, 'lst', starts out at length 43. The 'slice' operation lst[k:k+1] doesn't replace two separate indexed values with '', but wipes out one of the list entries. If you did a lst[k:k+5], you would wipe out five entries. Try it in the interpreter.

I'd recommend you don't try to wipe out those entries particularly in the list you are performing operations. It is shrinking in this case which means you go out of range and get an "IndexError". Store the values you want into another a list if you have to remove the lines that don't begin with "2".

List comprehensions work great in this case...

mynewlist = [x for x in lst if x[0] == '2']