Removing Line From Python List - All Lines Containing Certain Number

Question

I am looking to remove lines from a list that have a 0 in the 4th position. When I write out the file now it is not eliinating all the zero lines.

counter = 0
for j in all_decisions:
    if all_decisions[counter][4] == 0:
        all_decisions.remove(j)
    counter += 1

ofile = open("non_zero_decisions1.csv","a")

writer = csv.writer(ofile, delimiter=',')

for each in all_decisions:
    writer.writerow(each)

ofile.close()

Answer 1

Use a list comprehension.

all_decisions = [x for x in all_decisions if x[4] != 0]

Or, use filter .

all_decisions = filter(lambda x: x[4] != 0, all_decisions)

The way you're doing this is not a good idea because you're modifying all_decisions while you're iterating over it. If you wanted to do it in a loop, I would suggest something like:

temp = []
for x in all_decisions:
    if x[4] != 0:
        temp.append(x)
all_decisions = temp

But this is basically just a more verbose equivalent of the list comprehension and filter approaches I showed above.

Answer 2

I think the problem is in your loop which eliminates the lines:

counter = 0
for j in all_decisions:
    if all_decisions[counter][4] == 0:
        all_decisions.remove(j)
    counter += 1

If you remove an element, you also bump the counter. The consequence of that is that you're skipping lines. So you might miss lines to be removed. Try only bumping the counter if you didn't remove an element, ie

counter = 0
for j in all_decisions:
    if all_decisions[counter][4] == 0:
        all_decisions.remove(j)
    else:
        counter += 1

That being said, a more concise way to do what you want would be

with open("non_zero_decisions1.csv","a") as ofile:
  writer = csv.writer(ofile, delimiter=',')
  writer.writerows(d for d in all_decisions if d[4] != 0)

The with clause will take care of calling close on ofile after executing the code, even if an exception is thrown. Also, csv.writer features a writerows method which takes a list of rows. Thirdly, you can use a generator expression d for d in all_decisions if d[4] != 0 to replace your filtering loop.