python - 如何从行之间删除换行符而不从行尾删除换行符？

Question

My input is a big csv file with rows like:

"7807371008","Sat Jan 16 00:07:46 +0000 2010","@bigg_robb welcome to the party life of politics","T 33.417474,-86.705343","al","23845121","1381","502","Wed Mar 11 22:38:27 +0000 2009","2468"

My desired output is a new file with first and 3rd columns only with all special characters removed:

7807371008,  bigg robb welcome to the party life of politics

But there are some lines wich newline characters in between the text even though it is not technically the end of that row. In such cases, I'm getting the error:

IndexError: list index out of range

An example of such rows is:

"7807376607","Sat Jan 16 00:07:57 +0000 2010","RT @CBS8News:The commander of Gov. Riley's task
force on illegal gambling resigns after winning $2,300 at a MS casino.
gt;#conflictofinterest","Montgomery, Alabama","al","33358058","84","164","Mon Apr 20 00:48:37 +0000 2009","4509"

My code is:

import csv
import sys
import re

with open('al.csv') as f:
    for line in f:

        j = next(csv.reader([line]))
        id1 = j[0]
        id2 = re.sub('[^A-Za-z0-9\.]+',' ',id1)
        tt1 = j[2]
        tt2 = re.sub('[^A-Za-z0-9\.]+',' ',tt1)
        print id2.strip()+", "+tt2.lower()

How do I resolve this? Please help.

Answer 1

You should specified the comma , as your csv file delimiter (or a correct delimiter based on your file) also csv reader object hasn't lines that you loop over that you need to access to rows by looping over a reader object ( spamreader ) :

>>> import csv
>>> with open('al.csv', 'rb') as csvfile:
...     spamreader = csv.reader(csvfile, delimiter=',')
...     for row in spamreader:
            print re.sub('[^A-Za-z0-9\.]+',' ',row[2]) + row[0]