用熊猫读取格式错误的“ csv”文件

Question

I have a malformed "csv" file:

txt = """NAME;a;b;c
ATTR1;1;2;3
ATTR2;1;2;3;;;
ATTR3;1;2;3;
ATTR4;1;2;3"""

I there a way to use pandas pd.read_* toolbox to get the following pd.DataFrame :

|---+-------+---+---+---|
|   | 0     | 1 | 2 | 3 |
|---+-------+---+---+---|
| 0 | NAME  | a | b | c |
| 1 | ATTR1 | 1 | 2 | 3 |
| 2 | ATTR2 | 1 | 2 | 3 |
| 3 | ATTR3 | 1 | 2 | 3 |
| 4 | ATTR4 | 1 | 2 | 3 |
|---+-------+---+---+---|

?

PS I know how to do it with import csv

Thank you for ideas and BR, Lex

EDIT

This was a toy example from real file (which I again had to modify) ...

SRC = 'https://dl.dropboxusercontent.com/u/40513206/test.csv'
NA_VALUES = ['', '#N/A N/A', '#N/A Field Not Applicable', '#N/A Invalid Field',
         '#N/A Invalid Security', '#N/AN/A', '#N/A Limit', '#####', '#DIV/0!', 
         '#N/A', '#NAME?', '#NULL!', '#NUM!', '#REF!', '#VALUE!']
CSV_ENCODING = 'WINDOWS-1252'
S_ROWS = 6
NR_ROWS = 60
NR_COLS = 52 # correct nr. of columns, but not always known

dat_m = pd.read_csv(SRC, sep = ';', header = None, index_col = None, skiprows = S_ROWS, 
                nrows = NR_ROWS, encoding = CSV_ENCODING, na_values = NA_VALUES, names = range(NR_COLS))

Seems that if we use names parameter then NR_COLS must be >= actual nr. of columns in first row, if not so, then Index or MultiIndex is formed (based on actual columns), for example if NR_COLS = 50 then index has 2 levels, if NR_COLS = 49 then 3 levels etc.

All this is a result when I save Excel to csv , it seems to add sep = ';' to some rows and for some other reason I can not use xls (read) files directly.

So I will use large NR_COLS value or continue with csv library.

Thank you!

Answer 1

How about:

>>> txt = 'NAME;a;b;c\nATTR1;1;2;3\nATTR2;1;2;3;;;\nATTR3;1;2;3;\nATTR4;1;2;3'
>>> pd.read_csv(StringIO(txt),sep=";",names=range(4))
       0  1  2  3
0   NAME  a  b  c
1  ATTR1  1  2  3
2  ATTR2  1  2  3
3  ATTR3  1  2  3
4  ATTR4  1  2  3

[5 rows x 4 columns]

Sometimes when I don't know how many columns there are beforehand I do something silly like names=range(128) and then .dropna(how='all', axis=1) .