Trying to understand pandas dataframes

Question

I have a space-delimited .dat file, for which the first few lines look like this:

1 SDSSJ000005.95+145310.1 2.49900 * 0.000e+00 0.00 NA -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 0.000 0.000 NONE 
4 SDSSJ000009.27+020621.9 1.43200 UvS 0.000e+00 0.00 NA -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 0.000 0.000 NONE 
5 SDSSJ000009.38+135618.4 2.23900 QSO 0.000e+00 0.00 NA -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 0.000 0.000 NONE 
6 SDSSJ000011.37+150335.7 2.18000 * 0.000e+00 0.00 NA -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 0.000 0.000 NONE 
11 SDSSJ000030.64-064100.0 2.60600 QSO 0.000e+00 0.00 NA -999.000 -999.000 -999.000 -999.000 15.460 -999.000 -999.000 -999.000 -999.000 23.342 56.211 UV 
15 SDSSJ000033.05+114049.6 0.73000 UvS 0.000e+00 0.00 NA -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 -999.000 0.000 0.000 NONE 
27 LBQS2358+0038 0.95000 QSO 0.000e+00 0.00 NA 17.342 18.483 18.203 17.825 -999.000 -999.000 -999.000 -999.000 -999.000 23.301 56.572 UV 

They're astronomical measurements, and there are 29008 lines in the file. When I read the file with

import pandas as pd
data = pd.read_csv('todo.dat', sep = ' ',
                   names = ['no', 'NED', 'z', 'obj_type','S_21', 'power',
                            'SI_flag','U_mag', 'B_mag', 'V_mag', 'R_mag',
                            'K_mag', 'W1_mag', 'W2_mag', 'W3_mag', 'W4_mag',
                            'L_UV', 'Q', 'flag_uv'])

the dataframe shows [29008 rows x 19 columns] . I want to organise the data based on the column headed z (which is the third column -- index 2). Adding index_col='z' to the read_csv call gives me a KeyError: 'z' error, but using index_col = 2 doesn't give me an error. I thought pandas labelled the headers like a dictionary, so 'z' should be the key in the dictionary for that column. So why do I get an error when I refer to index 2 as 'z'?

Answer 1

To me this seems buggy, and possibly related to this issue . An easy work around is just to use set_index afterhand:

data = pd.read_csv('todo.dat', sep = ' ',
                   names = ['no', 'NED', 'z', 'obj_type','S_21', 'power',
                            'SI_flag','U_mag', 'B_mag', 'V_mag', 'R_mag',
                            'K_mag', 'W1_mag', 'W2_mag', 'W3_mag', 'W4_mag',
                            'L_UV', 'Q', 'flag_uv']).set_index('z')