熊猫数据框read_excel不会将空白的左上单元格视为列？

Question

I'm trying to read an Excel or CSV file into pandas dataframe. The file will read the first two columns only, and the top row of the first two columns will be the column names. The problem is when I have the first column of the top row empty in the Excel file.

            IDs
2/26/2010    2
3/31/2010    4
4/31/2010    2
5/31/2010    2

Then, the last line of the following code fails:

uploaded_file = request.FILES['file-name']
if uploaded_file.name.endswith('.csv'):
    df = pd.read_csv(uploaded_file, usecols=[0,1])
else:
    df = pd.read_excel(uploaded_file, usecols=[0,1])

ref_date = 'ref_date'
regime_tag = 'regime_tag'
df.columns = [ref_date, regime_tag]

Apparently, it only reads one column (ie the IDs). However, with read_csv , it reads both column, with the first column being unnamed . I want it to behave that way and read both columns regardless of whether the top cells are empty or filled. How do I go about doing that?

Answer 1

What's happening is the first "column" in the Excel file is being read in as an index, while in the CSV file it's being treated as a column / series.

I recommend you work the other way and amend pd.read_csv to read the first column as an index. Then use reset_index to elevate the index to a series:

if uploaded_file.name.endswith('.csv'):
    df = pd.read_csv(uploaded_file, usecols=[0,1], index_col=0)
else:
    df = pd.read_excel(uploaded_file, header=[0,1], usecols=[0,1])

df = df.reset_index()  # this will elevate index to a column called 'index'

This will give consistent output, ie first series will have label 'index' and the index of the dataframe will be the regular pd.RangeIndex .

You could potentially use a dispatcher to get rid of the unwieldy if / else construct:

file_flag = {True: pd.read_csv, False: pd.read_excel}
read_func = file_flag[uploaded_file.name.endswith('.csv')]

df = read_func(uploaded_file, usecols=[0,1], index_col=0).reset_index()