如何使用 Pandas 将 csv 中的多行读取到单个数据帧行中

Question

I have a file that has a comment on the first the line, followed by two lines with the names of the headers slippted across them and a third line with the name of the index. The file looks like this:

# 3 5 <-- this is a comment indicating how many rows and column are matrix data
head1 head2 head3
head4 head5
idx1 idx2 idx3

1.1 1.2 1.3 
1.4 1.5
2.1 2.2 2.3
2.4 2.5
3.1 3.2 3.3
3.4 3.5

How can I read the file with pandas in order to have a dataframe that looks like this?

        head1 head2 head3 head4 head5
idx1    1.1   1.2   1.3   1.4   1.5
idx2    2.1   2.2   2.3   2.4   2.5
idx3    3.1   3.2   3.3   3.4   3.5

Answer 1

You can specify the skiprows keyword of read_csv in order to create one data frame that contains all 3-value lines (by skipping the 2-valued ones) and then create another data frame which contains all the 2-value lines. Note that you can specify the header row with the header keyword.

So you can parse the csv file into two different data frames which you can concatenate later on.

As an example (assuming 3-valued lines are even line numbers and 2-valued lines are odd line numbers):

df3 = pd.read_csv(..., skiprows=lambda x: x%2 == 1)
df2 = pd.read_csv(..., skiprows=lambda x: x%2 == 0)

Then you can use concat in order to concatenate the two data frames into a single one:

df = pd.concat((df3, df2))