熊猫read_html方法中的错误？

Question

I think the read_html method of pandas is buggy when dealing with rowspan and/or colspan.

Example:

html_table = io.StringIO(u'''<table>
    <thead>
        <tr>
            <th rowspan="2">Time</th>
            <th rowspan="2">Temp</th>
            <th colspan="3">Cloud Cover</th>
        </tr>
        <tr>
            <th>Low</th>
            <th>Middle</th>
            <th>High</th>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td>22:00</td>
            <td>12C</td>
            <td>Lorem</td>
            <td>Ipsum</td>
            <td>Dolor</td>
        </tr>
    </tbody>
</table>''')

The output of pd.read_html(html_table) is

[                 Time Temp Cloud Cover    Low Middle  High
 0 2014-05-16 22:00:00  12C       Lorem  Ipsum  Dolor   NaN

 [1 rows x 6 columns]]

Is it a bug or am I doing something wrong?

Answer 1

pandas >= 0.24.0 understands colspan and rowspan attributes. As per the release notes :

result = pd.read_html("""
    <table>
      <thead>
        <tr>
          <th>A</th><th>B</th><th>C</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td colspan="2">1</td><td>2</td>
        </tr>
      </tbody>
    </table>""")

result

Out:

[   A  B  C
 0  1  1  2

Previously this would return the following:

[   A  B   C
 0  1  2 NaN]