将带有xml数据的pandas dataframe列转换为规范化列？

Question

I have a DataFrame in pandas , one of whose columns is a XML string. What I want to do is create one column for each of the xml nodes with column names in a normalised form. For example,

    id    xmlcolumn
    1     <main attr1='abc' attr2='xyz'><item><prop1>text1</prop1><prop2>text2</prop2></item></main>
    2     <main ........</main>

I want to convert this to a data frame like so:

id   main.attr1  main.attr2 main.item.prop1  main.item.prop2
1       abc        xyz          text1           text2
2      .....

How would I do that, while still keeping the existing columns in the DataFrame ?

Answer 1

The first step that needs to be done is to convert the XML string to a pandas Series (under the assumption, that there will always be the same amount of columns in the end). So you need a function like:

def convert_xml(raw):
    # some etree xml mangling

This can be achieved eg using the etree package in python. The returned series must have an index, where each entry in the index is the new column name to appear, eg for your example:

pd.Series(['abc', 'xyz'], index=['main.attr1', 'main.attr2'])

Given this function, you can do the following with pandas (mocking away the XML mangling):

frame = pd.DataFrame({'keep': [42], 'xml': '<foo></foo>'})
temp = frame['xml'].apply(convert_xml)
frame = frame.drop('xml', axis=1)
frame = pd.concat([frame, temp], axis=1)