Converting a suds object into a dataframe with Pandas

Question

I have a list that looks like this:

`[(deliveryObject){
   id = "0bf003ee0000000000000000000002a11cb6"
   start = 2019-01-02 09:30:00
   messageId = "68027b94b892396ed29581cde9ad07ff"
   status = "sent"
   type = "normal"
   }, (deliveryObject){
   id = "0bf0BE3ABFFDF8744952893782139E82793B"
   start = 2018-12-29 23:00:00
   messageId = "0bc403eb0000000000000000000000113404"
   status = "sent"
   type = "transactional"
 }, (deliveryObject){
   id = "0bf0702D03CB42D848CBB0B0AF023A87FA65"
   start = 2018-12-29 23:00:00
   messageId = "0bc403eb0000000000000000000000113403"
   status = "sent"
   type = "transactional"
   }
]`

When I call type() Python tells me it is a list.

When I convert it to a dataframe with pd.DataFrame(df) , the result looks like this:

Can anyone help me here? The dataframe is supposed to have column names such as "Id", "Start", "messageId" etc. but they just appear as the first element of each observation instead, with column names appearing as 0, 1 , 2 etc.

Any help is appreciated, thank you!

Answer 1

Ok, this doesn't look pretty but it works. I converted your list into a string:

import re
import pandas as pd

x = """[(deliveryObject){
   id = "0bf003ee0000000000000000000002a11cb6"
   start = 2019-01-02 09:30:00
   messageId = "68027b94b892396ed29581cde9ad07ff"
   status = "sent"
   type = "normal"
   }, (deliveryObject){
   id = "0bf0BE3ABFFDF8744952893782139E82793B"
   start = 2018-12-29 23:00:00
   messageId = "0bc403eb0000000000000000000000113404"
   status = "sent"
   type = "transactional"
 }, (deliveryObject){
   id = "0bf0702D03CB42D848CBB0B0AF023A87FA65"
   start = 2018-12-29 23:00:00
   messageId = "0bc403eb0000000000000000000000113403"
   status = "sent"
   type = "transactional"
   }
]"""

Then I used regex to somehow make a list of dictionaries:

a = re.sub(' =', ':', x)
a = re.sub('\(deliveryObject\)', '', a)

for x in ['id', 'start', 'messageId', 'status', 'type']:
    a = re.sub(x, '\''+x+'\'', a)

a = re.sub("(?<=[\"0])\n(?= +?[\'])", '\n,', a)
a = re.sub('(?<=[0])\n(?=,)', '\"\n', a)
a = re.sub('(?<=[:]) (?=[0-9])', ' \"', a)
a = re.sub('(?<= )\"(?=[\w])', '[\"', a)
a = re.sub('(?<=[\w])\"(?=\n)', '\"]', a)

Now you have a list of dictionaries. First row looks like this

list_of_dict = eval(a)
df = pd.DataFrame(list_of_dict[0])
print(df.head())

                                     id                start                         messageId status    type
0  0bf003ee0000000000000000000002a11cb6  2019-01-02 09:30:00  68027b94b892396ed29581cde9ad07ff   sent  normal

Add the rest of dictionaries from list_of_dict.

Please, feel free to improve my regex, I know it looks bad.

Answer 2

If this is for bronto and is using the SOAP and suds implementation. Then deliverObject is just a suds object.

You can do

from suds.client import Client

list_of_deliveryObjects = [(deliveryObject){
   id = "0bf003ee0000000000000000000002a11cb6"
   start = 2019-01-02 09:30:00
   messageId = "68027b94b892396ed29581cde9ad07ff"
   status = "sent"
   type = "normal"
   }, (deliveryObject){
   id = "0bf0BE3ABFFDF8744952893782139E82793B"
   start = 2018-12-29 23:00:00
   messageId = "0bc403eb0000000000000000000000113404"
   status = "sent"
   type = "transactional"
 }, (deliveryObject){
   id = "0bf0702D03CB42D848CBB0B0AF023A87FA65"
   start = 2018-12-29 23:00:00
   messageId = "0bc403eb0000000000000000000000113403"
   status = "sent"
   type = "transactional"
   }
]


data = [Client.dict(suds_object) for suds_object in list_of_deliveryObjects]
df = pd.DataFrame(data)

Answer 3

I did this:

import pandas as pd
lst =[{
   'id':"0bf003ee0000000000000000000002a11cb6",
   'start' : "2019-01-02 09:30:00",
   'messageId': "68027b94b892396ed29581cde9ad07ff",
   'status' : "sent",
   'type' : "normal"
   },{
   'id' :  "0bf0BE3ABFFDF8744952893782139E82793B",
   'start' :  "2018-12-29 23:00:00",
   'messageId' :  "0bc403eb0000000000000000000000113404",
   'status' :  "sent",
   'type' :  "transactional"
 }, {
   'id' :  "0bf0702D03CB42D848CBB0B0AF023A87FA65",
   'start' :  "2018-12-29 23:00:00",
   'messageId' :  "0bc403eb0000000000000000000000113403",
   'status' :  "sent",
   'type' :  "transactional"
   }]
df = pd.DataFrame(lst)
df

and got this(see attached image too):

    id  messageId   start   status  type
0   0bf003ee0000000000000000000002a11cb6    68027b94b892396ed29581cde9ad07ff    2019-01-02 09:30:00 sent    normal
1   0bf0BE3ABFFDF8744952893782139E82793B    0bc403eb0000000000000000000000113404    2018-12-29 23:00:00 sent    transactional
2   0bf0702D03CB42D848CBB0B0AF023A87FA65    0bc403eb0000000000000000000000113403    2018-12-29 23:00:00 sent    transactional

Result