How to structure a list with JSON objects in Python?

Question

I got a list in Python with Twitter user information and exported it with Pandas to an Excel file. One row is one Twitter user with nearly all information of the user ( name , @-tag , location etc.)

Here is my code to create the list and fill it with the user data:

def get_usernames(userids, api):
    fullusers = []
    u_count = len(userids)
    try:
        for i in range(int(u_count/100) + 1):
            end_loc = min((i + 1) * 100, u_count)
            fullusers.extend(
                api.lookup_users(user_ids=userids[i * 100:end_loc])
            )
        print('\n' + 'Done! We found ' + str(len(fullusers)) + ' follower in total for this account.' + '\n')
        return fullusers

    except:
        import traceback
        traceback.print_exc()
        print ('Something went wrong, quitting...')

The only problem is that every row is in JSON object and therefore one long comma-seperated string. I would like to create headers (no problem with Pandas) and only write parts of the string (ie ID or name ) to colums.

Here is an example of a row from my output.xlsx:

User(_api=<tweepy.api.API object at 0x16898928>, _json={'id': 12345, 'id_str': '12345', 'name': 'Jane Doe', 'screen_name': 'jdoe', 'location': 'Nirvana, NI', 'description': 'Just some random descrition')

I have two ideas, but I don't know how to realize them due to my lack of skills and experience with Python.

1. Create a loop which saves certain parts ( 'id' , 'name' etc.) from the JSON-string in colums.
2. Cut off the User(_api=<tweepy.api. API object at 0x16898928>, _json={ at the beginning and ) at the end, so that I may export they file as CSV.

Could anyone help me out with one of my two solutions or suggest a "simple" way to do this?

fyi: I want to do this to gather data for my thesis.

Answer 1

Try the python json library:

import json
jsonstring = "{'id': 12345, 'id_str': '12345', 'name': 'Jane Doe', 'screen_name': 'jdoe', 'location': 'Nirvana, NI', 'description': 'Just some random descrition')"

jsondict = json.loads(jsonstring)
# type(jsondict) == dictionary

Now you can just extract the data you want from it:

id = jsondict["id"]
name = jsondict["name"]
newdict = {"id":id,"name":name}