將 api 響應轉換為 pandas

Question

我想將 API 響應轉換為 pandas dataframe 以使其更易於操作。

下面是我到目前為止嘗試過的：

import requests
import pandas as pd

URL = 'https://api.gleif.org/api/v1/lei-records?page[size]=10&page[number]=1&filter[entity.names]=*'
r = requests.get(URL, proxies=proxyDict)
x = r.json()
x
out:

{'meta': {'goldenCopy': {'publishDate': '2020-07-14T00:00:00Z'},
  'pagination': {'currentPage': 1,
   'perPage': 10,
   'from': 1,
   'to': 10,
   'total': 1675786,
   'lastPage': 167579}},
 'links': {'first': 'https://api.gleif.org/api/v1/lei-records?filter%5Bentity.names%5D=%2A&page%5Bnumber%5D=1&page%5Bsize%5D=10',
  'next': 'https://api.gleif.org/api/v1/lei-records?filter%5Bentity.names%5D=%2A&page%5Bnumber%5D=2&page%5Bsize%5D=10',
  'last': 'https://api.gleif.org/api/v1/lei-records?filter%5Bentity.names%5D=%2A&page%5Bnumber%5D=167579&page%5Bsize%5D=10'},
 'data': [{'type': 'lei-records',
   'id': '254900RR9EUYHB7PI211',
   'attributes': {'lei': '254900RR9EUYHB7PI211',
    'entity': {'legalName': {'name': 'MedicLights Research Inc.',
      'language': None},
     'otherNames': [],
     'transliteratedOtherNames': [],
     'legalAddress': {'language': None,
      'addressLines': ['300 Ranee Avenue'],
      'addressNumber': None,
      'addressNumberWithinBuilding': None,
      'mailRouting': None,
      'city': 'Toronto',
      'region': 'CA-ON',
      'country': 'CA',
      'postalCode': 'M6A 1N8'},
     'headquartersAddress': {'language': None,
      'addressLines': ['76 Marble Arch Crescent'],
      'addressNumber': None,
      'addressNumberWithinBuilding': None,
      'mailRouting': None,
      'city': 'Toronto',
      'region': 'CA-ON',
      'country': 'CA',
      'postalCode': 'M1R 1W9'},
     'registeredAt': {'id': 'RA000079', 'other': None},
     'registeredAs': '002185472',
     'jurisdiction': 'CA-ON',
     'category': None,
     'legalForm': {'id': 'O90R', 'other': None},
     'associatedEntity': {'lei': None, 'name': None},
     'status': 'ACTIVE',
     'expiration': {'date': None, 'reason': None},
     'successorEntity': {'lei': None, 'name': None},
     'otherAddresses': []},
    'registration': {'initialRegistrationDate': '2020-07-13T21:09:50Z',
     'lastUpdateDate': '2020-07-13T21:09:50Z',
     'status': 'ISSUED',
     'nextRenewalDate': '2021-07-13T21:09:50Z',
     'managingLou': '5493001KJTIIGC8Y1R12',
     'corroborationLevel': 'PARTIALLY_CORROBORATED',
     'validatedAt': {'id': 'RA000079', 'other': None},
     'validatedAs': '002185472'},
    'bic': None},
   'relationships': {'managing-lou': {'links': {'related': 'https://api.gleif.org/api/v1/lei-records/254900RR9EUYHB7PI211/managing-lou'}},
    'lei-issuer': {'links': {'related': 'https://api.gleif.org/api/v1/lei-records/254900RR9EUYHB7PI211/lei-issuer'}},
    'direct-parent': {'links': {'reporting-exception': 'https://api.gleif.org/api/v1/lei-records/254900RR9EUYHB7PI211/direct-parent-reporting-exception'}},
    'ultimate-parent': {'links': {'reporting-exception': 'https://api.gleif.org/api/v1/lei-records/254900RR9EUYHB7PI211/ultimate-parent-reporting-exception'}}},
   'links': {'self': 'https://api.gleif.org/api/v1/lei-records/254900RR9EUYHB7PI211'}},
  {'type': 'lei-records',
   'id': '254900F9XV2K6IR5TO93',

然后我嘗試將它放入 pandas 並給我以下結果

f = pd.DataFrame(x['data'])
f

    type    id  attributes  relationships   links
0   lei-records     254900RR9EUYHB7PI211    {'lei': '254900RR9EUYHB7PI211', 'entity': {'le...   {'managing-lou': {'links': {'related': 'https:...   {'self': 'https://api.gleif.org/api/v1/lei-rec...
1   lei-records     254900F9XV2K6IR5TO93    {'lei': '254900F9XV2K6IR5TO93', 'entity': {'le...   {'managing-lou': {'links': {'related': 'https:...   {'self': 'https://api.gleif.org/api/v1/lei-rec...
2   lei-records     254900DIC0729LEXNL12    {'lei': '254900DIC0729LEXNL12', 'entity': {'le...   {'managing-lou': {'links': {'related': 'https:...   {'self': 'https://api.gleif.org/api/v1/lei-rec...

這不是預期的結果。 我什至嘗試使用以下代碼讀取_json：

g = pd.read_json(x.text)
g

which gives me the error


AttributeError: 'dict' object has no attribute 'text'

預期的 output 應如下所示：

lei                      entity.legalName.name            entity.legalAddress.addressLines       entity.legalAddress.city       entity.legalAddress.postalcode        status       registration.status

254900RR9EUYHB7PI211     MedicLights Research Inc.        300 Ranee Avenue                         Toronto                         M6A 1N8                              ACTIVE                ISSUED

感謝任何人的幫助

Answer 1

使用json_normalize像：

pd.json_normalize(x['data'])

Answer 2

Here is another method to use the pandas to normalize the json file using pandas.io.json.json_normalize from pandas.io.json library.

如何通過 Python Pandas 正確規范化 json