using curl to get a json output and struggling to make that output into a pandas dataframe

Question

Using the pandas json_normalise function I'm given this - raise NotImplementedError. Using DataFrame.from_dict(swaps, orient='columns') I'm given - ValueError: DataFrame constructor not properly called!

from subprocess import run
import pandas as pd

#using curl and subprocesses
swaps = run('curl --location --request POST https://www.rmb.co.za/rates-service/rates/historical \
    --form productType="Swaps" \
    --form top="13"', shell=True)

Is there something I'm missing?

Answer 1

You can use subprocess.run and try to capture the text, convert it to JSON and then to a DataFrame.

Or you can use requests to make your life a lot easier:

import requests

# This is equivalent to your curl command
response = requests.post("https://www.rmb.co.za/rates-service/rates/historical", data={
    "productType": "Swaps",
    "top": "13"
})
# Raise an exception if the request did not succeed
response.raise_for_status()

df = pd.DataFrame(response.json())