Read a json file from blob as dictionary

Question

I have a json file which I've stored in a blob. I want to read that file as a dictionary. For a normal file system I would do it like following-

with open(file_path) as f:
        data = json.load(f)

But when I'm giving the blob url to open it is not working. I can use pyspark functions to read from the blob and convert it to json like below -

df=spark.read.json(file_url)
data = list(map(lambda row: row.asDict(True), df.collect()))[0]

I'm wondering if I can do this in a simple manner like above as collect operations are a bit expensive in spark.

Answer 1

You can use pandas to read json from url. eg

import pandas as pd
json_data = pd.read_json("http://ip-api.com/json/54.148.84.95", typ='series')
json_data.head()