I'm trying to read a .json
file as dict()
in a python code from VM instance stored in google cloud storage bucket.
I tried reading json file as blob:
client = storage.Client()
bucket = client.get_bucket('bucket-id-here')
blob = bucket.get_blob('remote/path/to/file.json')
str_json = blob.download_as_string()
But I'm unable to decode the str_json
. Is my approach correct? if any other approach available please let me know.
I need something like:
# Method to load json
dict = load_json(gcs_path='gs://bucket_name/filename.json')
Here is an alternative way to reach it using the official Cloud Storage library:
# Import the Google Cloud client library and JSON library
from google.cloud import storage
import json
# Instantiate a Google Cloud Storage client and specify required bucket and file
storage_client = storage.Client()
bucket = storage_client.get_bucket('bucket_name')
blob = bucket.blob('filename.json')
# Download the contents of the blob as a string and then parse it using json.loads() method
data = json.loads(blob.download_as_string(client=None))
Use gcsfs to connect to google cloud bucket and then load it as json.
import gcsfs
import json
fs = gcsfs.GCSFileSystem(project='your-gcp-project-name')
url = "gs://bucket_name/filename.json"
with fs.open(url, 'rb') as f:
data=json.load(f)
This method using GCS File System gcsfs
an be used read files from Google Cloud storage.
# Reading gcs files with gcsfs
import gcsfs
import json
gcs_file_system = gcsfs.GCSFileSystem(project="gcp_project_name")
gcs_json_path = "gs://bucket_name/path/to/file.json"
with gcs_file_system.open(gcs_json_path) as f:
json_dict = json.load(f)
This method also works for images stored in GCS with skimage
as,
from skimage import io
with gcs_file_system.open(gcs_img_path) as f:
img = io.imread(f)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.