Access google cloud storage account objects from app engine

Question

Overview

I have a GCP storage bucket, which has a .json file and 5 jpeg files. In the .json file the image names match the jpeg file names. I want to know a way which i can access each of the object within the storage account based upon the image name.

Method 1 (Current Method):

Currently, a python script is been used to to get the images from the storage bucket. This is been done by looping through the .json file of image names, getting each individual image name, then building a URL based on the bucket/image name and retrieving the image and displaying it on a flask App Engine site.

This current method requires the bucket objects to be public, which poses a security issue with the internet granted access to this bucket, secondly it is computational expensive, with each image having to be pulled down from the bucket separately. The bucket will eventually contain 10000 images, which will result in the images been slow to load and display on the web page.

Requirement (New Method): Is there a method in which i can pull down images from the bucket, not all the images at once, and display them on a web page. I want to be able to access individual images from the bucket and display their corresponding image data, retrieved from the .json file.

Lastly i want to ensure that neither the bucket or the objects are public and can only be accessed via the app engine.

Thanks

Answer 1

Would be helpful to see the Python code that's doing the work right now. You shouldn't need the storage objects to be public. They should be able to be retrieved using the Google Cloud Storage (GCS) API and a service account token that has view-only permissions on storage (although depending on whether or not you know the object names and need to get the bucket name, it might require more permissions on the service account).

As for the performance, you could either do things on the front end to be smart about how many you're showing and fetch only what you want to display as the user scrolls, or you could paginate your results from the GCS bucket.

Links to the service account and API pieces here:

https://cloud.google.com/iam/docs/service-accounts https://cloud.google.com/storage/docs/reference/libraries#client-libraries-install-python

Information about pagination for retrieving GCS objects here:

How does paging work in the list_blobs function in Google Cloud Storage Python Client Library