upload file to a folder in gcs bucket using python

Question

I have a gcs bucket where there are multiple folders. I need to upload a file that is in my local to a particular folder in the GCS bucket. I am using the below code where it only uploads in the bucket directly but not in a folder of that bucket.

from google.cloud import storage

def upload_to_bucket(destination_blob_name, path_to_file, bucket_name):
    """ Upload data to a bucket"""
     
    # Explicitly use service account credentials by specifying the private key
    # file.
    storage_client = storage.Client.from_service_account_json(
        'service_account.json')


    bucket = storage_client.get_bucket(bucket_name)
    blob = bucket.blob(destination_blob_name)
    blob.upload_from_filename(path_to_file)
    
    return blob.public_url

print(upload_to_bucket('hello_world.py', 'hello_world.py', 'gcs_bucket_name'))

Folder structure:

gcs_bucket_name
     folder_1
     folder_2

Can anyone tell me how to upload into a folder of GCS bucket?

Answer 1

In GCS there is no something as “folders” since it is a flat namespace. What you see is only an illusion and actually a “folder” is part of the object name.

I'd suggest to read the documentation for this

Said this you must append the “path” to the destination_blob_name .

Then the function could be something like this where the default path is the root of the bucket.

def upload_to_bucket(destination_path="", destination_blob_name, path_to_file, bucket_name):
    """ Upload data to a bucket"""
     
    # Explicitly use service account credentials by specifying the private key
    # file.
    storage_client = storage.Client.from_service_account_json(
        'service_account.json')


    bucket = storage_client.get_bucket(bucket_name)
    blob = bucket.blob(destination_path+destination_blob_name)
    blob.upload_from_filename(path_to_file)
    
    return blob.public_url

And when you call it to upload a file to folder_2 it could be:

upload_to_bucket('folder_2/', 'hello_world.py', 'hello_world.py', 'gcs_bucket_name')

Answer 2

There aren't really folders in GCS. Your "folder" is just a name that gets prefixed to your file name. so blob = bucket.blob(destination_blob_name) should be blob = bucket.blob(folder_name + destination_blob_name)