[英]Writing a new file to a Google Cloud Storage bucket from a Google Cloud Function (Python)
I am trying to write a new file (not upload an existing file) to a Google Cloud Storage bucket from inside a Python Google Cloud Function.我正在尝试从 Python Google Cloud Function 内部将新文件(而不是上传现有文件)写入 Google Cloud Storage 存储桶。
I tried using google-cloud-storage
but it does not have the "open" attribute for the bucket.我尝试使用
google-cloud-storage
但它没有google-cloud-storage
桶的“打开”属性。
I tried to use the App Engine library GoogleAppEngineCloudStorageClient
but the function cannot deploy with this dependencies.我尝试使用 App Engine 库
GoogleAppEngineCloudStorageClient
但该函数无法使用此依赖项进行部署。
I tried to use gcs-client
but I cannot pass the credentials inside the function as it requires a JSON
file.我尝试使用
gcs-client
但我无法在函数内部传递凭据,因为它需要一个JSON
文件。
Any ideas would be much appreciated.任何想法将不胜感激。
Thanks.谢谢。
You have to create your file locally and then to push it to GCS.您必须在本地创建文件,然后将其推送到 GCS。 You can't create a file dynamically in GCS by using open.
您不能使用 open 在 GCS 中动态创建文件。
For this, you can write in the /tmp
directory which is an in memory file system.为此,您可以在
/tmp
目录中写入内存文件系统。 By the way, you will never be able to create a file bigger than the amount of the memory allowed to your function minus the memory footprint of your code.顺便说一句,您将永远无法创建大于函数允许的内存量减去代码的内存占用量的文件。 With a function with 2Gb, you can expect a max file size of about 1.5Gb.
使用 2Gb 的函数,您可以预期最大文件大小约为 1.5Gb。
Note: GCS is not a file system, and you don't have to use it like this注意:GCS 不是文件系统,你不必像这样使用它
from google.cloud import storage
import io
# bucket name
bucket = "my_bucket_name"
# Get the bucket that the file will be uploaded to.
storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket)
# Create a new blob and upload the file's content.
my_file = bucket.blob('media/teste_file01.txt')
# create in memory file
output = io.StringIO("This is a test \n")
# upload from string
my_file.upload_from_string(output.read(), content_type="text/plain")
output.close()
# list created files
blobs = storage_client.list_blobs(bucket)
for blob in blobs:
print(blob.name)
# Make the blob publicly viewable.
my_file.make_public()
You can now write files directly to Google Cloud Storage.您现在可以将文件直接写入 Google Cloud Storage。 It is no longer necessary to create a file locally and then upload it.
不再需要在本地创建文件然后上传。
You can use the blob.open() as follows:您可以使用 blob.open() 如下:
from google.cloud import storage
def write_file():
client = storage.Client()
bucket = client.get_bucket('bucket-name')
blob = bucket.blob('path/to/new-blob-name.txt')
## Use bucket.get_blob('path/to/existing-blob-name.txt') to write to existing blobs
with blob.open(mode='w') as f:
for line in object:
f.write(line)
You can find more examples and snippets here: https://github.com/googleapis/python-storage/tree/main/samples/snippets您可以在此处找到更多示例和片段: https : //github.com/googleapis/python-storage/tree/main/samples/snippets
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.