使用 Python API 從 Google Cloud Datalab 上傳文件到 Google Cloud Storage Bucket

Question

我正在嘗試使用 Python API 將筆記本本身內的 Datalab 實例中的文件上傳到我的 Google 存儲桶，但我無法弄清楚。 Google 在其文檔中提供的代碼示例在 Datalab 中似乎不起作用。 我目前正在使用 gsutil 命令，但想了解如何使用 Python API 執行此操作。

文件目錄（我想上傳位於 checkpoints 文件夾中的 python 文件）：

!ls -R

.:
checkpoints  README.md  tpot_model.ipynb

./checkpoints:
pipeline_2020.02.29_00-22-17.py  pipeline_2020.02.29_06-33-25.py
pipeline_2020.02.29_00-58-04.py  pipeline_2020.02.29_07-13-35.py
pipeline_2020.02.29_02-00-52.py  pipeline_2020.02.29_08-45-23.py
pipeline_2020.02.29_02-31-57.py  pipeline_2020.02.29_09-16-41.py
pipeline_2020.02.29_03-02-51.py  pipeline_2020.02.29_11-13-00.py
pipeline_2020.02.29_05-01-17.py

當前代碼：

import google.datalab.storage as storage
from pathlib import Path

bucket = storage.Bucket('machine_learning_data_bucket')


for file in Path('').rglob('*.py'):
    # API CODE GOES HERE

當前工作解決方案：

!gsutil cp checkpoints/*.py gs://machine_learning_data_bucket

Answer 1

這是對我有用的代碼：

from google.cloud import storage
from pathlib import Path

storage_client = storage.Client()
bucket = storage_client.bucket('bucket')

for file in Path('/home/jupyter/folder').rglob('*.py'):
    blob = bucket.blob(file.name)
    blob.upload_from_filename(str(file))
    print("File {} uploaded to {}.".format(file.name,bucket.name))

輸出：

File file1.py uploaded to bucket.
File file2.py uploaded to bucket.
File file3.py uploaded to bucket.

編輯

或者你可以使用：

import google.datalab.storage as storage
from pathlib import Path

bucket = storage.Bucket('bucket')

for file in Path('/home/jupyter/folder').rglob('*.py'):
    blob = bucket.object(file.name)
    blob.write_stream(file.read_text(), 'text/plain')
    print("File {} uploaded to {}.".format(file.name,bucket.name))

輸出：

File file1.py uploaded to bucket.
File file2.py uploaded to bucket.
File file3.py uploaded to bucket.