[英]how to upload sub-folder which is empty to S3 using python
The following code works fine except if there is a subfolder, which does not have any file inside, then the subfolder will not appear in S3. 以下代码可以正常工作,除非存在一个子文件夹,该子文件夹中没有任何文件,则该子文件夹不会出现在S3中。 eg if /home/temp/subfolder has no file, then subfolder will not show in S3.
例如,如果/ home / temp / subfolder没有文件,则子文件夹不会在S3中显示。 how to change the code so that the empty folder is also uploaded in S3?
如何更改代码,以便在S3中也上传空文件夹? I tried to write sth.
我试图写某事。 (see note below), but do not know how to call put_object() to the empty subfolder.
(请参阅下面的注释),但不知道如何将put_object()调用到空子文件夹。
#!/usr/bin/env python
import os
from boto3.session import Session
path = "/home/temp"
session = Session(aws_access_key_id='XXX', aws_secret_access_key='XXX')
s3 = session.resource('s3')
for subdir, dirs, files in os.walk(path):
# note: if not files ......
for file in files:
full_path = os.path.join(subdir, file)
with open(full_path, 'rb') as data:
s3.Bucket('my_bucket').put_object(Key=full_path[len(path)+1:],
Body=data)
besides, I tried to call this function to check if a subfolder or file exist or not. 此外,我尝试调用此函数来检查子文件夹或文件是否存在。 it works for file, but not subfolder.
它适用于文件,但不适用于子文件夹。 how to check if a subfolder exists or not?
如何检查子文件夹是否存在? (if a subfolder exists, I will not upload)
(如果存在子文件夹,我将不会上传)
def check_exist(s3, bucket, key):
try:
s3.Object(bucket, key).load()
except botocore.exceptions.ClientError as e:
return False
return True
BTW, I refer the above code from 顺便说一句,我从上面引用了上面的代码
check if a key exists in a bucket in s3 using boto3 使用boto3检查s3中存储桶中是否存在密钥
and 和
http://www.developerfiles.com/upload-files-to-s3-with-python-keeping-the-original-folder-structure/ http://www.developerfiles.com/upload-files-to-s3-with-python-keeping-the-original-folder-structure/
thanks them for sharing the code. 感谢他们分享代码。
Directories (folders, subfolders, etc.) do not exist in S3. 目录(文件夹,子文件夹等)在S3中不存在。
When you copy this file to an empty S3 bucket /mydir/myfile.txt
, only the file myfile.txt
is copied to S3. 当您将此文件复制到空的S3存储桶
/mydir/myfile.txt
,仅将文件myfile.txt
复制到S3。 The directory mydir
is not created as that string is part of the file name mydir/myfile.txt
. 未创建目录
mydir
因为该字符串是文件名mydir/myfile.txt
。 The actual file name is the full path, no subdirectories exist or are created. 实际文件名是完整路径,不存在或未创建任何子目录。
S3 simulates directories by using a prefix when listing files in the bucket. 在存储桶中列出文件时,S3通过使用前缀来模拟目录。 If you specify
mydir/
, then all of the S3 objects that start with mydir/
will be returned including objects such as mydir/anotherfolder/myotherfile.txt
. 如果指定
mydir/
,则将返回所有以mydir/
开头的S3对象,包括mydir/anotherfolder/myotherfile.txt
。 S3 supports a delimitor such as /
so that the appearance of subdirectories can be created. S3支持定界符(例如
/
以便可以创建子目录的外观。
Note: There is no /
at the beginning of a file name for S3 objects. 注意:S3对象的文件名开头没有
/
。
Listing Keys Hierarchically Using a Prefix and Delimiter 使用前缀和定界符分层列出密钥
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.