![](/img/trans.png)
[英]How to get last modified date of latest file from S3 with Boto Python?
[英]databricks - mounted S3 - how to get file metadata like last modified date (Python)
我已經在數據磚中安裝了s3存儲桶,可以看到文件列表,也可以使用python讀取文件
ACCESS_KEY = "XXXXXXXXXX"
SECRET_KEY = "XXXXXXXXXXXXXX"
ENCODED_SECRET_KEY = SECRET_KEY.replace("/", "%2F")
AWS_BUCKET_NAME = "testbucket"
MOUNT_NAME = "awsmount1"
dbutils.fs.mount("s3a://%s:%s@%s" % (ACCESS_KEY, ENCODED_SECRET_KEY, AWS_BUCKET_NAME), "/mnt/%s" % MOUNT_NAME)
display(dbutils.fs.ls("/mnt/%s/data" % MOUNT_NAME))
我想找出我正在讀取的文件的上次修改日期,但找不到太多,但是java選項Databricks讀取了Azure blob的Azure blob的上次修改日期 ,databricks中是否有python native選項來讀取文件元數據。
如果我理解正確,則需要使用python native sdk在Azure數據塊中裝入文件的最后修改日期。
這是從Azure blob獲取元數據信息的示例代碼:
from azure.storage.blob import BlockBlobService
block_blob_service = BlockBlobService(account_name='accoutName', account_key='accountKey')
container_name ='containerName'
block_blob_service.create_container(container_name)
generator = block_blob_service.list_blobs(container_name)
for blob in generator:
lastModified= BlockBlobService.get_blob_properties(block_blob_service,container_name,blob.name).properties.last_modified
print("\t Blob name: " + blob.name)
print(lastModified)
您可以在此處獲得更多詳細信息。
如果您正在尋找S3,那么我建議您使用Boto.oto3在使用(S3)Object python對象時為LastModified返回一個datetime對象:
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.last_modified
要將LastModified與今天的日期進行比較(Python3):
import boto3
from datetime import datetime, timezone
today = datetime.now(timezone.utc)
s3 = boto3.client('s3', region_name='eu-west-1')
objects = s3.list_objects(Bucket='my_bucket')
for o in objects["Contents"]:
if o["LastModified"] == today:
print(o["Key"])
希望能幫助到你。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.