difference between current timestamp and last modified timestamp of an S3 file using Glue python
Question
Using python in AWS glue, I need to check if the files in an s3 location was modified in last 2 hours and based on that I have to run a job. I have tried few methods but in vain. Let me know if anyone would be able to help with this.
1 answers
solution1
0 2021-11-01 14:25:24
You can use the job bookmark feature for this use-case. This keeps track of the last modified timestamp of files and only processes those that have not been processed yet.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
How to list last modified file in S3 using Python
What is difference between timestamp[ns] timestamp[s]?
How to get difference between current timestamp and a different timestamp?
process the files in S3 based on their timestamp using python and boto
Retrieving last modified file from a subdirectory in AWS s3 using Python
Getting the first and last timestamp for a packet type in a pcap file using python
Capture the last timestamp, without reading the complete file using Python
Invalid timestamp error when trying to get CSV file from and S3 bucket using boto3 module in python2.7
pygsheets last access or last modified timestamp
Python logging using custom timestamps (not current timestamp)
Related Tags