botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

Question

I'm downloading files from AWS S3 Bucket like so:

import boto3

s3client = boto3.client("s3")
s3 = boto3.resource('s3')

bucket_name = 'practice_bucket'
bucket = s3.Bucket(bucket_name)

for obj in bucket.objects.all():
    filename = obj.key.rsplit('/')[-1]
    s3client.download_file(bucket_name, obj.key, "/txt/" + filename)

When attempting to place some files under a subdirectory, eg /txt/ , I get the error:

botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

Oddly, it works on other file types using the same method, but doesn't work for ones with .json extension.

What could be the issue? I even tried without placing them in an absolute subdirectory path, and I get no error and downloads the file onto the same directory as the script downloading. But when I actually define the path to download the file to, I get the error.

Answer 1

Your code is correct.

botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

This error is thrown when the object you are trying to fetch is not present in the bucket.

Answer 2

升级 aiobotocore 为我解决了这个问题。

pip install --upgrade aiobotocore