WindowsError: [Error 5] Access is denied using boto/python

Question

I want to download all the files and folders in the bucket. This is my code

conn = boto.connect_s3(AWS_ACCESS_KEY_ID,
                           AWS_ACCESS_KEY_SECRET)
bucket = conn.get_bucket(bucket_name)
key = boto.s3.key.Key(bucket)
key.get_contents_to_filename('path/to/folder')

error-

File "C:\Python27\lib\site-packages\boto\s3\key.py", line 1726, in get_contents_to_filename
    os.remove(filename)
WindowsError: [Error 5] Access is denied: 'path/to/folder'

Please help me overcome the problem!

Answer 1

I was having this exact problem. The problem is that S3 Buckets don't have directories. The file structure is flat but every key is named as if it were a path name.

So if you have

bucket
|
|__dir1
   |
   |_file1
   |_file2

You'll actually have keys

bucket/dir1/
bucket/dir1/file1
bucket/dir1/file2

For me (in a Windows OS), the get_contents_to_filename works for file keys ( bucket/dir1/file1 ) but fails for directory keys ( bucket/dir1 ), which triggers os.remove(filename) and ultimately the PermissionError.

Instead you can try to recursively iterate through your directory structure and get_contents_to_filename for file keys and mkdir for directory keys (these end in a slash '/').

Answer 2

I Was facing the same issue with boto3. This is what I was trying-

s3.Object('<bucket>','<prefix>/<filename>').download_file('C:\myfolder')

I tried multiple things like running as an admin, giving different local paths, giving public user paths etc and nothing worked. The issue was that I was providing a folder path and not a file path. So this worked in the end-

s3.Object('<bucket>','<prefix>/<filename>').download_file('C:\myfolder\<filename>')