Unable to Download Images from S3 Bucket in Python/Code has Access Keys already

Question

For a project, I need to download some items in my S3 bucket. I have already seen similar posts about this topic; however, I hardcoded my access key ID and secret access key inside the program and am still unable to download them. Python keeps returning the error:

"botocore.exceptions.NoCredentialsError: Unable to locate credentials"

Despite providing my credentials, I am still unable to download. My code is provided below. Can anyone help me correct this?

import boto3
import os
"""import sys
import csv
import pandas as pd
import numpy as np
import tensorflow as tf"""
import nibabel as nib
from boto3.session import Session


aws_access_key_id = '********************'
aws_secret_access_key = '****************************************'
bucket1 = 'adnimcic'
mcic = [[], [], []]
mcicc = [[], [], []]
bucket2 = 'adnimcinc'
mcinc = [[], [], []]
bucket3 = 'adniresults'
results = []
s3_client = boto3.client('s3')


#connecting to S3
session = Session(aws_access_key_id, aws_secret_access_key)
s3 = session.resource('s3')
bucket1obj = s3.Bucket(bucket1)
#bucket2obj = s3.Bucket(bucket2)
#'MCIc_Segmented/ADNI_002_S_0729_MR_MP-RAGE_REPEAT_br_raw_20070225105857428_72_S27091_I41585_be_be_pve_2.nii.gz_extracted'
def concatenate(name):
    name = name.split('.')
    name.pop()
    name = name[0] + '.' + name[1]
    name = name.split('/')
    name = name[1]
    return name

def download(bucketname, key):
    path = '/Volumes/LaCie Mac/' + concatenate(key.key)
    s3_client.download_file(bucketname, key.key, path)

for key in bucket1obj.objects.all():
    if 'pve_0' and 'extracted' in key.key:
        mcic[0].append(key)
        download(bucket1, key)
for key in bucket1obj.objects.all():
    if 'pve_1' and 'extracted' in key.key:
        mcic[1].append(key)
        download(bucket1, key)
for key in bucket1obj.objects.all():
    if 'pve_2' and 'extracted' in key.key:
        mcic[2].append(key)
        download(bucket1, key)

Answer 1

Looking at your code, you have two different S3 objects, s3_client and s3 :

s3_client = boto3.client('s3')

...

session = Session(aws_access_key_id, aws_secret_access_key)
s3 = session.resource('s3')

...

def download(bucketname, key):
    path = '/Volumes/LaCie Mac/' + concatenate(key.key)
    s3_client.download_file(bucketname, key.key, path)

```

Looking at your code, it looks like you use your aws_access_key_id and aws_secret_access_key on your session object which you use on your s3 resource as expected, but you don't use the session on the s3_client .

If I understand your problem correctly, you should be able to resolve this issue by creating the client from your session, like:

session = Session(aws_access_key_id, aws_secret_access_key)
s3 = session.resource('s3')
s3_client = session.client('s3')