Heroku：S3 / boto3 - 错误：请使用 AWS4-HMAC-SHA256

Question

I have been struggling with this for days. Using S3 for staticfiles with Django + django-storages and Heroku.

First I create an S3 bucket 4f2xivbz443 and generated Access Keys (Access Key ID and Secret Access Key).

I installed django-storages https://django-storages.readthedocs.io/en/latest/ and followed the instructions on how to add and setup Amazon S3.

When I deploy i get this error:

           raise error_class(parsed_response, operation_name)

   botocore.exceptions.ClientError: An error occurred (InvalidRequest) when calling the PutObject operation: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

Here comes all the code.

settings.py (Amazon S3 settings are added in at the end)

import os
import dj_database_url

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/1.9/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = "=ax=ka-emu33ivw-y^u00p8#uvop#-ag#+4pm_s4-=da^chbuk"

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False

# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    # Disable Django's own staticfiles handling in favour of WhiteNoise, for
    # greater consistency between gunicorn and `./manage.py runserver`. See:
    # http://whitenoise.evans.io/en/stable/django.html#using-whitenoise-in-development
    'whitenoise.runserver_nostatic',
    'django.contrib.staticfiles',
    'storages',
    'images',
]

MIDDLEWARE_CLASSES = [
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'helloworld.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
            'debug': False,
        },
    },
]

WSGI_APPLICATION = 'helloworld.wsgi.application'


# Database
# https://docs.djangoproject.com/en/1.9/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]

# Internationalization
# https://docs.djangoproject.com/en/1.9/topics/i18n/

LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True

# Update database configuration with $DATABASE_URL.
db_from_env = dj_database_url.config(conn_max_age=500)
DATABASES['default'].update(db_from_env)

# Honor the 'X-Forwarded-Proto' header for request.is_secure()
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

# Allow all host headers
ALLOWED_HOSTS = ['*']

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.9/howto/static-files/

STATIC_ROOT = os.path.join(PROJECT_ROOT, 'staticfiles')
STATIC_URL = '/static/'

# Extra places for collectstatic to find static files.
STATICFILES_DIRS = [
    os.path.join(PROJECT_ROOT, 'static'),
]

# Simplified static file serving.
# https://warehouse.python.org/project/whitenoise/
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

# Amazon S3
if not DEBUG:
    SECRET_KEY = os.environ['SECRET_KEY']

    AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
    AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
    AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']

    STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
    DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

    STATIC_URL = 'http://' + AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/'
    ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'

wsgi.py

import os

from django.core.wsgi import get_wsgi_application
from whitenoise.django import DjangoWhiteNoise

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "helloworld.settings")

application = get_wsgi_application()
application = DjangoWhiteNoise(application)

requirements.txt

boto==2.45.0
boto3==1.4.4
botocore==1.5.7
dj-database-url==0.4.1
Django==1.10.4
django-storages==1.5.2
docutils==0.13.1
gunicorn==19.6.0
jmespath==0.9.1
olefile==0.44
Pillow==4.0.0
psycopg2==2.6.2
python-dateutil==2.6.0
s3transfer==0.1.10
six==1.10.0
whitenoise==3.2

images/models.py

from django.db import models

class ImageUpload(models.Model):
    image = models.ImageField()
    title = models.CharField(max_length=50, default=False)

    def __str__(self):
        return self.title

Heroku config vars

=== hidden-escarpment-87695 Config Vars
AWS_ACCESS_KEY_ID:       <mys3key>
AWS_SECRET_ACCESS_KEY:   <mysecretkey>
AWS_STORAGE_BUCKET_NAME: 4f2xivbz443
DATABASE_URL:            postgres://wslatgvzefvimv:eebae0eeba511f1b8e8fe5c3c23a28740182dcfd7eb02138e8826c809a6967f1@ec2-176-34-186-178.eu-west-1.compute.amazonaws.com:5432/d21efhek9bf2u
7
SECRET_KEY:              <myownsecretkey>

heroku build log

-----> Python app detected

     $ pip install -r requirements.txt

     $ python manage.py collectstatic --noinput

       Traceback (most recent call last):

         File "manage.py", line 10, in <module>

           execute_from_command_line(sys.argv)

         File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line

           utility.execute()

         File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/__init__.py", line 359, in execute

           self.fetch_command(subcommand).run_from_argv(self.argv)

         File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/base.py", line 294, in run_from_argv

           self.execute(*args, **cmd_options)

         File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/base.py", line 345, in execute

           output = self.handle(*args, **options)

         File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 193, in handle

           collected = self.collect()

         File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 124, in collect

           handler(path, prefixed_path, storage)

         File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 347, in copy_file

           self.storage.save(prefixed_path, source_file)

         File "/app/.heroku/python/lib/python3.6/site-packages/django/core/files/storage.py", line 54, in save

           return self._save(name, content)

         File "/app/.heroku/python/lib/python3.6/site-packages/storages/backends/s3boto3.py", line 452, in _save

           self._save_content(obj, content, parameters=parameters)

         File "/app/.heroku/python/lib/python3.6/site-packages/storages/backends/s3boto3.py", line 467, in _save_content

           obj.upload_fileobj(content, ExtraArgs=put_parameters)

         File "/app/.heroku/python/lib/python3.6/site-packages/boto3/s3/inject.py", line 509, in object_upload_fileobj

           ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)

         File "/app/.heroku/python/lib/python3.6/site-packages/boto3/s3/inject.py", line 427, in upload_fileobj

           return future.result()

         File "/app/.heroku/python/lib/python3.6/site-packages/s3transfer/futures.py", line 73, in result

           return self._coordinator.result()

         File "/app/.heroku/python/lib/python3.6/site-packages/s3transfer/futures.py", line 233, in result

           raise self._exception

         File "/app/.heroku/python/lib/python3.6/site-packages/s3transfer/tasks.py", line 126, in __call__

           return self._execute_main(kwargs)

         File "/app/.heroku/python/lib/python3.6/site-packages/s3transfer/tasks.py", line 150, in _execute_main

           return_value = self._main(**kwargs)

         File "/app/.heroku/python/lib/python3.6/site-packages/s3transfer/upload.py", line 679, in _main

           client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)

         File "/app/.heroku/python/lib/python3.6/site-packages/botocore/client.py", line 253, in _api_call

           return self._make_api_call(operation_name, kwargs)

         File "/app/.heroku/python/lib/python3.6/site-packages/botocore/client.py", line 543, in _make_api_call

           raise error_class(parsed_response, operation_name)

       botocore.exceptions.ClientError: An error occurred (InvalidRequest) when calling the PutObject operation: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

 !     Error while running '$ python manage.py collectstatic --noinput'.

       See traceback above for details.

       You may need to update application code to resolve this error.

       Or, you can disable collectstatic for this application:

          $ heroku config:set DISABLE_COLLECTSTATIC=1

       https://devcenter.heroku.com/articles/django-assets

 !     Push rejected, failed to compile Python app.

 !     Push failed

Answer 1

Heroku: S3 / boto3 - error:

Please use AWS4-HMAC-SHA256, For people facing AWS4-HMAC-SHA256 error in django-storages (images not loading).

Try using this values in settings.py:

if using boto3:

AWS_S3_REGION_NAME = 'us-east-2' #change to your region
AWS_S3_SIGNATURE_VERSION = 's3v4'

if using boto:

AWS_S3_HOST = 'us-east-2' #change to your region
S3_USE_SIGV4 = True

Answer 2

You shouldn't use S3 for storing static files while using WhiteNoise to serve them. (Using S3 for storing and serving media files is fine -- indeed, it's encouraged.)

Just remove this line:

STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

and this line

STATIC_URL = 'http://' + AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/'

And you can leave the other settings as they are.

If you want to use WhiteNoise with Cloudfront there are instructions here .

PS In your Heroku config vars you've posted your database credentials so anyone can connect to your database. You can reset them here .