无法使用Django从S3提供静态媒体

Question

I need to use Amazon S3 to serve my static and media files to my Django project.

However, I am facing a lot of issues with that. First, my code:

s3utils.py

from storages.backends.s3boto import S3BotoStorage

class FixedS3BotoStorage(S3BotoStorage):
def url(self, name):
    url = super(FixedS3BotoStorage, self).url(name)
    if name.endswith('/') and not url.endswith('/'):
        url += '/'
    return url

StaticS3BotoStorage = lambda: FixedS3BotoStorage(location='static')
MediaS3BotoStorage = lambda: FixedS3BotoStorage(location='media')

In settings.py

DEFAULT_FILE_STORAGE = 'SpareGuru.s3utils.MediaS3BotoStorage'
STATICFILES_STORAGE = 'SpareGuru.s3utils.StaticS3BotoStorage'

AWS_HOST = "s3-ap-southeast-1.amazonaws.com"
AWS_ACCESS_KEY_ID = 'xx'
AWS_SECRET_ACCESS_KEY = 'yy'
AWS_STORAGE_BUCKET_NAME = 'zz'

S3_URL = 'http://%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
MEDIA_DIRECTORY = '/media/'
STATIC_DIRECTORY = '/static/'

STATIC_URL = "/static/"

MEDIA_URL = "/media/"

STATIC_ROOT = S3_URL + STATIC_DIRECTORY
COMPRESS_ROOT = STATIC_ROOT
MEDIA_ROOT = S3_URL + MEDIA_DIRECTORY

Here are the issues I face:

1. When running ./manage.py collectstatic , it starts to upload the files to S3 and after a couple of files, I get Broken Pipe error.

2. When trying to run the webpage, I get the error: 'https://zz.s3.amazonaws.com/static/css/bootstrap.min.css?Signature=sign&Expires=1438359937&AWSAccessKeyId=xx' isn't accessible via COMPRESS_URL ('/static/') and can't be compressed .

No idea what's going on here.

Edit: My previous bucket policy allowed for Read only access. So maybe that could have been the reason that my compressor was unable to create files on S3. I have updated the policy but still doesn't work. Updated policy is:

{
"Statement": [
    {
        "Sid": "PublicReadForGetBucketObjects",
        "Effect": "Allow",
        "Principal": {
            "AWS": "*"
        },
        "Action": [
            "s3:GetObject"
        ],
        "Resource": [
            "arn:aws:s3:::zz/*"
        ]
    },
    {
        "Action": "s3:*",
        "Effect": "Allow",
        "Resource": [
            "arn:aws:s3:::zz",
            "arn:aws:s3:::zz/*"
        ],
        "Principal": {
            "AWS": [
                "my-arn:/goes=here"
            ]
        }
    }
]
}

And the CORS Configuration for my bucket is:

<CORSConfiguration>
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>Authorization</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

Answer 1

I have this running on my own Django project. I went and checked out the differences. One thing you might try is adding:

AWS_QUERYSTRING_AUTH = False

to your settings.py. That could help with at least your issue #2.

Here's a full example from my settings.py. Note: I don't use S3 for static files, only media.

from boto.s3.connection import SubdomainCallingFormat
AWS_CALLING_FORMAT = SubdomainCallingFormat()
DEFAULT_FILE_STORAGE = 'project.s3utils.MediaRootS3BotoStorage'
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID', None)
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_KEY', None)
AWS_SECRET_KEY = os.getenv('AWS_SECRET_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_QUERYSTRING_AUTH = False
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, '../',  'static')
MEDIA_ROOT = os.path.join(BASE_DIR, '../..',  'media')
MEDIA_URL = 'http://{!s}.s3.amazonaws.com/media/'.format(AWS_STORAGE_BUCKET_NAME)

Answer 2

I believe your static and media urls/roots are incorrect. They should follow the format below:

MEDIA_URL = S3_URL + MEDIA_DIRECTORY
STATIC_URL = S3_URL + STATIC_DIRECTORY
MEDIA_ROOT = '/home/host/app/static_root/media'
STATIC_ROOT = '/home/host/app/static_root/static'

Note: Change the MEDIA_ROOT and STATIC_ROOT to how your project is configured.

EDIT:

Sorry, didn't see you were using compressor. Try doing the following:

COMPRESS_URL = S3_URL + STATIC_DIRECTORY
STATIC_URL = COMPRESS_URL
MEDIA_URL = S3_URL + MEDIA_DIRECTORY
MEDIA_ROOT = '/home/host/app/static_root/media'
STATIC_ROOT = '/home/host/app/static_root/static'