Upload video to amazon s3 using django and python

Question

All i am trying is to upload a video to amazon s3 instead of saving in media as django does regularly. So created a below model

def upload_to_amazon_s3(instance, filename):
    aws_access_key_id = settings.AWS_ACCESS_KEY_ID
    aws_secret_access_key = settings.AWS_SECRET_ACCESS_KEY

    file_name = 'videos/{0}_{1}/{2}'.format(instance.user.id, instance.user.username, filename)

    conn = boto.connect_s3(aws_access_key_id, aws_secret_access_key)
    try:
        bucket = conn.get_bucket("ipitch_videos", validate=True)
    except S3ResponseError:
        bucket = conn.create_bucket('ipitch_videos')

    #Get the Key object of the bucket
    k = Key(bucket)
    #Crete a new key with id as the name of the file
    k.key=file_name

    #Upload the file
    result = k.set_contents_from_file(instance.video_file)

    # we need to make it public so it can be accessed publicly
    # using a URL like http://s3.amazonaws.com/bucket_name/key
    k.make_public()

    get_s3_obj = Key(bucket,file_name)
    endpoint_url = get_s3_obj.generate_url(expires_in=0, query_auth=False)
    return endpoint_url

class Video(DateTimeModel):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    name = models.CharField(max_length=255)
    video_file = models.FileField(upload_to=upload_to_amazon_s3)

I am getting the endpoint_url value from amazon s3 Url for example as https://ipitch_videos.s3.amazonaws.com/videos/1_username/super.mpg

So when i upload a video with from front end form or an API with all fields user, name, video_file , a record is being created in to the database successfully but the following three problems occur

1. Video record instance is saving but splitting / after https:// from endpoint_url that comes from amazon and using only https:/ instead of https:// as below

2. Along with the amazon uploading process, a video file was being created locally too with the path that comes from endpoint_url like for example if end point url is https://ipitch_videos.s3.amazonaws.com/videos/1_username/super.mpg django is creating the following folders under media as below

So how to avoid django creating video files locally too ?

Answer 1

You can use S3BotoStorage.

$ pip install django-storages boto Add 'storages' to INSTALLED_APPS:

INSTALLED_APPS = (
      ...,
      'storages',
 )

adding another storage class:

class MediaStorage(S3BotoStorage):
    location = settings.MEDIAFILES_LOCATION

and in settings.py:

MEDIAFILES_LOCATION = 'media'
MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)
DEFAULT_FILE_STORAGE = 'custom_storages.MediaStorage'