AWS Cors Issue - Node.js

Question

I am currently experiencing an issue when requesting images stored on AWS S3 (simple storage) with the CORS header. I have set up the CORS configuaration on the AWS console - and the setup is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>Authorization</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

However, the response I get when requesting these images that I've stored are a little all over the place and quite intermittent. Sometimes the image is returned with the headers requred and sometimes it is not. I am really not sure why this happens. It also seems that the effect worsens when I try to make more than one request for an image with the Access-Control-Allow-Origin header set as * on a page (eg If I need 10 images to be retrieved all with cross origin headers).

These are the headers that I need:

Access-Control-Allow-Methods:GET
Access-Control-Allow-Origin:*
Access-Control-Max-Age:3000

I'm really not sure what I'm doing incorrectly. I have made sure that each image tag has the crossOrigin="anonymous" attribute added to it too, but again, no luck.

The reason for me needing these images to work cross origin is because I have installed an angular plugin which allows the user to crop images and store the cropped versions of the images as base64 strings. However, I get the following error when trying to retrive them.

These are the headers for an image that is returned correctly:

Request URL:https://trajansmarket.s3.amazonaws.com/be5bbda0-b04a-11e5-81d3-dd7ff3efeebc.jpg
Request Method:GET
Status Code:304 Not Modified
Remote Address:54.231.252.131:443

Response Headers
view source
Access-Control-Allow-Methods:GET
Access-Control-Allow-Origin:*
Access-Control-Max-Age:3000
Cache-Control:public, max-age=31536000
Date:Tue, 12 Jan 2016 21:13:03 GMT
ETag:"77bdbe9b517acc8cba86024c592bce3f"
Last-Modified:Fri, 01 Jan 2016 05:46:21 GMT
Server:AmazonS3
Vary:Origin, Access-Control-Request-Headers, Access-Control-Request-Method
x-amz-id-2:F3OQpOHsAqySk9LNwwoJXVATVIByr4Gtvz953ZoL7DdB/dtE9nYwo99R59Rj6RzZc3dcHyk6wWY=
x-amz-request-id:CD220FF1F6EE6CA9

Request Headers
view source
Accept:image/webp,image/*,*/*;q=0.8
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6,ms;q=0.4
Connection:keep-alive
Host:trajansmarket.s3.amazonaws.com
If-None-Match:"77bdbe9b517acc8cba86024c592bce3f"
Origin:http://91.121.220.161:3000
Referer:http://91.121.220.161:3000/
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36

And here's one without the headers:

Request URL:https://trajansmarket.s3.amazonaws.com/c0671e00-b04a-11e5-81d3-
dd7ff3efeebc.jpg
Request Method:GET
Status Code:200 OK (from cache)
Remote Address:54.231.252.135:443

Response Headers
Accept-Ranges:bytes
Cache-Control:public, max-age=31536000
Content-Length:142102
Content-Type:application/octet-stream
Date:Tue, 12 Jan 2016 00:35:36 GMT
ETag:"beb93f56e3a2a65b983addd8af35c26c"
Last-Modified:Fri, 01 Jan 2016 05:46:25 GMT
Server:AmazonS3
x-amz-id-2:5XvaOd8bxMr5zwK317DfDMbk2+kzu3Zd7rsf2xl0hxwI40Oc4KDnQpgzD3sgtCRm9SXGqa93Mh0=
x-amz-request-id:FD3EB1978C38013B

Request Headers
Provisional headers are shown
Accept:image/webp,image/*,*/*;q=0.8
Origin:http://91.121.220.161:3000
Referer:http://91.121.220.161:3000/
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
X-DevTools-Emulate-Network-Conditions-Client-Id:498F45FE-5D49-4AE0-AF58-F81B9AFD48AF

I'm just wondering if anyone would have any idea of why this would be happening. Any help would be greatly appreciated.

Answer 1

This was a very frustrating and I am still yet to work out the reason for AWS S3 intermittently returned the headers that I need for CORS.

I have since thought of a workaround which is to 'download' and store the images that I need from amazon in a local folder - allow the user to 'crop' the image and store it before then deleting these images from the local folder.

To stream the image files to a local folder I used the fs.createWriteStream method on the .getObject method from s3. An example of this can be found at: enter link description here

This has taken away the need for me to actually request the images with the CORS headers as when they are stored locally, the headers are no longer required. I can then save the base64 that is generated by my cropper directive and store this on amazon S3 easily.

If the user navigates away before cropping the local images I delete them from the local folder so it doesn't get clogged up.

I hope this is of some help to those who are also having issues with the CORS headers - even though it's just a workaround.

Answer 2

Well I've seen this problem in a number of different forms: one is where you are serving a page in S3 that access a nodejs backend in either an EC2 or in Elastic Beanstalk.

In one case it was the browser I was using IE10 that threw errors because the browser required Preflight options to be set.

In the other cases I was using Restify in Elastic Beanstalk and Angular in S3. I added the Restify-cors middleware package to the request:

var corsMiddleWare = require('restify-cors-middleware'); //npm install this package
var cors = corsMiddleWare ({
    allowHeaders:['Authorization', 'API-Token', 'API-Token-Expiry']
 });
server.pre(cors.preflight);
server.use(cors.actual);
//rest of server definition

This seemed to work. In the case of express there is the node package express-cors:

var cors = require('cors');
app.use(cors());

In either case the key is that all requests must have the headers set properly, hence we add them to middleware. (app/server.use) How can I support cors when using restify

Are you using vanilla node? In that case you need to add the headers on every request you make to s3.

Answer 3

This was definitely an issue with it being a cached response for me. It seems to lose some of the headers in cache. I was able to get it to work 100% of the time by appending a random time parameter to the image name, as in the example below.

img = new Image();
img.src = "https://s3.amazonaws.com/bucket/img.png?t=" + (new Date().getTime() / 1000);    
img.crossOrigin = "Anonymous";