Download speed from S3 and EC2 storage, request from Node js and browser

Question

Me and my team are creating a mobile game in which a map is available. We store json information in multiple files - each file represents a tile on the map. To render the map, we download the files and process them to create streets, buildings etc. I want to choose the best way to download tile files to the mobile devices but I didn't have the possibility to do this test on the mobile devices so i used a browser and node js scripts.

I used a 100KB json file. Uploaded it on an S3 bucket and on EC2 storage. I wrote a few node scripts to connect to the S3 or EC2:

• GET request from Node js local script to S3 bucket (bucket.zone.amazonaws.com/file) - ~650ms
• GET request from Node js local script to Node js server run on EC2 instance which connects to S3 - ~1032ms
• GET request from Node js local script to Node js server run on EC2 instance which loads the file from local storage - ~833ms

The difference between the last two values is actually the time added for EC2 instance to access the file from the bucket. And the reason for making a request to S3 from EC2 is that I know that connections between AWS services is really fast.

The other test I made was from the browser (Firefox):

• Directly accessed the S3 bucket (bucket.zone.amazonaws.com/file) - ~624ms with values between 400ms and 1000ms
• Through the Apache server on EC2 (domain/file) - ~875ms with values between 649ms and 1090ms
• Through Node js server which connects to S3 bucket (run on EC2) (domain:port) - ~1014ms with values between 680ms and 1700ms
• Through Node js server which loads the file from local storage (run on EC2) (domain:port) - ~ 965ms with values between 600ms and 1700ms

My question is why is such a big difference between accessing the file from the browser and accessing it through Node script?

For logging the times, I made each request for 10 times and I did the mean of times. EC2 instance is micro, in Ireland. The bucket is situated in Ireland too.

Answer 1

I propose several clue that may help you to profile.

1. Cache, when you use script to get the json data. The cache mechanism would not work. While in browser, it will honor the cache header and may fetch from the cache, thus decrease the speed.

2. GZip header, I think you would not enable gzip to compress your data in nodejs server. I am not sure if you configured such on Apache. Imagine a json file with 100k, if it is compressed, the transfer time would surely be decreased.

Thanks

Answer 2

因此，我认为这个问题已经没有意义了，因为即使硬刷新页面，时间也越来越多。