Amazon Linux EC2 Webserver / MYSQL Upgrade – Traffic causing error establishing a database connection

Question

To give you a little background, I currently have a website that allows users to upload photos. The website was initially on a GoDaddy shared server, but recent surges in traffic have forced me to explore other options. During peak hours, the site contains 400+ active visitors, which when combined with user uploads, forces the shared server to shut down.

I have a small amount of experience with setting up servers through AWS and attempted to place the website on a c1.medium instance, Amazon Linux. The website along with the MYSQL Database is on the same instance. While I have read that this is in general frowned upon, I have similarly read that moving the database to another instance would not significantly increase speeds. Unfortunately, the c1.medium instance also was unable to support the traffic and I soon received an error Establishing a Database connection. The site does load on occasion, so the problem stems from the traffic load and not an actual problem with the database.

My question is whether the problem revolves solely around MySQL? The database itself when backed up is around 250MB. Is the issue caused by input / output requests to the database? I read posts with people with similar problems in which they stated that installing MySQL 5.6 solved the problem, but also have read that MySQL 5.6 is slower than MySQL 5.5, which is my current version.

After conducting some preliminary research I started to believe that I could resolve the problem by increasing the IPOS of the EBS. Originally I had it set the IPOS as standard, but changed it to Provisioned IOPS and 30x the size of the EBS (ie, 60GB – 1800 IOPS). This once again appeared to have little impact. Do I need to upgrade my instance? What measures should I be focused on when deciding on the instance? It appears that the cheapest instance with high network performance and EBS optimized would be c3.xlarge. Suggestions?

Answer 1

Several things to consider:

1)Separate the database server from the web server

Your database should not share resources with your web server. They will both perform poorly as the result. It is easier to find what the bottle-neck is.

2) Upgrade to MySQL 5.6

In all the benchmarks that I have seen and done 5.6 performs better than 5.5

3) Configure your database to take advantage of your resources

Depending on the storage engine and the memory allocated in your machine configure MySQL for example set innodb_buffer_pool_size to 70% of the (DEDICATED) RAM

4) Monitor MySQL and check slow query log

Slow query log shows the queries that are slow and inefficient

5) Learn to use EXPLAIN

EXPLAIN shows query plan in MySQL run EXPLAIN on slow queries to tune them

6) Use Key-Value Stores to Cache queries

Using Memcached or Redis cache queries so they don't hit your database and return repeated queries from the memory

7) Increasing IOPS and Scaling Out

Increasing IOPS and getting better hardware helps but using efficient queries is much more effective. Queries and application most of the time are a greater contributing factor to performance problems

8) Replication

To help with concurrency consider moving to a MySQL Master/Slave replication , if you still had issues.

Final Note: use EBS because the storage on EC2 is ephemeral and will not persistent.

Answer 2

We recently did extensive research on the performance bottlenecks associated with massive end-user peaks across our global customer base, and the analysis actually indicates the database as - by far - the most frequent cause of slowdowns or even crashes. The report ( https://queue-it.com/trend-report ) includes best practice advice from our customers on how to improve the situation, which you may find helpful.