Distributing PHP PDF generation across servers

Question

I have to generate 5-10MB PDFs from a product catalogue database with images.

I've been using dompdf (and a few others) which have been working great. The trouble is the time it takes to execute, around 20 seconds.

I wonder if there is a way to offload these tasks to some form of distributed computing service such as Amazon's offerings (elastic beanstalk maybe?). We can produce certain pages across a few instances then stitch them back together when they have completed their tasks.

Does anyone have any service recommendations for this?

Answer 1

The best way to architect ( considering the scalability factor ) would be use the components like SQS, SNS, S3, EC2. This enable you to grow the processing / EC2 capacity to Scale up / Scale Out without much of work on other services.

You can start with a single Instance which is deployed with the pdf generators like (dompdf) with a portal

1. User Initiates a pdf creation Job using the Custom developed Portal ( Running out of EC2 )
2. On successful initiation / acknowledgement of the the pdf job - you can prompt the user with the Job ID
3. The Process running in the EC2 can finish the pdf job and put it back to S3
4. On successful completion of the EC2 the job can trigger SNS notification / SES emails
5. The status of the job and running status can be tracked in DB services like SimpleDB or DynamoDb
6. The completed PDF can be obtained from S3 using the Job Portal with the Job ID

Answer 2

elastic beanstalk (EB) is deploy tool, I don't think it is suitable for your case.

The idea you are talking about is very similar as the exist aws service transcoding , but that's for audio and video. Maybe you can take time to investigate how transcoding works and make the same for your pdf generator. Or you can contact AWS engineer for your idea, maybe they can develop a new service for you. It seems a common request.

My thought for your problem currently is to set an scaling environment as @NAveen recommend, but more focus on autocaling.

• Set a launch configuration with user data (include how to install the web service and pdf generator) and desired instance type (micro, small or large, etc), ELB (elastic load balancer is set in this step as well)
• assign this launch configuration to an autoscaling group, you can set min instance and max instance size.
• set scale up and scale down rule to increase/decrease ec2 instance numbers.

So when the request is sent to ELB, it will assign to attached instances and distribute the load to each instance, and will scale instance number automatically with the real traffic.