EC2 instance for Python 3 script

Question

I have a script written in Python and it's taking a fairly long time to process (around 30 minutes) on my laptop. I was thinking I could create a EC2 instance in AWS and try to see if it's possible to speed up the process. I have a AWS account so my question:

Which EC2 instance type should I create in order to run the process in a faster way? Process reads a csv file does some calculations and then writes csv with results. Script bottleneck is in the mathematical calculations as csv files are fairly small.

I can go with either a free tier or paid tier instance.

Answer 1

I would say go with p2 extra large If you have to use ec2.

Try to understand what's causing this delay. Which library are you using to read csv. There are various ways in python through which you can manipulate the csv file For example see the image:

Image source here

NumPy, SciPy, joblib and hdf5 are the recommended options for quickly saving and loading csv data.

Try to change your algorithm. In my experience pandas is not speedy when it comes to csv operations. Try to tweak your code if that dosent work switch to p2 extra large