python-分析抓取数据时的最佳方法

Question

Newbie here. I have managed to put together a script which scrapes some information from a website. This happens daily, and the data is saved on a csv file. content of each file is similar to this:

date, ticker, company name, momentum indicator, other ratios....
2016-08-19, GSK, GlaxoSmithKline, 42, ....
2016-08-19, RDSB, Royal Dutch Shell, 98, .....
....

I have accumulated 3 months worth of daily data, so around 80 files. (Every row in the file has the same date and then the different shares). What I would like to do now is to check, on a share by share basis, the evolution of the momentum indicator and other ratios.

for example, I think I should end up with a series of lists such as

GSK_momentum_indicator = (42, 43, 38, 47,...) 
RDSB_momentum_indicator = (98, 91, 77, 79,...)

Now, as a newbie, I have 2 questions: 1) what do you think is the best approach for this? Is it using lists, dictionaries, anything else? 2) how did you decide the above? are there guidelines for which strategy to use? is there a good resource I can read as a newbie to learn more about this subject?

thanks!

PS. in case it makes a difference, I'm using python 3.5.2.

Answer 1

In order to process the data you've collected, you could use one of the python modules, csv or pandas . The csv module is used to read/write data from/to csv files and then you can convert the data into python lists and dictionaries and use accordingly. For detailed docs go here .

But if you have large dataset then you should go for pandas which a specialized tool for data analysis. The pandas.read_csv function takes the name of the csv file as argument and returns a DataFrame object on which you can perform various operation. For detailed docs go here .