如何从 .csv 网络文件中提取数据并存储到变量中，并可以选择稍后将其存储在数据库中

Question

I'm start to learn python and need your help. I program in my last time only with Matlab.

I want to pul as a while loop data form a .csv web file and if the date variable change, append the data to the old and store it in an (cell-)array with the option later to store it in the database.

So my code in python:

import csv
import urllib.request

url = "http://download.finance.yahoo.com/d/quotes.csv?s=AGRO.BA,PR15.BA,&f=sl1d1t1c1ohgv&e=.csv"
ftpstream = urllib.request.urlopen(url)
csvfile = csv.reader(ftpstream.read().decode('utf-8')) #with the appropriate encoding
data = [row for row in csvfile]

the output is: (the data means: tick, value, date, time, change, mean, high, low, value)

['AGRO.BA', '3.14', '10/28/2015', '4:49pm', '+0.09', '3.05', '3.25', '3.00', '150988'] ['BA-C.BA', '115.00', '10/21/2015', '11:41am', '+0.00', '115.00', '115.00', '115.00', '100'] ...

As I can understand it in matlab, python save data in 2 rows as strings in data.

1. Question: How can I save it in data in cellarrays with different variable types?

2. Question: How can I save the date as an datenum (datenumber)?

3. Question: How can I change the format of time in 24h format without 'am' or 'pm' and so store it as time or double not as string?

4. Question: How can I pul the data in while loop and if the time change, the program load the old stored data(cell-)array and attend the new data to the old data (not overwrite but attend to the old!)?

Thanks!

Answer 1

Sigh. What you're supposed to do if someone posts a pointer like that, is use Google to find documents relating to "Python" and "pickle". Without knowing it's called pickle, it takes a lot longer to find the python documentation for what you need.

However, for the benefit of posterity, here's the absolute basics, as two separate interpreter sessions

Example of saving some Python objects (a dict and a list-structure), in a way that allows the objects to be recreated later

 >>> d=dict(a=1,c=3,e=5)
 >>> d
 {'a': 1, 'c': 3, 'e': 5}
 >>> L=['hello','goodbye',['a','list','in','a','list']]

 >>> f = open('temp.tmp','wb')  # open a pickle file
 >>> import pickle
 >>> pickle.dump( d, f)  # save d to the pickle file
 >>> pickle.dump( L, f)  # followed by 
 >>> f.close()

and exit. What's in the file can now be read back in by another Python program. Note that there are multiple versions of the pickle protocol, so if you want to read in to a different version of Python to the one that did the writing, you may need to find out about the optional protocol argument to the pickle functions. That aside, let's read that file ....

>>> import pickle 
>>> f=open('temp.tmp','rb')
>>> something=pickle.load(f)
>>> something
{'c': 3, 'a': 1, 'e': 5}
>>> something2 = pickle.load(f)
>>> something2
['hello', 'goodbye', ['a', 'list', 'in', 'a', 'list']]
>>> 

You can pickle most sorts of Python objects but not all of them. Lists and dicts and nests of them are fine.