阅读单行泡菜python

Question

im doing some simulations with python and save the results using pickle.dump(). Now i have an enormous amount of data wich i need to read. The data consist of matrices (a lot of lines of numbers) each saved in a single file. I need to be able to read the information efficiently.

• How can i read a single line of the matrix withot uploading the complete file?

• How can i read a specific set of lines (say from the line 1000 to the 2000) whithout uploading the complete file? Here is the code i used to save the data:

   for j in range(61): for l in range(31): X = zeros([100000,100]) K = k[j] A = a[l] X = RK4(deriv,dt,pasos,xo,n,w,b,K,g,A,V) pickle.dump( X, open( 'Vrc'+'dk_'+ str(K) + 'alpha_' + str(A) , "wb" ) ) 

Answer 1

Try using generators when doing pickle.load(): yield pickle.load(...)

Also have a look at cPickle .