简体   繁体   中英

Python sub-module class instance generation

I want to place the generation of random numbers into a sub-module ("rv_gen.py"); it is not clear to me whether an instance of the random number array (or of the class?) is generated upon the import statement, or only when assigning this to the variable rvclass (in the main module)?

Puzzling is the timing: the output (see below) differs after I change the size of the random number array, and takes unexpectedly long (I expect a few sec for 10M normal rvs):

After a change (and saving) of "rv_gen", the first three lines of the output below until the second print('rvs generation finished!') are printed immediately, however the next print statement 'Time for import:...' takes several minutes (not just "3.1 seconds" as stated). Then the remaining three prints follow immediately. I had expected the main time lag before each 'rvs generation finished' print...?

If I just rerun the main module without any change to the sub-module, the output is similar except missing the first line print('rvs generation finished!') below, and does not take additional minutes before the statement now reading Time for import: 0.3 seconds .

Main module "main.py":

import timeit
lastt   = timeit.default_timer()
import rv_gen
thist   = timeit.default_timer(); print('Time for import: %4.1f seconds.' % ( thist-lastt)) ; lastt = thist     
rvclass = rv_gen.s
thist   = timeit.default_timer(); print('Time for assignment: %4.1f seconds.' % ( thist-lastt)) ; lastt = thist     
print(rvclass.rans[:5])

Module "rv_gen.py":

import numpy as np
class struct:
    pass
s      = struct()
s.rans = np.random.normal(size=1000*1000*50) 
print('rvs generation finished!')

Output (after updating rv_gen.py):

rvs generation finished!
Reloaded modules: rv_gen
rvs generation finished!
Time for import:  3.1 seconds.
Time for assignment:  0.0 seconds.
[ 0.925 -0.728  2.387 -0.683 -0.021]

Edit: Now getting a "MemoryError": I just reran the code after changing the random array size to 10M, and got the following output (again the first rvs generation finished! printout appearing immediately, the rest after about 3 minutes):

rvs generation finished!
[autoreload of rv_gen failed: Traceback (most recent call last):
  File "C:\Anaconda\lib\site-packages\IPython\extensions\autoreload.py", line 245, in check
    superreload(m, reload, self.old_objects)
  File "C:\Anaconda\lib\site-packages\IPython\extensions\autoreload.py", line 450, in superreload
    update_generic(old_obj, new_obj)
  File "C:\Anaconda\lib\site-packages\IPython\extensions\autoreload.py", line 387, in update_generic
    update(a, b)
  File "C:\Anaconda\lib\site-packages\IPython\extensions\autoreload.py", line 357, in update_class
    update_instances(old, new)
  File "C:\Anaconda\lib\site-packages\IPython\extensions\autoreload.py", line 312, in update_instances
    update_instances(old, new, obj.__dict__, visited)
  File "C:\Anaconda\lib\site-packages\IPython\extensions\autoreload.py", line 317, in update_instances
    update_instances(old, new, obj, visited)
  File "C:\Anaconda\lib\site-packages\IPython\extensions\autoreload.py", line 302, in update_instances
    visited.update({id(obj):obj})
MemoryError
]
Reloaded modules: rv_gen
rvs generation finished!
Time for import:  0.4 seconds.
Time for assignment:  0.0 seconds.
[-0.091  1.087 -0.329  0.359 -1.884]

I am using Anaconda Spyder 3.3.6 and IPython.

the statements in the rv_gen would run upon the time of import. There is no much gap between your prints and even the generation of 100k random numbers will take no time in morden PCs

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM