[英]Program class is stuck/idle and does not execute remaining calls after 1st call in Anaconda/Command Line Prompt but works in Spyder
I am trying to use the anaconda prompt to run my python script. 我正在尝试使用anaconda提示符运行我的python脚本。 It runs smoothly on the first call but stops there. 它在第一次通话时运行平稳,但在此停止。 I tried on Spyder, it works but I would like it to work on anaconda prompt or command line. 我在Spyder上尝试过,它可以工作,但我希望它在anaconda提示符或命令行下工作。 Any reason why? 有什么原因吗?
from decompress import decompress
from reddit import reddit
from clean import clean
from wikipedia import wikipedia
def main():
dir_of_file = r"D:\Users\Jonathan\Desktop\Reddit Data\Demo\\"
print('0. Path: ' + dir_of_file)
reddit_repo = reddit()
wikipedia_repo = wikipedia()
pattern_filter = "*2007*&*2008*"
print('1. Creating data lake')
reddit_repo.download_files(pattern_filter,"https://files.pushshift.io/reddit/submissions/",dir_of_file,'s')
reddit_repo.download_files(pattern_filter,"https://files.pushshift.io/reddit/comments/",dir_of_file,'c')
if __name__ == "__main__":
main()
The RS Downloaded is this line of code being ran: RS Downloaded是正在运行的以下代码行:
reddit_repo.download_files(pattern_filter,"https://files.pushshift.io/reddit/submissions/",dir_of_file,'s')
Added the class/function 添加了类/功能
class reddit:
def multithread_download_files_func(self,list_of_file):
filename = list_of_file[list_of_file.rfind("/")+1:]
path_to_save_filename = self.ptsf_download_files + filename
if not os.path.exists(path_to_save_filename):
data_content = None
try:
request = urllib.request.Request(list_of_file)
response = urllib.request.urlopen(request)
data_content = response.read()
except urllib.error.HTTPError:
print('HTTP Error')
except Exception as e:
print(e)
if data_content:
with open(path_to_save_filename, 'wb') as wf:
wf.write(data_content)
print(self.present_download_files + filename)
def download_files(self,filter_files_df,url_to_download_df,path_to_save_file_df,prefix):
#do some processing
matching_fnmatch_list.sort()
p = ThreadPool(200)
p.map(self.multithread_download_files_func, matching_fnmatch_list)
It was the download which was taking a lot of time. 下载需要花费大量时间。 I have changed network and it worked as expected. 我更改了网络,它按预期工作。 So there is no issue with cmd or anaconda prompt 因此,cmd或anaconda提示符没有问题
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.