[英]The thread hangs using FTP LIST with Python
I'm using ftplib for connecting and getting file list from FTP server. 我正在使用ftplib从FTP服务器连接和获取文件列表。 The problem I have is that the connection hangs from time to time and I don't know why.
我的问题是连接不时挂起,我不知道为什么。 I'm running python script as a daemon, using threads.
我正在使用线程将python脚本作为守护程序运行。 See what I mean:
明白了吗:
def main():
signal.signal(signal.SIGINT, signal_handler)
app.db = MySQLWrapper()
try:
app.opener = FTP_Opener()
mainloop = MainLoop()
while not app.terminate:
# suspend main thread until the queue terminates
# this lets to restart the queue automatically in case of unexpected shutdown
mainloop.join(10)
while (not app.terminate) and (not mainloop.isAlive()):
time.sleep(script_timeout)
print time.ctime(), "main: trying to restart the queue"
try:
mainloop = MainLoop()
except Exception:
time.sleep(60)
finally:
app.db.close()
app.db = None
app.opener = None
mainloop = None
try:
os.unlink(PIDFILE)
except:
pass
# give other threads time to terminate
time.sleep(1)
print time.ctime(), "main: main thread terminated"
MainLoop() has some functions for FTP connect, download specific files and disconnect from the server. MainLoop()具有一些用于FTP连接,下载特定文件以及与服务器断开连接的功能。
Here's how I get the file's list: 这是我获取文件列表的方法:
file_list = app.opener.load_list()
And how FTP_Opener.load_list() function looks like: 以及FTP_Opener.load_list()函数的外观如下:
def load_list(self):
attempts = 0
while attempts<=ftp_config.load_max_attempts:
attempts += 1
filelist = []
try:
self._connect()
self._chdir()
# retrieve file list to 'filelist' var
self.FTP.retrlines('LIST', lambda s: filelist.append(s))
filelist = self._filter_filelist(self._parse_filelist(filelist))
return filelist
except Exception:
print sys.exc_info()
self._disconnect()
sleep(0.1)
print time.ctime(), "FTP Opener: can't load file list"
return []
Why sometimes the FTP connection hangs and how can I monitor this? 为什么有时FTP连接挂起,我该如何监控? So if it happens I would like to terminate the thread somehow and start a new one.
因此,如果发生这种情况,我想以某种方式终止线程并启动一个新线程。
Thanks 谢谢
If you are building for robustness, I would highly recommend that you look into using an event-driven method. 如果要增强鲁棒性,我强烈建议您考虑使用事件驱动的方法。 One such which have FTP support is Twisted ( API ).
具有FTP支持的一种就是Twisted ( API )。
The advantage is that you don't block the thread while waiting for i/O and you can create simple timer functions to monitor your connections if you so prefer. 优点是您在等待I / O时不会阻塞线程,并且可以根据需要创建简单的计时器函数来监视连接。 It also scales a lot better.
它的伸缩性也更好。 It is slightly more complicated to code using event-driven patterns, so if this is just a simple script it may or may not be worth the extra effort, but since you write that you are writing a daemon, it might be worth looking into.
使用事件驱动的模式编写代码稍微复杂一些,因此,如果这只是一个简单的脚本,则可能值得或不值得付出额外的努力,但是由于您编写的是编写守护程序,因此值得研究。
Here is an example of an FTP client: ftpclient.py 这是FTP客户端的示例: ftpclient.py
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.