[英]How to improve read remote system files with high performance in python
I have written code for reading the remote system files that is working fine but it is taking much time to read remote files. 我已经编写了用于读取远程系统文件的代码,该代码工作正常,但是读取远程文件需要很多时间。 I want to improve performance for reading. 我想提高阅读性能。
I have also used python Threading for reading the remote system files. 我还使用python Threading读取远程系统文件。 that is also taking much time. 这也需要很多时间。 now anyone let me know better suggestion for that. 现在有人让我知道更好的建议。
I have used this code for reading remote system file, 我已使用此代码读取远程系统文件,
root_folder="\\\\192.168.1.1\\C$"
try:
use_dict={}
use_dict['remote']=unicode(root_folder)
use_dict['password']=unicode("example")
use_dict['username']=unicode("example")
win32net.NetUseAdd(None, 2, use_dict)
print "Network connection established"
except:
print "Network connection failed"
for root, dirnames, filenames in os.walk(root_folder):
for filename in filenames:
match=os.path.join(root, filename)
datafile = file(match)
for line in datafile:
for li in line:
print li
from this code, 45 min time is taking for reading the remote system files. 从此代码中,花费45分钟时间来读取远程系统文件。 and If i read same files in local way then it is taking only 5 min. 如果我以本地方式读取相同的文件,则只需5分钟。 so I am not able to increase performance to read. 因此我无法提高阅读性能。 please let me know for increase performance for reading... 请让我知道如何提高阅读性能...
thanks... 谢谢...
You can try multiprocessing. 您可以尝试多处理。 In this example a process reads from network and the other one prints data and they are connected by a Queue. 在此示例中,一个进程从网络读取数据,另一个进程打印数据,并且它们通过队列连接。
from multiprocessing import Process, Queue
def readfiles(q):
root_folder="\\\\192.168.1.1\\C$"
try:
use_dict={}
use_dict['remote']=unicode(root_folder)
use_dict['password']=unicode("example")
use_dict['username']=unicode("example")
win32net.NetUseAdd(None, 2, use_dict)
print "Network connection established"
except:
print "Network connection failed"
for root, dirnames, filenames in os.walk(root_folder):
for filename in filenames:
match=os.path.join(root, filename)
datafile = file(match)
for line in datafile:
q.put(line)
q.close()
if __name__ == '__main__':
q = Queue()
p = Process(target=readfiles, args=(q,))
p.start()
while p.is_alive() or not q.empty():
for li in q.get():
print li
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.