[英]CPU usage/ speed tradeoff when reading from a named pipe in python
I want to read from a pipe asynchronously in python.我想在 python 中异步读取 pipe。 The pipe is fed continuously from another process (a javascript process). pipe 从另一个进程(javascript 进程)连续进料。 I am using pipes for a matter of speed.我使用管道是为了速度。
I don't want to loop and see if new lines are appended to the file, because it takes 100% of the CPU.我不想循环查看文件是否附加了新行,因为它占用了 100% 的 CPU。 I have tried watchgod
awatch
, but it adds a 300ms delay.我试过watchgod
awatch
,但它增加了 300 毫秒的延迟。 My tradeoff is an asyncio.sleep
of 1ms, it takes 10% of the CPU but adds 1ms delay.我的权衡是 1ms 的asyncio.sleep
,它占用了 10% 的 CPU,但增加了 1ms 的延迟。
import json
import time
import aiofiles
# from watchgod import awatch
async def amain():
async with aiofiles.open('pipe_a', mode='r') as f:
# async for changes in awatch('pipe_a'): # CPU=0% TIME=300ms
while True:
async for line in f:
x = json.loads(line)
print(f"pong: {time.time() * 1000 - x['time']} ms")
await asyncio.sleep(0.001) # # CPU=10% TIME=1.5ms
# await asyncio.sleep(0) # # CPU=100% TIME=0.5ms
import asyncio
loop = asyncio.get_event_loop()
loop.run_until_complete(amain())
What is the right way to get 0.5ms delay, and 0% CPU usage?什么是获得 0.5ms 延迟和 0% CPU 使用率的正确方法?
Watchgod awatch
is doing an async loop itself. Watchgod awatch
本身正在执行异步循环。 So that's why it took 300ms as the delay of the default loop is 300ms.这就是为什么它需要 300 毫秒,因为默认循环的延迟是 300 毫秒。
On Linux, to watch file change, one must use inotify .在 Linux 上,要查看文件更改,必须使用inotify 。 There is an async library for inotify on python: aionotify . python 上有一个用于 inotify 的异步库: aionotify 。 Waiting for file changes using notify does not block the CPU and catches the file event as soon as it appears.使用 notify 等待文件更改不会阻塞 CPU 并会在文件事件出现时立即捕获它。
The solution takes 0% CPU usage and a 0.5ms delay:该解决方案占用 0% 的 CPU 使用率和 0.5 毫秒的延迟:
import asyncio
import aiofiles
import aionotify
import json
import time
watcher = aionotify.Watcher()
watcher.watch(alias='logs', path='pipe_a', flags=aionotify.Flags.MODIFY)
# Prepare the loop
loop = asyncio.get_event_loop()
async def amain():
await watcher.setup(loop)
async with aiofiles.open('pipe_a', mode='r') as f:
try:
while True:
_ = await watcher.get_event()
async for line in f:
x = json.loads(line)
print(f"pong: {time.time() * 1000 - x['time']} ms")
finally:
watcher.close()
loop.run_until_complete(amain())
loop.stop()
loop.close()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.