[英]Why is there a built-in delay when reading with popen()?
I'm executing a long-running (and often blocked) command via popen() : "ls -R /" 我正在通过popen()执行长时间运行(并且经常被阻止)的命令:“ ls -R /”
Problem: popen() reads into a buffer that you supply, and it seemingly attempts to populate the ENTIRE buffer before returning. 问题:popen()读入您提供的缓冲区,并且看似试图在返回之前填充整个缓冲区。 This causes it to block quite often (if your buffer is large).
这导致它经常阻塞(如果您的缓冲区很大)。
The solution would seem to be to make the underling fd non-blocking. 解决方案似乎是使基础fd不阻塞。 When I do this, popen() still blocks, usually about 1 second each time.
当我这样做时,popen()仍然会阻塞,通常每次大约1秒。 Why is this happening?
为什么会这样呢?
Here is my code. 这是我的代码。 Make sure to compile with -std=c++11:
确保使用-std = c ++ 11进行编译:
#include <cstdio>
#include <iostream>
#include <sys/time.h>
#include <unistd.h>
#include <fcntl.h>
static constexpr size_t SIZE = 65536;
struct Time
{
friend std::ostream &operator<<(std::ostream &os, Time const &t)
{
(void)t;
timeval tv;
gettimeofday(&tv, nullptr);
os << tv.tv_sec << "." << std::fixed << tv.tv_usec << " ";
return os;
}
};
int main()
{
FILE *file;
file = popen("ls -R /", "r");
if(!file)
{
std::cerr << "Could not open app: " << errno;
return -1;
}
// Make it non-blocking
int fd = fileno(file);
fcntl(fd, F_SETFL, O_NONBLOCK);
char buffer[SIZE];
Time t;
while(true)
{
int rc = fread(buffer, 1, SIZE, file);
if(rc <= 0)
{
if(EAGAIN == errno)
{
usleep(10);
continue;
}
std::cerr << t << "Error reading: " << errno << std::endl;
break;
}
std::cerr << t << "Read " << rc << std::endl;
}
pclose(file);
return 0;
}
Output (notice that they are about 1 second apart, even though the fd is nonblocking and I only have a 10mS pause in the loop): 输出(请注意,即使fd处于非阻塞状态并且我在循环中只有10mS的暂停,它们之间的间隔约为1秒):
1429625100.983786 Read 4096
1429625101.745369 Read 4096
1429625102.426967 Read 4096
1429625103.185273 Read 4096
1429625103.834241 Read 4096
1429625104.512131 Read 4096
1429625105.188010 Read 4096
1429625105.942257 Read 4096
1429625106.642877 Read 4096
First, you should use read
rather than fread
. 首先,您应该使用
read
而不是fread
。 The stdio functions have their own layer of buffering beyond the OS's, so they can block even on non-blocking file descriptors. stdio函数在OS之外具有自己的缓冲层,因此它们甚至可以在非阻塞文件描述符上阻塞。 Use
read
to avoid this. 使用
read
可以避免这种情况。
Second, you need to stop ls
from buffering its output. 其次,您需要停止
ls
缓冲其输出。 The default behavior for programs that link to glibc is to use line buffering when stdout is connecting to a TTY, and full buffering when it is connected to a pipe or redirected to a file. 链接到glibc的程序的默认行为是,当stdout连接到TTY时使用行缓冲,而当它连接到管道或重定向到文件时使用全缓冲。 Full buffering means the output is only flushed when the 4KB buffer fills up, rather than flushing every time a newline is output.
完全缓冲意味着仅在4KB缓冲区填满时才刷新输出,而不是在每次输出换行符时刷新。
You can use stdbuf
to override this behavior. 您可以使用
stdbuf
覆盖此行为。 Note that it will only work for programs that use C streams and link to glibc. 请注意,它仅适用于使用C流并链接到glibc的程序。 That is most programs, but not all.
那是大多数程序,但不是全部。
popen("stdbuf -oL ls -R /", "r");
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.