std::process，来自缓冲区的 stdin 和 stdout

Question

I have a command cmd and three Vec<u8> : buff1 , buff2 , and buff3 . I want to execute cmd , using buff1 as stdin , and capturing stdout into buff2 and stderr into buff3 . And I'd like to do all this without explicitly writing any temporary files.

std::process seems to allow all of those things, just not all at the same time.

If I use Command::new(cmd).output() it will return the buffers for stdout and stderr , but there's no way to give it stdin .

If I use Command::new(cmd).stdin(Stdio::piped()).spawn() then I can child.stdin.as_mut().unwrap().write_all(buff1) but I can't capture stdout and stderr .

As far as I can tell, there's no way to call Command::new(cmd).stdout(XXX) to explicitly tell it to capture stdout in a buffer, the way it does by default with .output() .

It seems like something like this should be possible:

Command::new(cmd)
.stdin(buff1)
.stdout(buff2)
.stderr(buff3)
.output()

Since Rust makes a Vec<u8> look like a File , but Vec doesn't implement Into<Stdio>

Am I missing something? Is there a way to do this, or do I need to read and write with actual files?

Answer 1

If you're ok with using an external crate, the subprocess crate supports this use case:

let (buff2, buff3) = subprocess::Exec::cmd(cmd)
    .stdin(buff1)
    .communicate()?
    .read()?;

Doing this with std::process::Command is trickier than it seems because the OS doesn't make it easy to connect a region of memory to a subprocess's stdin. It's easy to connect a file or anything file-like, but to feed a chunk of memory to a subprocess, you basically have to write() in a loop. While Vec<u8> does implement std::io::Read , you can't use it to construct an actual File (or anything else that contains a file descriptor/handle). So you need to feed the data into the subprocess manually, and at the same time you need to drain its stdout and stderr, being careful to avoid a deadlock. The documentation describes a possible way of doing it using just the standard library.

Answer 2

If you want to read and write with buffers, you need to use the piped forms. The reason is that, at least on Unix, input and output to a process are done through file descriptors. Since a buffer cannot intrinsically be turned into a file descriptor, it's required to use a pipe and both read and write incrementally. The fact that Rust provides an abstraction for buffers doesn't allow you to avoid the fact that the operating system doesn't, and Rust doesn't abstract this for you.

However, since you'll be using pipes for both reading and writing, you'll need to use something like select so you don't deadlock. Otherwise, you could end up trying to write when your subprocess was not accepting new input because it needed data to be read from its standard output. Using select or poll (or similar) permits you to determine when each of those file descriptors are ready to be read or written to. In Rust, these functions are in the libc crate; I don't believe that Rust provides them natively. Windows will have some similar functionality, but I have no clue what it is.

It should be noted that unless you are certain that the subprocess's output can fit into memory, it may be better to process it in a more incremental way. Since you're going to be using select , that shouldn't be too difficult.