Pexpect Mulitple Line Output

Question

Question

How do you get the output of a command with multiple lines of output using pexpect?

Example

This code works, albeit with the output smashed into one line:

child = pexpect.spawn('ping -c 3 1.1.1.1')
child.expect(pexpect.EOF)
print(child.before)

However, this code does not work:

child = pexpect.spawn('hostname')
child.expect(pexpect.EOF)
print(child.before)

child.seldline('ping -c 3 1.1.1.1')
child.expect(pexpect.EOF)
print(child.before)

How would I get this second code to work?

Background

I have commands that I need to run to get connected (replaced here with hostname) and then commands that output mulitiple lines (replaced here with ping) that I cannot seem to get the output from. If I look for any string other than EOF, I get an EOF exception...

The commands I am actually running are here if you need proof:

• Automate `lxc-attach` through ssh with Python

The answer in this other question may be deprecated because this section of code copied exactly just outputs b'' over and over again.

• What does pexpect pattern ".+" do?

Answer 1

By default, the data is of bytes , not str . This is why "the output smashed into one line". See the following example:

$ cat foo.py
import pexpect

child = pexpect.spawn('ping -c 2 127.0.0.1')
child.expect(pexpect.EOF)
print(child.before)


$ python3 foo.py
b'PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.\r\n64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.027 ms\r\n64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.056 ms\r\n\r\n--- 127.0.0.1 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 30ms\r\nrtt min/avg/max/mdev = 0.027/0.041/0.056/0.015 ms\r\n'
$

You can decode() when printing:

$ cat foo.py
import pexpect

child = pexpect.spawn('ping -c 2 127.0.0.1')
child.expect(pexpect.EOF)
print(child.before.decode(encoding='utf8') )  # utf8 is the default for bytes.decode()


$ python3 foo.py
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.024 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.054 ms

--- 127.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 11ms
rtt min/avg/max/mdev = 0.024/0.039/0.054/0.015 ms

$

Or you can specify the encoding when calling spawn() :

$ cat foo.py
import pexpect

child = pexpect.spawn('ping -c 2 127.0.0.1', encoding='utf8')
child.expect(pexpect.EOF)
print(child.before)


$ python3 foo.py
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.053 ms

--- 127.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 20ms
rtt min/avg/max/mdev = 0.036/0.044/0.053/0.010 ms

$

Answer 2

Spawn a PID that will stay open

The real question: Why the second example doesn't work.

The pexpect.spawn object ('child' here) points to a process id (PID). The example I was trying to use was not working because hostname was running and then exiting. In my real usecase, I was using ssh and then several other necessary steps before the long output command (represented by ping here).

Starting your multi-step process with a command that keeps running will solve that issue. Either of these Examples will work:

child = pexpect.spawn('ssh user@host')
child = pexpect.spawn('bin/bash')

I switched to the latter which spins up a new shell that I can interact with. This allowed me to add some error handling to the ssh connection and reuse the code several times within the one shell.

Note that if you exit the ssh connection or bash shell respectively, you will need to spawn a new 'child' to send more commands.

Use a non-blocking read

Extra Detail: Fixing the first/working example's output.

A this code will return the output of the last command without changing it.

def try_read(child):
    """Based on pexpect.pxssh.try_read_prompt"""
    total_timeout = 3
    timeout = 0.5
    inter_char_timeout = 0.1
    begin = time.time()
    expired = 0
    prompt = ''
    while expired < total_timeout:
        try:
            prompt += child.read_nonblocking(size=1, timeout=timeout)
            expired = time.time() - begin # updated total time expired
            timeout = inter_char_timeout
        except TimeoutError:
            print("read ended with TimeoutError")
            break
        except pexpect.TIMEOUT:
            print("read ended with pexpect.Timeout")
            break
        except pexpect.EOF:
            print("read ended with pexpect.EOF")
            break
    return prompt