printing pexpect before output is a single line with \r\n instead of actual CR/LF output

Question

I will apologize in advance that I am new to Python3 and stuck on understanding the actual output as opposed to what I am expecting to receive.

Here is my code:

    import pexpect
    ping = pexpect.spawn('ping -c 5 8.8.8.8')
    result = ping.expect([pexpect.EOF, pexpect.TIMEOUT])
    print(ping.before)

The actual output is a single line starting with "b" and \r\n added instead of the actual lines wrapping.

I expect this:

    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    64 bytes from 8.8.8.8: icmp_seq=1 ttl=128 time=78.1 ms
    64 bytes from 8.8.8.8: icmp_seq=2 ttl=128 time=76.5 ms
    64 bytes from 8.8.8.8: icmp_seq=3 ttl=128 time=76.2 ms
    64 bytes from 8.8.8.8: icmp_seq=4 ttl=128 time=76.8 ms
    64 bytes from 8.8.8.8: icmp_seq=5 ttl=128 time=77.0 ms

But instead my output is one giant line:

    b'PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.\r\n64 bytes from 8.8.8.8: icmp_seq=1 ttl=128 time=78.1 ms\r\n64 bytes from 8.8.8.8: icmp_seq=2 ttl=128 time=76.5 ms\r\n64 bytes from 8.8.8.8: icmp_seq=3 ttl=128 time=76.2 ms\r\n64 bytes from 8.8.8.8: icmp_seq=4 ttl=128 time=76.8 ms\r\n64 bytes from 8.8.8.8: icmp_seq=5 ttl=128 time=77.0 ms\r\n\r\n--- 8.8.8.8 ping statistics ---\r\n5 packets transmitted, 5 received, 0% packet loss, time 4004ms\r\nrtt min/avg/max/mdev = 76.214/76.951/78.149/0.683 ms\r\n'

Does anyone have any idea?

Answer 1

The problem here is that your output is in the form of a byte-string -- also known as the bytes type in Python. This data type is often encountered when interacting with data/content that originated outside of the application. Essentially, Python doesn't know definitively what encoding standard to use while attempting to interpret the data in a buffer. Rather than attempting to guess at it, Python will instead leave it to the developer to decide how to interpret the raw data contained in the byte-string. This allows compatibility with character encoding systems that use more than one byte per character (eg, UTF-16 and UTF-32 ). However, this also means that developers have to convert the data from bytes to str before they can interact with such data in their applications.

In your case, you should likely interpret the data as UTF-8. To do this, you would replace your current print line with:

print(ping.before.decode('utf-8', 'ignore'))

To clarify that command, we are interacting with the bytes object to call the decode(...) method, and we are providing the decode method first with the character encoding codec that we would like the data to be interpreted as, and second with the name of the error-handling approach we want to use. The error-handling approach for a decode has a slew of possible values, but two common approaches are ignore and strict , with strict being the default handler.

The ignore handler essentially tells the decode method that if it comes across a value which it cannot interpret/decode, it should silently discard that value and continue decoding the remaining values.

On the other hand, strict tells the decode method that it should halt all processing and raise an error (specifically the UnicodeError ).

You can read more on encode/decode error-handling options on the Python documentation page for Codec Error Handling .