在 Python 中将文件作为字符串文字读取

Question

I am working with a CType function that requires a byte string that is being read in from a file. If I put the string into the script, it will work, as long as I denote the string as a string literal (ie with 'r') and then convert it to a byte string. But if I just read it in as a byte string it does not work. Is there a way to read in a file as a string literal?

if __name__ == '__main__':
    a = r"\x00hello"
    with open('some_file', 'rb') as f: # some file contains only "\x00hello"
        b = f.read()
    c = b"\x00hello"

    x = CtypeObj.Function(a.encode('utf-8', errors='ignore')) # success!
    y = CtypeObj.Function(b)                                  # failure!
    z = CtypeObj.Function(c)                                  # failure!

Answer 1

The line that you point at as a success likely isn't doing what you think it is doing either:

a = r"\x00hello"

That defines a string of 9 characters, \ , x , etc. Calling a.encode('utf-8', errors='ignore') takes that string and encodes the characters in the string using utf-8 and returns a bytes value of that encoding. (which CtypeObj.Function() accepts)

I would assume that you don't really want that \00 part passed to the function?

Reading from the 'rb' mode file gets you a bytes value as well, but the encoding of the file will be the encoding of that bytes value. If you need it to be utf-8 encoding (and the file might not be), then you should instead open the file as 'r' , read the value as a string, and encode with b.encode('utf-8') .

And finally this line:

c = b"\x00hello"

This just creates a length 6 bytes value, with the first byte being the 0 byte, and the rest the values for the 5 letters. However, that's not automatically a utf-8 encoding, and certainly not the same as you had before. Again, it would seem you don't want that \x00 at the start, since it's very unusual for a string to start with a null character like that.

As indicated in the comments, r"\x00hello" and 'hello' are all string literals, but that's only meaningful in the context of code. In terms of data, you only have strings of characters ( str ) and bytes values (sometimes called a string of bytes). A "literal" is a way to write either in code directly:

s = 'hello'   # a string literal
b = b'hello'  # a bytes literal for the same text (under most encodings)

s == b.decode()  # True
b == s.encode()  # True

If you read a file using mode 'r' , you get strings. If you use a file using mode 'rb' , you get bytes.

Answer 2

Try this:

if __name__ == '__main__':
    with open('./file.txt', 'rb') as f:
        # read `\x00hello` from file, remove trailing newline
        line = f.read().rstrip()
        # decode the unicode escapes, then re-encode
        line = line.decode('unicode-escape').encode('utf-8')

    print(line)
    print(b'\x00hello')

    print(line == b'\x00hello')

Adapted from advice from this answer .

---

[~] $ cat file.txt
\x00hello
[~] $ python script.py
b'\x00hello'
b'\x00hello'
True