以所选编码将文本文件读入字节数组的最佳方法？

Question

Now i use something like that:

Encoding.UTF8.GetBytes(File.ReadAllText(filename))

Any suggestions how to do that better?

And what encoding uses File.ReadAllBytes(filename) method?

PS I need utf-8 byte arrays to store text files in db

Answer 1

Best way to read file into byte array in selected encoding?

Character Encoding is about storing text in binary form, as sequences of specific bytes for each character. Another way of thinking about it is that the Encoding system is what gives meaning to some bytes. Without the context that some bytes represents text, the bytes are just bytes.

Files are just bytes too; And they can be interpreted however you want your application to interpret them.

When you decode bytes you are giving meaning to those bytes according the encoding system used. For text encodings, you start with bytes and end up with characters.
You can't " decode " bytes from a file into a byte array. That doesn't give meaning to the bytes or produce any characters.

You can decode bytes into strings using a specific encoding though:

string allLinesFromFileAsAuto = File.ReadAllText(filename);
string allLinesFromFileAsUTF8 = File.ReadAllText(filename, Encoding.UTF8);
string allLinesFromFileAsASCII = File.ReadAllText(filename, Encoding.ASCII);

All three of these methods convert bytes from the same file into strings, but the resulting strings will be different depending on the encoding you use.

And what encoding uses File.ReadAllBytes(filename) method?

File.ReadAllBytes(filename) does not use any encoding. Files are just bytes. This method pulls all of a file's bytes into a byte array. You still have to decode those bytes into strings after getting that byte array. But this only works well for plaintext files.

I need utf-8 byte arrays to store files in db

Is this because your database uses UTF-8 encoding?
The encoding of a database defines how text is stored ( as binary ). Binary data can be stored as-is, byte-for-byte, as "blobs" in most databases, regardless of the encoding.

Answer 2

ReadAllText will try to infer the encoding of the file and convert it to .NET strings. Your first example will then convert those to UTF-8 bytes no matter what the source encoding was.

Depending on the size of the files, this could be costly to load it all to memory twice. You can do things to read chunks of the source file and convert it that way.

ReadAllBytes reads the raw file as a series of bytes, there's no encoding/decoding for that.

If you are storing non-text files in the database, you should not encode the file as UTF-8.