大文件的Base64？

Question

I'm working on a simple file and folder transfer program in c# using the tcp protocol.

The method I use to transfer a file is to divide it into small pieces. Each of these pieces will be encoded in Base64 and inserted into a more complex JSON message which will then be sent to the other side. So every message will always be in text form thanks to Base64.

I don't have much experience with networks and I decided to make this choice because reading on the internet, sending a stream of bytes without any coding could risk that some part of the message could be interpreted by the router or firewall as a command risking interrupt the connection.

My problem is that my software also works with very large files (> 10GB) and the base64 encoding only increases the data size and makes the transfer very slow because the cpu works so much to encode/decode continuously.

Now I would like to know:

• Is it true that for example, sending a simple file converted only in bytes without first being encoded in any format, risks causing a problem to the router or firewall?
• Is it correct to use Base64 for large file transfers?
• Are there better alternatives?

Answer 1

Is it true that for example, sending a simple file converted only in bytes without first being encoded in any format, risks causing a problem to the router or firewall?

Not at all. It is perfectly normal to do, for example all that HTTPS communication is binary - and it works.

Is it correct to use Base64 for large file transfers?

While one can do this it is a useless overhead if the underlying layer supports binary data.

Are there better alternatives?

Use binary directly. Don't encode or decode anything. Don't wrap it into layers like JSON which are not actually needed for transfer.