Good way to send a large file over a network in C#?

Question

I am trying to build an application that can request files from a service running on another machine in the network. These files can be fairly large (500mb + at times). I was looking into sending it via TCP but I'm worried that it may require that the entire file be stored in memory.

There will probably only be one client. Copying to a shared directory isn't acceptable either. The only communication required is for the client to say "gimme xyz" and the server to send it (and whatever it takes to ensure this happens correctly).

Any suggestions?

Answer 1

Here is an easier way. Using BITS (Background Intelligent Transfer Service). Its already built into WinXP and Vista. Its basically what drives Windows Updates.

http://blogs.msdn.com/powershell/archive/2009/01/11/transferring-large-files-using-bits.aspx

http://blogs.msdn.com/jamesfi/archive/2006/12/23/how-to-use-bits-to-transfer-files.aspx

Here is a nice managed BITS wrapper someone wrote and how to use it.

http://www.codeproject.com/KB/cs/Managed_BITS.aspx

Answer 2

This article may help you. It is about sending large files in .NET. Check the link:

http://codetechnic.blogspot.com/2009/02/sending-large-files-over-tcpip.html

Answer 3

Be careful with BITS. It is a very good protocol but not a critical part of the windows update program. We found that hardly any of our corporate clients allowed the BITS update onto their machines; therefore we couldn't build an app that relied on it.

Answer 4

您可以使用.NET中的套接字来传输文件和数据。

Answer 5

您可能想要考虑WCF流 。

Answer 6

Use FTP via the open source edtFTPnet library. Fast and simple.

Answer 7

Personally I'd go for something that balances speed, reliability and economical code, so I'd base it on a TCP network stream. The client-side of the code would look like this:

internal class Client
{
    private FileStream _fs;     
    private long _expectedLength;

    public void GetFileFromServer(string localFilename)
    {            
        if (File.Exists(localFilename))
            File.Delete(localFilename);

        _fs = new FileStream(localFilename, FileMode.Append);

        var ipEndpointServer = new IPEndPoint(IPAddress.Parse({serverIp}), {serverPort});

        // an object that wraps tcp client
        var client = new TcpClientWrapper(ipEndpointServer, "");
        client.DataReceived += DataReceived;
    }

    private void DataReceived(object sender, DataReceivedEventArgs e)
    {
        var data = e.Data;

        // first packet starts with 4 bytes dedicated to the length of the file
        if (_expectedLength == 0)
        {
            var headerBytes = new byte[4];
            Array.Copy(e.Data, 0, headerBytes, 0, 4);
            _expectedLength = BitConverter.ToInt32(headerBytes, 0);
            data = new byte[e.Data.Length - 4];
            Array.Copy(e.Data, 4, data, 0, data.Length);
        }

        _fs.WriteAsync(e.Data, 0, e.Data.Length);

        if (_fs.Length >= _expectedLength)
        {                                
            // transfer has finished
        }
    }
}

Then have a server class to serve the file. Note how the whole file isn't loaded into memory, instead it is read in chunks from a FileStream .

internal class Server
{
    private TcpServer _tcpServer;
    private NetworkStream _stream;        

    public void StartServer()
    {
        // fire up a simple Tcp server
        _tcpServer = new TcpServer({serverPort}, "test");
        _tcpServer.ClientConnected += ClientConnected;
    }

    private void ClientConnected(object sender, TcpClientConnectedEventArgs e)
    {            
        // an incoming client has been detected ... send the file to that client!
        _stream = e.Client.GetStream();
        SendFileToClient({pathToFile});
    }

    private void SendFileToClient(string pathToFile)
    {
        // open the file as a stream and send in chunks
        using (var fs = new FileStream(pathToFile, FileMode.Open))
        {
            // send header which is file length
            var headerBytes = new byte[4];
            Buffer.BlockCopy(BitConverter.GetBytes(fs.Length + 4), 0, headerBytes, 0, 4);
            _stream.Write(headerBytes, 0, 4);

            // send file in block sizes of your choosing
            var buffer = new byte[100000];
            int bytesRead = 0;
            while ((bytesRead = fs.Read(buffer, 0, buffer.Length)) > 0)
            {
                _stream.Write(buffer, 0, bytesRead);
            }
            _stream.Flush();
        }
    }
}

The TcpClientWrapper is pretty much boiler plate code with the System.Net.Sockets.TcpClient object and the underlying NetworkStream object. I don't really need to post this as well, but just to give some pointers ther construction would contain something like this:

_tcp = new Net.TcpClient();
_tcp.Connect(remoteEp);
_stream = _tcp.GetStream();
_stream.BeginRead(_receivedData, 0, _receivedData.Length, DataReceivedAsync, null);

and the DataReceivedAsync method is boilerplate socket data handling and would raise an event o share the received data back to the consumer (the client in this case):

private void DataReceivedAsync(IAsyncResult ar)
{
    var receivedBytes = _stream.EndRead(ar);

    if (receivedBytes > 0)
    {
        var data = new byte[receivedBytes];
        Array.Copy(_receivedData, 0, data, 0, receivedBytes);
        DataReceived?.Invoke(this, new DataReceivedEventArgs(data));

        _receivedData = new byte[ReceiveBufferSize];
        _stream.BeginRead(_receivedData, 0, _receivedData.Length, DataReceivedAsync, null);
    }       
}   

The event to ship data from the wrapper back to the client:

public EventHandler<DataReceivedEventArgs> DataReceived;
public class DataReceivedEventArgs : EventArgs
{
    public DataReceivedEventArgs(byte[] data) { Data = data; }
    public byte[] Data { get; }
}

Answer 8

使用TransmitFile （这是一个Win32函数;也许它也是.NET库的一种方法）。

Answer 9

If FTP was an option then I'd go with that for the sake of simplicity. Otherwise you're into a world of TCP/IP socket programming.

Answer 10

如果机器上存在文件，为什么不将它们放在文件夹中，使该文件夹成为IIS中的虚拟目录，并使用基于内容的路由和/或URL重写将请求路由到它们。