HTTPS代理實現，如何檢測一個完成的請求

Question

我正在嘗試用 C# 編寫一個簡單的異步 https 代理服務器。

我想知道當請求完成時我應該如何檢測/處理，以及如何退出我的 bActive 循環，假設這樣的循環是合適的。

非常感謝我的方法是否正確以及我可以做些什么來改進邏輯的一些指示。

我似乎遇到了問題是，它需要一個端點回應與網絡延遲沿着時間意味着我DataAvailable doenst總是有數據，但可能仍然存在一些發送。 需要睡眠和另一次嘗試，這反過來會導致請求的完成時間過長。

1. 監聽 TCP 連接

2. 提取 CONNECT 標頭並打開到請求服務器的連接

3. 將 requestStream 復制到 proxyStream

4. 將 proxyStream 復制到 requestStream

5. 休眠等待數據並重復 3 - 4，直到兩個流上都沒有數據可用。 然后跳出循環並關閉連接。

public async Task Start()
{
    listener.Start();

    while (listen)
    {
        if (listener.Pending())
        {
            HandleClient(await listener.AcceptTcpClientAsync());
        }
        else
        {
            await Task.Delay(100); //<--- timeout
        }
    }
}

private static async Task HandleClient(TcpClient clt)
{

    var bytes = new byte[clt.ReceiveBufferSize];
    var hostHeaderAvailable = 0;
    NetworkStream requestStream = null;
    int count;
    const string connectText = "connect";
    const string hostText = "Host: ";
    bool bActive = true;
    List<Task> tasks = new List<Task>();


    try
    {
        using (NetworkStream proxyStream = clt.GetStream())
        using (TcpClient requestClient = new TcpClient())
        {
            proxyStream.ReadTimeout = 100;
            proxyStream.WriteTimeout = 100;


            while (bActive)
            {
            
                if (proxyStream.DataAvailable && hostHeaderAvailable == 0)
                {
                    count = await proxyStream.ReadAsync(bytes, 0, bytes.Length);

                    var text = Encoding.UTF8.GetString(bytes);
                    Console.WriteLine(text);

                    if (text.ToLower().StartsWith(connectText))
                    {
                        // extract the url and port
                        var host = text.Remove(0, connectText.Length + 1);
                        var hostIndex = host.IndexOf(" ", StringComparison.Ordinal);
                        var hostEntry = host.Remove(hostIndex).Split(new[] { ":" }, StringSplitOptions.None);
                        // connect to the url and prot supplied
                        await requestClient.ConnectAsync(hostEntry[0], Convert.ToInt32(hostEntry[1]));
                        requestStream = requestClient.GetStream();

                        requestStream.ReadTimeout = 100;
                        requestStream.WriteTimeout = 100;

                        // send 200 response to proxyStream 
                        const string sslResponse = "HTTP/1.0 200 Connection established\r\n\r\n";
                        var sslResponseBytes = Encoding.UTF8.GetBytes(sslResponse);
                        await proxyStream.WriteAsync(sslResponseBytes, 0, sslResponseBytes.Length);

                        // delay here seems to prevent the following proxyStream.read from failing as data is not yet avaiable
                        // without it the loop runs and has to timeout before running again
                        await Task.Delay(1);
                    }
                }
                hostHeaderAvailable++;


                if (requestStream == null || !requestClient.Connected || !clt.Connected)
                {
                    bActive = false;
                    break;
                }

                Console.WriteLine(proxyStream.DataAvailable || requestStream.DataAvailable);

                if (proxyStream.DataAvailable || requestStream.DataAvailable)
                { 
                    Task task = proxyStream.CopyToAsync(requestStream);
                    Task task2 = requestStream.CopyToAsync(proxyStream);

                    tasks.Add(task);
                    tasks.Add(task2);

                    await Task.WhenAll(tasks).ConfigureAwait(false);
                    bActive = false;
                    break;
                }

                await Task.Delay(10);
            }
        }
    }
    catch (Exception e)
    {
        Console.WriteLine(e.ToString());
    }

    clt.Close();
}

較早的嘗試使用ReadAsync / WriteAsync太長時間無法響應並且仍然存在超時問題。

1. 監聽 TCP 連接

2. 提取 CONNECT 標頭並打開到請求服務器的連接

3. 從 requestStream 讀取數據並復制到 proxyStream

4. 等待檢查數據是否在任一流上可用

5. 如果數據可用從 proxyStream 讀取並寫入 requestStream

6. 如果數據可用從 requestStream 讀取並寫入 proxyStream

7. 休眠等待數據並重復 5 - 6 直到在 eitboth 流上沒有數據可用。 然后跳出循環並關閉連接。

private static TcpListener listener = new TcpListener(IPAddress.Parse("192.168.0.25"), 13000);
private static bool listen = true;


public async Task Start()
{
    listener.Start();

    while (listen)
    {
        if (listener.Pending())
        {
            await HandleClient(await listener.AcceptTcpClientAsync());
        }
        else
        {
            await Task.Delay(100); 
        }
    }
}


private static async Task HandleClient(TcpClient clt)
{

    var bytes = new byte[clt.ReceiveBufferSize];
    var hostHeaderAvailable = 0;
    NetworkStream requestStream = null;
    int count;
    const string connectText = "connect";
    const string hostText = "Host: ";

    bool bActive = true;

    try
    {
        using (NetworkStream proxyStream = clt.GetStream())
        using (TcpClient requestClient = new TcpClient())
        {
            while (bActive)
            {
                while (proxyStream.DataAvailable)
                {
                    // handle connect
                    if (hostHeaderAvailable == 0)
                    {
                        count = await proxyStream.ReadAsync(bytes, 0, bytes.Length);

                        var text = Encoding.UTF8.GetString(bytes);
                        Console.WriteLine(text);

                        if (text.ToLower().StartsWith(connectText))
                        {
                            // extract the url and port
                            var host = text.Remove(0, connectText.Length + 1);
                            var hostIndex = host.IndexOf(" ", StringComparison.Ordinal);
                            var hostEntry = host.Remove(hostIndex).Split(new[] { ":" }, StringSplitOptions.None);
                            // connect to the url and prot supplied
                            await requestClient.ConnectAsync(hostEntry[0], Convert.ToInt32(hostEntry[1]));
                            requestStream = requestClient.GetStream();
                            // send 200 response to proxyStream 
                            const string sslResponse = "HTTP/1.0 200 Connection established\r\n\r\n";
                            var sslResponseBytes = Encoding.UTF8.GetBytes(sslResponse);
                            await proxyStream.WriteAsync(sslResponseBytes, 0, sslResponseBytes.Length);

                            // delay here seems to prevent the following proxyStream.read from failing as data is not yet avaiable
                            // without it the loop runs and has to timeout before running again
                            await Task.Delay(20);
                        }
                    }
                    hostHeaderAvailable++;

                    if (requestClient.Connected && hostHeaderAvailable > 1)
                    {
                        count = await proxyStream.ReadAsync(bytes, 0, bytes.Length);
                        await requestStream.WriteAsync(bytes, 0, count);
                    }
                }

                while (requestStream.DataAvailable)
                {
                    count = await requestStream.ReadAsync(bytes, 0, bytes.Length);
                    await proxyStream.WriteAsync(bytes, 0, count);
                }


                // attempt to detect a timeout / end of data avaiable
                var timeout = 0;
                while (!proxyStream.DataAvailable && !requestStream.DataAvailable)
                {
                    if (timeout > 5)
                    {
                        bActive = false;
                        break;
                    }

                    await Task.Delay(10);
                    timeout++;
                }
            }

        }

    }
    catch (Exception e)
    {
        Console.WriteLine(e.ToString());
    }
}

更新

根據 AgentFire 的回答，我現在得到了以下工作代碼：

public static async Task HandleDisconnect(TcpClient tcp, TcpClient tcp2, CancellationToken cancellationToken)
{
    while (true)
    {
        if (tcp.Client.Poll(0, SelectMode.SelectRead))
        {
            byte[] buff = new byte[1];

            if (tcp.Client.Receive(buff, SocketFlags.Peek) == 0)
            {
                // Client disconnected
                Console.WriteLine("The requesting client has dropped its connection.");
                cancellationToken = new CancellationToken(true);
                break;
            }
        }
        if (tcp2.Client.Poll(0, SelectMode.SelectRead))
        {
            byte[] buff = new byte[1];

            if (tcp2.Client.Receive(buff, SocketFlags.Peek) == 0)
            {
                // Server disconnected
                Console.WriteLine("The destination client has dropped its connection.");
                cancellationToken = new CancellationToken(true);
                break;
            }
        }

        await Task.Delay(1);
    }
}


private static async Task HandleClient(TcpClient clt)
{
    List<Task> tasks            = new List<Task>();
    var bytes                   = new byte[clt.ReceiveBufferSize];
    var hostHeaderAvailable     = 0;
    NetworkStream requestStream = null;
    const string connectText    = "connect";

    try
    {
        using (NetworkStream proxyStream = clt.GetStream())
        using (TcpClient requestClient = new TcpClient())
        {
            proxyStream.ReadTimeout = 100;
            proxyStream.WriteTimeout = 100;

            if (proxyStream.DataAvailable && hostHeaderAvailable == 0)
            {
                await proxyStream.ReadAsync(bytes, 0, bytes.Length);

                var text = Encoding.UTF8.GetString(bytes);
                Console.WriteLine(text);

                if (text.ToLower().StartsWith(connectText))
                {
                    // extract the url and port
                    var host = text.Remove(0, connectText.Length + 1);
                    var hostIndex = host.IndexOf(" ", StringComparison.Ordinal);
                    var hostEntry = host.Remove(hostIndex).Split(new[] { ":" }, StringSplitOptions.None);
                    // connect to the url and prot supplied
                    await requestClient.ConnectAsync(hostEntry[0], Convert.ToInt32(hostEntry[1]));
                    requestStream = requestClient.GetStream();

                    requestStream.ReadTimeout = 100;
                    requestStream.WriteTimeout = 100;

                    // send 200 response to proxyStream 
                    const string sslResponse = "HTTP/1.0 200 Connection established\r\n\r\n";
                    var sslResponseBytes = Encoding.UTF8.GetBytes(sslResponse);
                    await proxyStream.WriteAsync(sslResponseBytes, 0, sslResponseBytes.Length);
                }
            }
            hostHeaderAvailable++;

            CancellationToken cancellationToken = new CancellationToken(false);

            Task task               = proxyStream.CopyToAsync(requestStream, cancellationToken);
            Task task2              = requestStream.CopyToAsync(proxyStream, cancellationToken);
            Task handleConnection   = HandleDisconnect(clt, requestClient, cancellationToken);

            tasks.Add(task);
            tasks.Add(task2);
            tasks.Add(handleConnection);
                
            await Task.WhenAll(tasks).ConfigureAwait(false);

            // close conenctions
            clt.Close();
            clt.Dispose();
            requestClient.Close();
            requestClient.Dispose();
        }
    }
    catch (Exception e)
    {
        Console.WriteLine(e.ToString());
    }
}

更新

嘗試使用 CancellationTokenSource

CancellationTokenSource source = new CancellationTokenSource();
CancellationToken cancellationToken = source.Token;
TaskFactory factory = new TaskFactory(cancellationToken);

tasks.Add(factory.StartNew(() => {proxyStream.CopyToAsync(requestStream);}, cancellationToken));
tasks.Add(factory.StartNew(() => {requestStream.CopyToAsync(proxyStream);}, cancellationToken));
tasks.Add(factory.StartNew(async () => {
    //wait for this to retur, then cancel the token
    await HandleDisconnect(clt, requestClient);
    source.Cancel();
}, cancellationToken));

try
{
    await factory.ContinueWhenAll(tasks.ToArray(),
                                                 (results) =>
                                                 {
                                                     Console.WriteLine("Tasks complete");
                                                 }, cancellationToken);
}
catch (AggregateException ae)
{
    foreach (Exception e in ae.InnerExceptions)
    {
        if (e is TaskCanceledException)
            Console.WriteLine("Unable to compute mean: {0}",
                              ((TaskCanceledException)e).Message);
        else
            Console.WriteLine("Exception: " + e.GetType().Name);
    }
}
finally
{
    source.Dispose();
}

更新

public static class extensionTcpClient{

    public static bool CheckIfDisconnected(this TcpClient tcp)
    {
        if (tcp.Client.Poll(0, SelectMode.SelectRead))
        {
            byte[] buff = new byte[1];

            if (tcp.Client.Receive(buff, SocketFlags.Peek) == 0)
            {
                // Client disconnected
                return false;
            }
        }

        return true;
    }
}


class ProxyMaintainer
{

    private static TcpListener listener = new TcpListener(IPAddress.Parse("192.168.0.25"), 13000);

    public ProxyMaintainer()
    {
    }

    public async Task Start()
    {
        Console.WriteLine("###############################");
        Console.WriteLine("Listening on 192.168.0.25:13000");
        Console.WriteLine("###############################\n");

        listener.Start();

        while (listen)
        {
            if (listener.Pending())
            {
                HandleClient(await listener.AcceptTcpClientAsync());
            }
            else
            {
                await Task.Delay(100); //<--- timeout
            }
        }
    }


    private static async Task Transport(NetworkStream from, NetworkStream to, Func<bool> isAlivePoller, CancellationToken token)
    {
        byte[] buffer = new byte[4096];

        while (isAlivePoller())
        {
            while (from.DataAvailable)
            {
                int read = await from.ReadAsync(buffer, 0, buffer.Length, token).ConfigureAwait(false);
                await to.WriteAsync(buffer, 0, read, token);
            }

            // Relieve the CPU a bit.
            await Task.Delay(10, token).ConfigureAwait(false);
        }
    }


    private static async Task HandleClient(TcpClient clientFrom)
    {
        var hostHeaderAvailable = 0;
        int count;
        var bytes = new byte[clientFrom.ReceiveBufferSize];
        const string connectText = "connect";
        NetworkStream toStream = null;

        using (var fromStream = clientFrom.GetStream())
        using(TcpClient clientTo = new TcpClient())
        using (var manualStopper = new CancellationTokenSource())
        {
            count = await fromStream.ReadAsync(bytes, 0, bytes.Length);

            var text = Encoding.UTF8.GetString(bytes);
            Console.WriteLine(text);

            if (text.ToLower().StartsWith(connectText))
            {
                // extract the url and port
                var host = text.Remove(0, connectText.Length + 1);
                var hostIndex = host.IndexOf(" ", StringComparison.Ordinal);
                var hostEntry = host.Remove(hostIndex).Split(new[] { ":" }, StringSplitOptions.None);
                // connect to the url and prot supplied
                await clientTo.ConnectAsync(hostEntry[0], Convert.ToInt32(hostEntry[1]));
                toStream = clientTo.GetStream();

                // send 200 response to proxyStream 
                const string sslResponse = "HTTP/1.0 200 Connection established\r\n\r\n";
                var sslResponseBytes = Encoding.UTF8.GetBytes(sslResponse);
                await fromStream.WriteAsync(sslResponseBytes, 0, sslResponseBytes.Length);
            }
            
            bool Poller() => clientFrom.CheckIfDisconnected() && clientTo.CheckIfDisconnected();

            Task one = Transport(fromStream, toStream, Poller, manualStopper.Token);
            Task two = Transport(toStream, fromStream, Poller, manualStopper.Token);

            await Task.WhenAll(one, two).ConfigureAwait(false);
            //await one; await two; // To get exceptions if you want them and there are any.
            // Alternatively, you can use Task.WhenAll to get exceptions aggregated for you.
        }

        Console.WriteLine("Closing connection");
    }



}

Answer 1

好吧，告訴你什么。 對於 HTTP，數據可用性僅在於一個參數（如果我們省略 WebSocket 之類的東西），該參數稱為Connection並作為 Header 作為兩種可能狀態之一傳遞： Close或Keep-Alive 。

如果客戶端選擇Close ，服務器必須在請求被提供后立即關閉連接，而Keep-Alive告訴服務器，如果它不想，它可以為另一個請求保持連接打開。

讓我們考慮這兩種情況。

如果客戶端選擇 Keep-Alive，則連接將無限期地持久並按預期工作。 但：

如果任何一方斷開連接，有一種簡單的方法可以檢測到。 這段代碼是在 StackOverflow 上找到的，並被告知它仍然可以完美運行：

public static bool CheckIfDisconnected(this TcpClient tcp)
{
    if (tcp.Client.Poll(0, SelectMode.SelectRead))
    {
        byte[] buff = new byte[1];

        if (tcp.Client.Receive(buff, SocketFlags.Peek) == 0)
        {
            // Client disconnected
            return true;
        }
    }

    return false;
}

所以我相信你，作為一個代理服務器，根本沒有義務管理連接狀態，可以把它留給實際的通信方。 您所要做的就是檢測您的連接 - 代理或請求 - 何時被丟棄，丟棄另一個並收工。

PS 現在，您還詢問了異步性。

我必須補充一點，TCP 連接被認為是全雙工的。 這意味着您可以自由創建兩個異步運行的任務，讀取和寫入它們自己的接收器。 我的想法是，這將是最佳的行動方案。

Answer 2

回答你的其他問題

您仍在使用Stream.CopyToAsync ，正如我告訴過您的那樣，只要任何通信方決定在發送另一塊數據之前稍等Stream.CopyToAsync ， Stream.CopyToAsync不會成功。

您的解決方案也有些過於復雜。

我會這樣說：

async Task Transport(NetworkStream from, NetworkStream to, Func<bool> isAlivePoller, CancellationToken token)
{
    byte[] buffer = new byte[4096];

    while (isAlivePoller())
    {
        while (from.DataAvailable)
        {
            int read = await from.ReadAsync(buffer, 0, buffer.Length, token).ConfigureAwait(false);
            await to.WriteAsync(buffer, 0, read, token).ConfigureAwait(false);
        }

        // Relieve the CPU a bit.
        await Task.Delay(100, token).ConfigureAwait(false);
    }
}

然后在你的主代碼中：

using TcpClient clientFrom = ...;
using TcpClient clientTo = ...;
using var fromStream = clientFrom.GetStream();
using var toStream = clientTo.GetStream();
using var manualStopper = new CancellationTokenSource();

bool Poller() => clientFrom.CheckIfDisconnected() && clientTo.CheckIfDisconnected();

Task one = Transport(fromStream, toStream, Poller, stopper.Token);
Task two = Transport(toStream, fromStream, Poller, stopper.Token);

await Task.WhenAny(one, two).ConfigureAwait(false);
//await one; await two; // To get exceptions if you want them and there are any.
// Alternatively, you can use Task.WhenAll to get exceptions aggregated for you.

到這里你就大功告成了。