HTTPS代理实现，如何检测一个完成的请求

Question

我正在尝试用 C# 编写一个简单的异步 https 代理服务器。

我想知道当请求完成时我应该如何检测/处理，以及如何退出我的 bActive 循环，假设这样的循环是合适的。

非常感谢我的方法是否正确以及我可以做些什么来改进逻辑的一些指示。

我似乎遇到了问题是，它需要一个端点回应与网络延迟沿着时间意味着我DataAvailable doenst总是有数据，但可能仍然存在一些发送。 需要睡眠和另一次尝试，这反过来会导致请求的完成时间过长。

1. 监听 TCP 连接

2. 提取 CONNECT 标头并打开到请求服务器的连接

3. 将 requestStream 复制到 proxyStream

4. 将 proxyStream 复制到 requestStream

5. 休眠等待数据并重复 3 - 4，直到两个流上都没有数据可用。 然后跳出循环并关闭连接。

public async Task Start()
{
    listener.Start();

    while (listen)
    {
        if (listener.Pending())
        {
            HandleClient(await listener.AcceptTcpClientAsync());
        }
        else
        {
            await Task.Delay(100); //<--- timeout
        }
    }
}

private static async Task HandleClient(TcpClient clt)
{

    var bytes = new byte[clt.ReceiveBufferSize];
    var hostHeaderAvailable = 0;
    NetworkStream requestStream = null;
    int count;
    const string connectText = "connect";
    const string hostText = "Host: ";
    bool bActive = true;
    List<Task> tasks = new List<Task>();


    try
    {
        using (NetworkStream proxyStream = clt.GetStream())
        using (TcpClient requestClient = new TcpClient())
        {
            proxyStream.ReadTimeout = 100;
            proxyStream.WriteTimeout = 100;


            while (bActive)
            {
            
                if (proxyStream.DataAvailable && hostHeaderAvailable == 0)
                {
                    count = await proxyStream.ReadAsync(bytes, 0, bytes.Length);

                    var text = Encoding.UTF8.GetString(bytes);
                    Console.WriteLine(text);

                    if (text.ToLower().StartsWith(connectText))
                    {
                        // extract the url and port
                        var host = text.Remove(0, connectText.Length + 1);
                        var hostIndex = host.IndexOf(" ", StringComparison.Ordinal);
                        var hostEntry = host.Remove(hostIndex).Split(new[] { ":" }, StringSplitOptions.None);
                        // connect to the url and prot supplied
                        await requestClient.ConnectAsync(hostEntry[0], Convert.ToInt32(hostEntry[1]));
                        requestStream = requestClient.GetStream();

                        requestStream.ReadTimeout = 100;
                        requestStream.WriteTimeout = 100;

                        // send 200 response to proxyStream 
                        const string sslResponse = "HTTP/1.0 200 Connection established\r\n\r\n";
                        var sslResponseBytes = Encoding.UTF8.GetBytes(sslResponse);
                        await proxyStream.WriteAsync(sslResponseBytes, 0, sslResponseBytes.Length);

                        // delay here seems to prevent the following proxyStream.read from failing as data is not yet avaiable
                        // without it the loop runs and has to timeout before running again
                        await Task.Delay(1);
                    }
                }
                hostHeaderAvailable++;


                if (requestStream == null || !requestClient.Connected || !clt.Connected)
                {
                    bActive = false;
                    break;
                }

                Console.WriteLine(proxyStream.DataAvailable || requestStream.DataAvailable);

                if (proxyStream.DataAvailable || requestStream.DataAvailable)
                { 
                    Task task = proxyStream.CopyToAsync(requestStream);
                    Task task2 = requestStream.CopyToAsync(proxyStream);

                    tasks.Add(task);
                    tasks.Add(task2);

                    await Task.WhenAll(tasks).ConfigureAwait(false);
                    bActive = false;
                    break;
                }

                await Task.Delay(10);
            }
        }
    }
    catch (Exception e)
    {
        Console.WriteLine(e.ToString());
    }

    clt.Close();
}

较早的尝试使用ReadAsync / WriteAsync太长时间无法响应并且仍然存在超时问题。

1. 监听 TCP 连接

2. 提取 CONNECT 标头并打开到请求服务器的连接

3. 从 requestStream 读取数据并复制到 proxyStream

4. 等待检查数据是否在任一流上可用

5. 如果数据可用从 proxyStream 读取并写入 requestStream

6. 如果数据可用从 requestStream 读取并写入 proxyStream

7. 休眠等待数据并重复 5 - 6 直到在 eitboth 流上没有数据可用。 然后跳出循环并关闭连接。

private static TcpListener listener = new TcpListener(IPAddress.Parse("192.168.0.25"), 13000);
private static bool listen = true;


public async Task Start()
{
    listener.Start();

    while (listen)
    {
        if (listener.Pending())
        {
            await HandleClient(await listener.AcceptTcpClientAsync());
        }
        else
        {
            await Task.Delay(100); 
        }
    }
}


private static async Task HandleClient(TcpClient clt)
{

    var bytes = new byte[clt.ReceiveBufferSize];
    var hostHeaderAvailable = 0;
    NetworkStream requestStream = null;
    int count;
    const string connectText = "connect";
    const string hostText = "Host: ";

    bool bActive = true;

    try
    {
        using (NetworkStream proxyStream = clt.GetStream())
        using (TcpClient requestClient = new TcpClient())
        {
            while (bActive)
            {
                while (proxyStream.DataAvailable)
                {
                    // handle connect
                    if (hostHeaderAvailable == 0)
                    {
                        count = await proxyStream.ReadAsync(bytes, 0, bytes.Length);

                        var text = Encoding.UTF8.GetString(bytes);
                        Console.WriteLine(text);

                        if (text.ToLower().StartsWith(connectText))
                        {
                            // extract the url and port
                            var host = text.Remove(0, connectText.Length + 1);
                            var hostIndex = host.IndexOf(" ", StringComparison.Ordinal);
                            var hostEntry = host.Remove(hostIndex).Split(new[] { ":" }, StringSplitOptions.None);
                            // connect to the url and prot supplied
                            await requestClient.ConnectAsync(hostEntry[0], Convert.ToInt32(hostEntry[1]));
                            requestStream = requestClient.GetStream();
                            // send 200 response to proxyStream 
                            const string sslResponse = "HTTP/1.0 200 Connection established\r\n\r\n";
                            var sslResponseBytes = Encoding.UTF8.GetBytes(sslResponse);
                            await proxyStream.WriteAsync(sslResponseBytes, 0, sslResponseBytes.Length);

                            // delay here seems to prevent the following proxyStream.read from failing as data is not yet avaiable
                            // without it the loop runs and has to timeout before running again
                            await Task.Delay(20);
                        }
                    }
                    hostHeaderAvailable++;

                    if (requestClient.Connected && hostHeaderAvailable > 1)
                    {
                        count = await proxyStream.ReadAsync(bytes, 0, bytes.Length);
                        await requestStream.WriteAsync(bytes, 0, count);
                    }
                }

                while (requestStream.DataAvailable)
                {
                    count = await requestStream.ReadAsync(bytes, 0, bytes.Length);
                    await proxyStream.WriteAsync(bytes, 0, count);
                }


                // attempt to detect a timeout / end of data avaiable
                var timeout = 0;
                while (!proxyStream.DataAvailable && !requestStream.DataAvailable)
                {
                    if (timeout > 5)
                    {
                        bActive = false;
                        break;
                    }

                    await Task.Delay(10);
                    timeout++;
                }
            }

        }

    }
    catch (Exception e)
    {
        Console.WriteLine(e.ToString());
    }
}

更新

根据 AgentFire 的回答，我现在得到了以下工作代码：

public static async Task HandleDisconnect(TcpClient tcp, TcpClient tcp2, CancellationToken cancellationToken)
{
    while (true)
    {
        if (tcp.Client.Poll(0, SelectMode.SelectRead))
        {
            byte[] buff = new byte[1];

            if (tcp.Client.Receive(buff, SocketFlags.Peek) == 0)
            {
                // Client disconnected
                Console.WriteLine("The requesting client has dropped its connection.");
                cancellationToken = new CancellationToken(true);
                break;
            }
        }
        if (tcp2.Client.Poll(0, SelectMode.SelectRead))
        {
            byte[] buff = new byte[1];

            if (tcp2.Client.Receive(buff, SocketFlags.Peek) == 0)
            {
                // Server disconnected
                Console.WriteLine("The destination client has dropped its connection.");
                cancellationToken = new CancellationToken(true);
                break;
            }
        }

        await Task.Delay(1);
    }
}


private static async Task HandleClient(TcpClient clt)
{
    List<Task> tasks            = new List<Task>();
    var bytes                   = new byte[clt.ReceiveBufferSize];
    var hostHeaderAvailable     = 0;
    NetworkStream requestStream = null;
    const string connectText    = "connect";

    try
    {
        using (NetworkStream proxyStream = clt.GetStream())
        using (TcpClient requestClient = new TcpClient())
        {
            proxyStream.ReadTimeout = 100;
            proxyStream.WriteTimeout = 100;

            if (proxyStream.DataAvailable && hostHeaderAvailable == 0)
            {
                await proxyStream.ReadAsync(bytes, 0, bytes.Length);

                var text = Encoding.UTF8.GetString(bytes);
                Console.WriteLine(text);

                if (text.ToLower().StartsWith(connectText))
                {
                    // extract the url and port
                    var host = text.Remove(0, connectText.Length + 1);
                    var hostIndex = host.IndexOf(" ", StringComparison.Ordinal);
                    var hostEntry = host.Remove(hostIndex).Split(new[] { ":" }, StringSplitOptions.None);
                    // connect to the url and prot supplied
                    await requestClient.ConnectAsync(hostEntry[0], Convert.ToInt32(hostEntry[1]));
                    requestStream = requestClient.GetStream();

                    requestStream.ReadTimeout = 100;
                    requestStream.WriteTimeout = 100;

                    // send 200 response to proxyStream 
                    const string sslResponse = "HTTP/1.0 200 Connection established\r\n\r\n";
                    var sslResponseBytes = Encoding.UTF8.GetBytes(sslResponse);
                    await proxyStream.WriteAsync(sslResponseBytes, 0, sslResponseBytes.Length);
                }
            }
            hostHeaderAvailable++;

            CancellationToken cancellationToken = new CancellationToken(false);

            Task task               = proxyStream.CopyToAsync(requestStream, cancellationToken);
            Task task2              = requestStream.CopyToAsync(proxyStream, cancellationToken);
            Task handleConnection   = HandleDisconnect(clt, requestClient, cancellationToken);

            tasks.Add(task);
            tasks.Add(task2);
            tasks.Add(handleConnection);
                
            await Task.WhenAll(tasks).ConfigureAwait(false);

            // close conenctions
            clt.Close();
            clt.Dispose();
            requestClient.Close();
            requestClient.Dispose();
        }
    }
    catch (Exception e)
    {
        Console.WriteLine(e.ToString());
    }
}

更新

尝试使用 CancellationTokenSource

CancellationTokenSource source = new CancellationTokenSource();
CancellationToken cancellationToken = source.Token;
TaskFactory factory = new TaskFactory(cancellationToken);

tasks.Add(factory.StartNew(() => {proxyStream.CopyToAsync(requestStream);}, cancellationToken));
tasks.Add(factory.StartNew(() => {requestStream.CopyToAsync(proxyStream);}, cancellationToken));
tasks.Add(factory.StartNew(async () => {
    //wait for this to retur, then cancel the token
    await HandleDisconnect(clt, requestClient);
    source.Cancel();
}, cancellationToken));

try
{
    await factory.ContinueWhenAll(tasks.ToArray(),
                                                 (results) =>
                                                 {
                                                     Console.WriteLine("Tasks complete");
                                                 }, cancellationToken);
}
catch (AggregateException ae)
{
    foreach (Exception e in ae.InnerExceptions)
    {
        if (e is TaskCanceledException)
            Console.WriteLine("Unable to compute mean: {0}",
                              ((TaskCanceledException)e).Message);
        else
            Console.WriteLine("Exception: " + e.GetType().Name);
    }
}
finally
{
    source.Dispose();
}

更新

public static class extensionTcpClient{

    public static bool CheckIfDisconnected(this TcpClient tcp)
    {
        if (tcp.Client.Poll(0, SelectMode.SelectRead))
        {
            byte[] buff = new byte[1];

            if (tcp.Client.Receive(buff, SocketFlags.Peek) == 0)
            {
                // Client disconnected
                return false;
            }
        }

        return true;
    }
}


class ProxyMaintainer
{

    private static TcpListener listener = new TcpListener(IPAddress.Parse("192.168.0.25"), 13000);

    public ProxyMaintainer()
    {
    }

    public async Task Start()
    {
        Console.WriteLine("###############################");
        Console.WriteLine("Listening on 192.168.0.25:13000");
        Console.WriteLine("###############################\n");

        listener.Start();

        while (listen)
        {
            if (listener.Pending())
            {
                HandleClient(await listener.AcceptTcpClientAsync());
            }
            else
            {
                await Task.Delay(100); //<--- timeout
            }
        }
    }


    private static async Task Transport(NetworkStream from, NetworkStream to, Func<bool> isAlivePoller, CancellationToken token)
    {
        byte[] buffer = new byte[4096];

        while (isAlivePoller())
        {
            while (from.DataAvailable)
            {
                int read = await from.ReadAsync(buffer, 0, buffer.Length, token).ConfigureAwait(false);
                await to.WriteAsync(buffer, 0, read, token);
            }

            // Relieve the CPU a bit.
            await Task.Delay(10, token).ConfigureAwait(false);
        }
    }


    private static async Task HandleClient(TcpClient clientFrom)
    {
        var hostHeaderAvailable = 0;
        int count;
        var bytes = new byte[clientFrom.ReceiveBufferSize];
        const string connectText = "connect";
        NetworkStream toStream = null;

        using (var fromStream = clientFrom.GetStream())
        using(TcpClient clientTo = new TcpClient())
        using (var manualStopper = new CancellationTokenSource())
        {
            count = await fromStream.ReadAsync(bytes, 0, bytes.Length);

            var text = Encoding.UTF8.GetString(bytes);
            Console.WriteLine(text);

            if (text.ToLower().StartsWith(connectText))
            {
                // extract the url and port
                var host = text.Remove(0, connectText.Length + 1);
                var hostIndex = host.IndexOf(" ", StringComparison.Ordinal);
                var hostEntry = host.Remove(hostIndex).Split(new[] { ":" }, StringSplitOptions.None);
                // connect to the url and prot supplied
                await clientTo.ConnectAsync(hostEntry[0], Convert.ToInt32(hostEntry[1]));
                toStream = clientTo.GetStream();

                // send 200 response to proxyStream 
                const string sslResponse = "HTTP/1.0 200 Connection established\r\n\r\n";
                var sslResponseBytes = Encoding.UTF8.GetBytes(sslResponse);
                await fromStream.WriteAsync(sslResponseBytes, 0, sslResponseBytes.Length);
            }
            
            bool Poller() => clientFrom.CheckIfDisconnected() && clientTo.CheckIfDisconnected();

            Task one = Transport(fromStream, toStream, Poller, manualStopper.Token);
            Task two = Transport(toStream, fromStream, Poller, manualStopper.Token);

            await Task.WhenAll(one, two).ConfigureAwait(false);
            //await one; await two; // To get exceptions if you want them and there are any.
            // Alternatively, you can use Task.WhenAll to get exceptions aggregated for you.
        }

        Console.WriteLine("Closing connection");
    }



}

Answer 1

好吧，告诉你什么。 对于 HTTP，数据可用性仅在于一个参数（如果我们省略 WebSocket 之类的东西），该参数称为Connection并作为 Header 作为两种可能状态之一传递： Close或Keep-Alive 。

如果客户端选择Close ，服务器必须在请求被提供后立即关闭连接，而Keep-Alive告诉服务器，如果它不想，它可以为另一个请求保持连接打开。

让我们考虑这两种情况。

如果客户端选择 Keep-Alive，则连接将无限期地持久并按预期工作。 但：

如果任何一方断开连接，有一种简单的方法可以检测到。 这段代码是在 StackOverflow 上找到的，并被告知它仍然可以完美运行：

public static bool CheckIfDisconnected(this TcpClient tcp)
{
    if (tcp.Client.Poll(0, SelectMode.SelectRead))
    {
        byte[] buff = new byte[1];

        if (tcp.Client.Receive(buff, SocketFlags.Peek) == 0)
        {
            // Client disconnected
            return true;
        }
    }

    return false;
}

所以我相信你，作为一个代理服务器，根本没有义务管理连接状态，可以把它留给实际的通信方。 您所要做的就是检测您的连接 - 代理或请求 - 何时被丢弃，丢弃另一个并收工。

PS 现在，您还询问了异步性。

我必须补充一点，TCP 连接被认为是全双工的。 这意味着您可以自由创建两个异步运行的任务，读取和写入它们自己的接收器。 我的想法是，这将是最佳的行动方案。

Answer 2

回答你的其他问题

您仍在使用Stream.CopyToAsync ，正如我告诉过您的那样，只要任何通信方决定在发送另一块数据之前稍等Stream.CopyToAsync ， Stream.CopyToAsync不会成功。

您的解决方案也有些过于复杂。

我会这样说：

async Task Transport(NetworkStream from, NetworkStream to, Func<bool> isAlivePoller, CancellationToken token)
{
    byte[] buffer = new byte[4096];

    while (isAlivePoller())
    {
        while (from.DataAvailable)
        {
            int read = await from.ReadAsync(buffer, 0, buffer.Length, token).ConfigureAwait(false);
            await to.WriteAsync(buffer, 0, read, token).ConfigureAwait(false);
        }

        // Relieve the CPU a bit.
        await Task.Delay(100, token).ConfigureAwait(false);
    }
}

然后在你的主代码中：

using TcpClient clientFrom = ...;
using TcpClient clientTo = ...;
using var fromStream = clientFrom.GetStream();
using var toStream = clientTo.GetStream();
using var manualStopper = new CancellationTokenSource();

bool Poller() => clientFrom.CheckIfDisconnected() && clientTo.CheckIfDisconnected();

Task one = Transport(fromStream, toStream, Poller, stopper.Token);
Task two = Transport(toStream, fromStream, Poller, stopper.Token);

await Task.WhenAny(one, two).ConfigureAwait(false);
//await one; await two; // To get exceptions if you want them and there are any.
// Alternatively, you can use Task.WhenAll to get exceptions aggregated for you.

到这里你就大功告成了。