无法分配请求的地址（使用 Docker 时）

Question

I'm using this library and all works well when I run my application locally. I've implemented TorSharp in a NetworkHelper class:

using Knapcode.TorSharp;
using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;

namespace SWP.Helpers
{
    public class NetworkHelper
    {
        private int _maxRequests = 100;

        private int _failedAttempts = 10;

        private HttpClient _httpClient = null;

        private TorSharpProxy _proxy = null;

        public async Task<string> GetHtmlAsync(Uri url)
        {
            try
            {
                if (_httpClient == null)
                {
                    var settings = new TorSharpSettings
                    {
                        ZippedToolsDirectory = Path.Combine(Path.GetTempPath(), "TorZipped"),
                        ExtractedToolsDirectory = Path.Combine(Path.GetTempPath(), "TorExtracted"),
                        PrivoxySettings =
                        {
                            Port = 1337,

                        },
                        TorSettings =
                        {
                            ControlPort = 1338,
                            ControlPassword = "foobar"
                        }
                    };

                    // download tools
                    await new TorSharpToolFetcher(settings, new HttpClient()).FetchAsync();

                    // execute
                    _proxy = new TorSharpProxy(settings);
                    var handler = new HttpClientHandler
                    {
                        Proxy = new WebProxy(new Uri("http://localhost:" + settings.PrivoxySettings.Port))
                    };

                    var httpClient = new HttpClient(handler);
                    _httpClient = new HttpClient(handler);

                    await _proxy.ConfigureAndStartAsync();
                }

                string response = await _httpClient.GetStringAsync(url);

                if (_maxRequests == 0)
                {
                    await _proxy.GetNewIdentityAsync();
                    _maxRequests = 100;
                    Console.WriteLine("Ottenimento nuova identità...");
                    var newIP = await _httpClient.GetStringAsync("http://api.ipify.org");
                    Console.WriteLine("Nuovo IP: " + newIP);
                }

                _maxRequests--;

                return response;
            }
            catch (Exception)
            {
                if (_failedAttempts != 0)
                {
                    Console.WriteLine("Failed request, retry ...");
                    _failedAttempts--;
                    await Task.Delay(15000);
                }
                else
                {
                    //_proxy.Stop();
                    throw;
                }

                return await GetHtmlAsync(url);
            }
        }
    }
}

So when I have to grab the html from a web page I simply do:

string html = await _parser.NetworkHelper.GetHtmlAsync('somelink');

Now the prroblem is that when I run the application locally, so in Visual Studio, everything works well, but when I run it inside a Docker container I get this error:

[16-01-2021 11:55:25]
Jan 16 11:57:33.001 [notice] Tor 0.4.4.6 (git-2a8b789ea6f308d0) running on Linux with Libevent 2.1.11-stable, OpenSSL 1.1.1i, Zlib 1.2.8, Liblzma N/A, and Libzstd N/A.
Jan 16 11:57:33.031 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jan 16 11:57:33.031 [notice] Read configuration file "/tmp/TorExtracted/tor-linux64-10.0.8/Data/Tor/torrc".
Jan 16 11:57:33.034 [notice] Opening Socks listener on 127.0.0.1:19050
Jan 16 11:57:33.063 [notice] Opened Socks listener on 127.0.0.1:19050
Jan 16 11:57:33.159 [notice] Opening Control listener on 127.0.0.1:1338
Jan 16 11:57:33.175 [notice] Opened Control listener on 127.0.0.1:1338
Jan 16 11:57:33.175 [warn] Fixing permissions on directory /tmp/TorExtracted/tor-linux64-10.0.8/Data/Tor
Jan 16 11:57:33.000 [warn] You are running Tor as root. You don't need to, and you probably shouldn't.
/tmp/TorExtracted/privoxy-linux64-3.0.29/usr/sbin/privoxy: error while loading shared libraries: libbrotlidec.so.1: cannot open shared object file: No such file or directory


Jan 16 11:57:33.000 [notice] Bootstrapped 0% (starting): Starting
Jan 16 11:57:33.000 [notice] Starting with guard context "default"
Jan 16 11:57:34.000 [notice] Bootstrapped 5% (conn): Connecting to a relay
Jan 16 11:57:34.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
Jan 16 11:57:34.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay
Jan 16 11:57:34.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done
Jan 16 11:57:34.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection
Jan 16 11:57:34.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
Jan 16 11:57:35.000 [notice] Bootstrapped 30% (loading_status): Loading networkstatus consensus
Jan 16 11:57:36.000 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus.
Jan 16 11:57:36.000 [notice] Bootstrapped 40% (loading_keys): Loading authority key certs
Jan 16 11:57:37.000 [notice] The current consensus has no exit nodes. Tor can only build internal paths, such as paths to onion services.
Jan 16 11:57:37.000 [notice] Bootstrapped 45% (requesting_descriptors): Asking for relay descriptors
Jan 16 11:57:37.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/6896, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.)
Jan 16 11:57:37.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/6896, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.)
Jan 16 11:57:38.000 [notice] Bootstrapped 50% (loading_descriptors): Loading relay descriptors
Failed request, retry ...
Jan 16 11:57:39.000 [notice] The current consensus contains exit nodes. Tor can build exit and internal paths.
Jan 16 11:57:42.000 [notice] Bootstrapped 57% (loading_descriptors): Loading relay descriptors
Jan 16 11:57:42.000 [notice] Bootstrapped 64% (loading_descriptors): Loading relay descriptors
Jan 16 11:57:43.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
Jan 16 11:57:43.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
Jan 16 11:57:43.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
Jan 16 11:57:44.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
Jan 16 11:57:44.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
Jan 16 11:57:44.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
Jan 16 11:57:45.000 [notice] Bootstrapped 100% (done): Done
Failed request, retry ...
System.Net.Http.HttpRequestException: Cannot assign requested address ---> System.Net.Sockets.SocketException: Cannot assign requested address
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.EstablishProxyTunnel(CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at System.Net.Http.HttpClient.GetStringAsyncCore(Task`1 getTask)
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 72
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 101
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Helpers.NetworkHelper.GetHtmlAsync(Uri url) in /src/SWP/Helpers/NetworkHelper.cs:line 104
at SWP.Controllers.CountryController.GetDomesticCountries() in /src/SWP/Controllers/CountryController.cs:line 28
at SoccerWay.Business.CountryManager.GetCountries(Nullable`1 international) in /src/SoccerWay/Business/Manager/Utility/Country.Utility.cs:line 59
at SoccerWay.Scraper.StartAsync(String instance) in /src/SoccerWay/Scraper.cs:line 55
there is the following line that caught my attention:

/tmp/TorExtracted/privoxy-linux64-3.0.29/usr/sbin/privoxy: error while loading shared libraries: libbrotlidec.so.1: cannot open shared object file: No such file or directory

This is the Dockerfile that I use to compile the Docker image:

FROM mcr.microsoft.com/dotnet/core/sdk:2.1 AS build
WORKDIR /src

# Copy csproj and restore as distinct layers
COPY /SoccerWay/*.csproj SoccerWay/
COPY /SWP/*.csproj SWP/
RUN dotnet restore SWP/*.csproj
RUN dotnet restore SoccerWay/*.csproj 

# Copy and build app and libraries
COPY SoccerWay/ SoccerWay/
COPY SWP/ SWP/
WORKDIR /src/SoccerWay
RUN dotnet build -c release 

FROM build AS publish
RUN dotnet publish -c release -o /app

# Final stage/image
FROM mcr.microsoft.com/dotnet/core/runtime:2.1
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "SoccerWay.dll"]

How can I fix this?

I've created a demo application to replicate the problem, you can download from here .

UPDATE

Tried the solution proposed but still getting the same problem, the Dockerfile have now this structure:

FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build

# add missing packages
RUN apt update -y && apt-get install -y libbrotli1 libmbedtls-dev && apt-get clean

WORKDIR /src

# Copy csproj and restore as distinct layers
COPY /test-app/*.csproj test-app/
RUN dotnet restore test-app/*.csproj 

# Copy and build app and libraries
COPY test-app/ test-app/
WORKDIR /src/test-app
RUN dotnet build -c release 

FROM build AS publish
RUN dotnet publish -c release -o /app

# Final stage/image
FROM mcr.microsoft.com/dotnet/core/runtime:3.1
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "test-app.dll"]

what I did for run the container is the following:

docker build -t test .
docker run -d --name test_1 --restart=always test

and all the http requests will fail, also I get this error:

Unhandled exception. System.Net.Http.HttpRequestException: Cannot assign requested address

---> System.Net.Sockets.SocketException (99): Cannot assign requested address

at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)

--- End of inner exception stack trace ---

Answer 1

The error message is pretty clear - your image is missing required packages.
Just add them to the final image .
I tested your Dockerfile with added dependencies and it worked fine.

## Final stage/image
FROM mcr.microsoft.com/dotnet/core/runtime:3.1

## add missing packages
RUN apt update -y && apt-get install -y libbrotli1 libmbedtls12 && apt-get clean

WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "test-app.dll"]

Answer 2

Maybe your application requires special Linux capabilities not available to a Docker container by default, especially Network-related capabilities. Try running your container with --privileged flag (or privileged: true with Docker Compose).

For finer-grained capabilities, see Runtime privilege and Linux caps docs