Translating cURL to HttpClient.PutAsync

Question

I am trying to get a few cURL commands translated to a C# implementation, but I am running into an Exception which I can't solve.

I tried to gather up as much information as I can in the hopes that somebody can help me further. Here it goes...

The cURL statement:

curl -i -X PUT "http://[ipaddress]:[port]/webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=[username]&op=CREATE"

The C# version:

var response = await client.PutAsync(
    "http://[ipaddress]:[port]/webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=[username]&op=CREATE",
    null);

The C# version results in a WebException :

HResult = -2146233088

Message = The remote name could not be resolved: 'ip-172-31-9-79.eu-central-1.compute.internal'

The server I am connecting to, is a Hadoop server. There are multiple instances running on that server. So when I store a file, the server replies with information on which instance this file is stored (so that I can reference that instance when I want to write to this file).

Based on the error message, it seems to be that it receives some kind of reference to an IP address it can't access (which makes sense, as that ip address is an internal IP address in that Hadoop server.

I used Wireshark to find the difference in the requests which are sent.

Using cURL:

Frame 57: 204 bytes on wire (1632 bits), 204 bytes captured (1632 bits) on interface 0
Ethernet II, Src: IntelCor_da:f4:44 (fc:f8:ae:da:f4:44), Dst: AsustekC_32:7d:b0 (ac:22:0b:32:7d:b0)
Internet Protocol Version 4, Src: 192.168.1.107, Dst: [ipaddress]
Transmission Control Protocol, Src Port: 60454, Dst Port: 50070, Seq: 1, Ack: 1, Len: 150
Hypertext Transfer Protocol
    PUT /webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): PUT /webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE HTTP/1.1\r\n]
            [PUT /webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: PUT
        Request URI: /webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE
            Request URI Path: /webhdfs/v1/[appname]/staging/sensors/aap.txt
            Request URI Query: user.name=hdfs&op=CREATE
                Request URI Query Parameter: user.name=hdfs
                Request URI Query Parameter: op=CREATE
        Request Version: HTTP/1.1
    Host: [ipaddress]:50070\r\n
    User-Agent: curl/7.50.0\r\n
    Accept: */*\r\n
    \r\n
    [Full request URI: http://[ipaddress]:50070/webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE]
    [HTTP request 1/1]
    [Response in frame: 59]

Using the HttpClient :

Frame 381: 209 bytes on wire (1672 bits), 209 bytes captured (1672 bits) on interface 0
Ethernet II, Src: IntelCor_da:f4:44 (fc:f8:ae:da:f4:44), Dst: AsustekC_32:7d:b0 (ac:22:0b:32:7d:b0)
Internet Protocol Version 4, Src: 192.168.1.107, Dst: [ipaddress]
Transmission Control Protocol, Src Port: 60541, Dst Port: 50070, Seq: 1, Ack: 1, Len: 155
Hypertext Transfer Protocol
    PUT /webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): PUT /webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE HTTP/1.1\r\n]
            [PUT /webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: PUT
        Request URI: /webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE
            Request URI Path: /webhdfs/v1/[appname]/staging/sensors/aap.txt
            Request URI Query: user.name=hdfs&op=CREATE
                Request URI Query Parameter: user.name=hdfs
                Request URI Query Parameter: op=CREATE
        Request Version: HTTP/1.1
    Host: [ipaddress]:50070\r\n
    Content-Length: 0\r\n
    Connection: Keep-Alive\r\n
    \r\n
    [Full request URI: http://[ipaddress]:50070/webhdfs/v1/[appname]/staging/sensors/aap.txt?user.name=hdfs&op=CREATE]
    [HTTP request 1/1]
    [Response in frame: 383]

To me, the only notable (but for me meaningless) differences are:

• User-Agent: curl/7.50.0\\r\\n
• Accept: / \\r\\n

vs

• Content-Length: 0\\r\\n
• Connection: Keep-Alive\\r\\n

I have very little experience with REST programming, and clearly not enough knowledge on analyzing web requests. Some help / guidance / explanation would be greatly appreciated.

Just for completeness:

• Yes I am aware that there are Windows executables for cURL, but I run on Raspberry PI / Windows IoT. As far as I know there is nothing for that specific platform (yet)
• In the above statements I left out ipaddress/appname/etc intentionally just to be safe side wrt security issues

Answer 1

I found out that the only thing which goes wrong is reading the response in C#. The HTTP responses monitored over Wireshark are identical (using cURL.exe and using PutAsync in C#).

I was able to ignore the redirect and be given the opportunity to handle the response myself.

So disabling auto redirect (and handle the response myself) solved my problem:

var httpClientHandler = new HttpClientHandler {AllowAutoRedirect = false};
var client = new HttpClient(httpClientHandler);