How to store return data from multiple async requests using HttpClient() until all requests complete?

Question

I need to iterate through a series of post requests for data from a url. The requests need to be async. What is the best way to check all returns have completed, and to store that data until all data returned? Prefer to use HttpClient()

Code snippet:

     HttpContent httpContent = new FormUrlEncodedContent(postData);           
     HttpResponseMessage response =  client.PostAsync("/mydata", httpContent).Result; 

     var responsecode = (int)response.StatusCode; 

     if (response.IsSuccessStatusCode)
     {
         var responseBodyAsText = await response.Content.ReadAsStringAsync();
         return responseBodyAsText;
     }
     else
     {
         return responsecode +" " +response.ReasonPhrase; 
     }

`

Answer 1

Well, first, bear in mind that you've got an async problem here:

 HttpContent httpContent = new FormUrlEncodedContent(postData);           

 // whoops! .Result makes this line synchronous.
 HttpResponseMessage response =  client.PostAsync("/mydata", httpContent).Result;

 var responsecode = (int)response.StatusCode; 

 if (response.IsSuccessStatusCode)
 {
     var responseBodyAsText = await response.Content.ReadAsStringAsync();
     return responseBodyAsText;
 }
 else
 {
     return responsecode +" " +response.ReasonPhrase; 
 }

Next, since HttpClient gives you a Task<HttpResponseMessage> , it seems that you want to have all of the responses, as strings, as something like, say, a Task<IEnumerable<string>> , right? We can easily write a function to turn a Task<HttpResponseMessage> into a Task<string> :

public async Task<string> ReadResultAsync(Task<HttpResponseMessage> responseTask)
{
    var response = await responseTask;

    var responsecode = (int)response.StatusCode; 

    if (response.IsSuccessStatusCode)
    {
        var responseBodyAsText = await response.Content.ReadAsStringAsync();

        return responseBodyAsText;
    }
    else
    {
        return responsecode + " " + response.ReasonPhrase; 
    }
}

Now let's say you've got some collection of post data that you need to turn into this async collection of strings, called myData :

// bear in mind, this code doesn't (necessarily) need to be in a
//  method marked "async". If you want to await on resultsTask, though,
//  it would need to be in an async method.

var tasks = myData
    .Select(x => new FormUrlEncodedContent(x)) //  IEnumerable<FormUrlEncodedContent>
    .Select(x => client.PostAsync("/mydata", x)) // IEnumerable<Task<HttpResponseMessage>>
    .Select(x => ReadResultAsync(x)) // IEnumerable<Task<string>>
    .ToArray(); // Task<string>[]

var resultsTask = Task.WhenAll(tasks); // here is Task<string[]>

Answer 2

write a an asynchronous function that fires your post data (i just used your code, please add correct error/exception handling):

async Task<string> PostDataAsync(Dictionary<string, string> postData)
{
    var httpContent = new FormUrlEncodedContent(postData);           
    var response = await client.PostAsync("/mydata", httpContent).ConfigureAwait(false); 

    var responsecode = (int)response.StatusCode; 

    if (response.IsSuccessStatusCode)
    {
        var responseBodyAsText = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
        return responseBodyAsText;
    }
    else
    {
       return responsecode +" " +response.ReasonPhrase; 
    }
}

Now, let's say you have a list of post data 'List> postDataCollection`, then build your requests

var postRequests = postDataCollection.Select(pd => PostDataAsync(pd)).ToArray();

and then wait for all of them to finish

var postResponses = await Task.WhenAll(postRequests);