How to make concurrent API calls in C#?

Question

I am needing to pull custom fields from a mortgage API. The problem is that there are 11000 records in total and it takes 1 second per API request. I want to find a way to send requests asynchronously and in parallel to make this more efficient.

I have tried looping through all the requests then having a Task.WaitAll() to wait for a response to return. I only receive two responses then the application waits indefinitely.

I first set up a static class for the HttpClient

 public static class ApiHelper
    {
        public static HttpClient ApiClient { get; set; }

        public static void InitializeClient()
        {
            ApiClient = new HttpClient();
            ApiClient.DefaultRequestHeaders.Add("ContentType", "application/json");
        }
    }

I gather my mortgage ID list and loop through the API Post Calls

        static public DataTable GetCustomFields(DataTable dt, List<string> cf, string auth)
        {

                //set auth header
                ApiHelper.ApiClient.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", auth);

                //format body
                string jsonBody = JArray.FromObject(cf).ToString();
                var content = new StringContent(jsonBody, Encoding.UTF8, "application/json");



                var responses = new List<Task<string>>();


                foreach (DataRow dr in dt.Rows)
                {

                    string guid = dr["GUID"].ToString().Replace("{", "").Replace("}", ""); //remove {} from string

                    responses.Add(GetData(guid, content));

                }

                Task.WaitAll(responses.ToArray());
                //some code here to process through the responses and return a datatable

                return updatedDT;

        }

Each API call requires the mortgage ID (GUID) in the URL

  async static Task<string> GetData(string guid, StringContent json)
        {

            string url = "https://api.elliemae.com/encompass/v1/loans/" + guid + "/fieldReader";
            Console.WriteLine("{0} has started .....", guid);
            using (HttpResponseMessage response = await ApiHelper.ApiClient.PostAsync(url, json))
            {
                if (response.IsSuccessStatusCode)
                {
                    Console.WriteLine("{0} has returned response....", guid);
                    return await response.Content.ReadAsStringAsync();
                }
                else
                {
                    Console.WriteLine(response.ReasonPhrase);
                    throw new Exception(response.ReasonPhrase);
                }

            }

        }

I'm only testing 10 records right now and send all 10 requests. But I only receive two back.

Result is here .

Could you please advise me on the right way of sending the concurrent API calls?

Answer 1

All GetData Task are using the same HttpClient singleton instance. The HttpClient cannot server multiple calls at the same time. Best practice is to use a Pool of HttpClient to ensure there is no Task accessing the same HttpClient at the same time.

Also, be careful throwing exception in Task, it will stops the WaitAll() at first thrown exception

Solution I've posted the entire project here: https://github.com/jonathanlarouche/stackoverflow_58137212
This solution send 25 requests using a max sized pool of [3];

Basically, the ApiHelper contains an HttpClient pool , using the Generic class ArrayPool<T> . You can use any other Pooling library, I just wanted to post a self-contained solution .

Suggested ApiHelper Bellow, this class now contains a pool and a Use method that receive an Action , an Item from the pool will be "rented" for the duration of the Action, then it will be returned into the pool via the ArrayPool.Use function. Use function receive also the apiToken to change the Request Auth Header.

public static class ApiHelper
{
    public static int PoolSize { get => apiClientPool.Size; }

    private static ArrayPool<HttpClient> apiClientPool = new ArrayPool<HttpClient>(() => {
        var apiClient = new HttpClient();
        apiClient.DefaultRequestHeaders.Add("ContentType", "application/json");
        return apiClient;
    });

    public static Task Use(string apiToken, Func<HttpClient, Task> action)
    {
        return apiClientPool.Use(client => {
            client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", apiToken);
            return action(client);
        });
    }
}

GetData function. Get Data will receive the apiToken and will await the ApiHelper.Use function. New instance of StringContent() object need to be done in this function as it cannot be reused across different Http Post Call.

async static Task<string> GetData(string apiToken, Guid guid, string jsonBody)
{

    string url = "https://api.elliemae.com/encompass/v1/loans/" + guid + "/fieldReader";
    Console.WriteLine("{0} has started .....", guid);
    string output = null;
    await ApiHelper.Use(apiToken, (client) => 
    {
        var json = new StringContent(jsonBody, Encoding.UTF8, "application/json");
        return client.PostAsync(url, json).ContinueWith(postTaskResult =>
        {

            return postTaskResult.Result.Content.ReadAsStringAsync().ContinueWith(s => {

                output = s.Result;
                return s;
            });
        });
    });
    Console.WriteLine("{0} has finished .....", guid);
    return output;
}

ArrayPool

public class ArrayPool<T>
{
    public int Size { get => pool.Count(); }
    public int maxSize = 3;
    public int circulingObjectCount = 0;
    private Queue<T> pool = new Queue<T>();
    private Func<T> constructorFunc;

    public ArrayPool(Func<T> constructorFunc) {
        this.constructorFunc = constructorFunc;
    }

    public Task Use(Func<T, Task> action)
    {
        T item = GetNextItem(); //DeQueue the item
        var t = action(item);
        t.ContinueWith(task => pool.Enqueue(item)); //Requeue the item
        return t;
    }

    private T GetNextItem()
    {
        //Create new object if pool is empty and not reached maxSize
        if (pool.Count == 0 && circulingObjectCount < maxSize)
        {
            T item = constructorFunc();
            circulingObjectCount++;
            Console.WriteLine("Pool empty, adding new item");
            return item;
        }
        //Wait for Queue to have at least 1 item
        WaitForReturns();

        return pool.Dequeue();
    }

    private void WaitForReturns()
    {
        long timeouts = 60000;
        while (pool.Count == 0 && timeouts > 0) { timeouts--; System.Threading.Thread.Sleep(1); }
        if(timeouts == 0)
        {
            throw new Exception("Wait timed-out");
        }
    }
}