Using HttpClient how can I send GET requests and return a response in a loop that changes the endpoint

Question

Thank you for taking the time to read this.

So my goal is to pull some data from a website that has tons of pages by changing the Url in a loop.

ex.

    //want to change part or the url with this array.
    string[] catColors = string[]{"black","brown","orange"}; 

      for (int i = 0; i < catColors.Length; i++){

    string endpoint = $"www.ilovecats.com/color/{catColors[i]}";

      using (HttpClient client = new HttpClient())
                    using (HttpResponseMessage response = await client.GetAsync(endpoint))
                    using (HttpContent content = response.Content)
                    {
                        string data = await content.ReadAsStringAsync();
                        String result = Regex.Replace(data,REGEX,String.Empty);

                        if (data != null)
                        {
                          saveCat.Color = catColors[i].ToString();
                          saveCat.Info = result.Substring(266);
                          catholderList.Add(saveCat);
                        }

//...

What is happening is only the first request is returning the body content. the others are returning empty response bodies.

for ex.

cat1.color = black, cat1.Info ="really cool cat";
cat2.color = brown, cat2.Info =""; // nothing in body
//....same with other 

Is their a better way of accomplishing this as I dont want to change the endpoint manually for 1000s of records.

Answer 1

saveCat.Color = catColors[i].ToString();
saveCat.Info = result.Substring(266);
catholderList.Add(saveCat);

You only have a single saveCat object here. Within your loop, you never create a new object, so you keep changing the existing object. And you also add that single object to the list three times.

So in the end, you should end up with three identical objects inside of that list.

What you should do instead is create a new object for every iteration:

catholderList.Add(new Cat
{
    Color = catColors[i].ToString(),
    Info = result.Substring(266),
});