HttpWebRequest.GetResponse Returns 404 but URL works in browser

Question

In a console application in C#.NET, I need to get the XML from this URL:

https://bdm.insee.fr/series/sdmx/data/SERIES_BDM/000008630?startPeriod=2019

When I put the URL in my browser in works fine.

But when I try to get it in C#, it throws a 404 error.

I've tried everything I found on here which looked like my problem but I keep getting a 404.

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(@"https://bdm.insee.fr/series/sdmx/data/series_bdm/000008630?startPeriod=2019");
    request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11.0) like Gecko";
    request.Accept = "application/xml";
    HttpWebResponse response = null;
    try
    {
        response = (HttpWebResponse)
        request.GetResponse(); //I get the 404 not found here.
    }
    catch (WebException e)
    {
        Console.WriteLine("\r\nWebException Raised. The following error occured : {0}", e.Status);
    }

I don't think it's a proxy problem because I had that problem before and I added a few lines in the App.config to take care of that. Moreover for example I don't get a 404 for this url:

http://nds.nokia.com/uaprof/NN95_8GB-1r100.xml

One of my colleague told me it was an SDMX and not a just XML but wasn't able to help me.

I read somewhere something about the SDMXSource library. But I don't understand how to use it to make a request to get the XML. I found a 54 pages documentation on how to create a generic dissemination system usitng sdmxSource but I couldn't find a simple way to make my request work.

If someone can point me to where I should look, or find whatI made wrong, it would be a big help.

Thank you very much

Virgil

Answer 1

The problem seems to be that the url you're hitting is case-sensitive. The one that works is:

https://bdm.insee.fr/series/sdmx/data/SERIES_BDM/000008630?startPeriod=2019

While the one that doesn't is:

https://bdm.insee.fr/series/sdmx/data/series_bdm/000008630?startPeriod=2019

Please note the segment: SERIES_BDM . It should be upper cased.

"BUT WHY?" you might say:

While case-sensitivity in URLs could result kinda weird for those who work mostly on Windows-based OSes, it's perfectly normal when working in Unix-based systems. The history goes back to when servers returned only physical files when asked for a given resource and that's another story you can learn more about.

This is an interesting thread: https://webmasters.stackexchange.com/questions/90339/why-are-urls-case-sensitive/90341

You should go for more of course;-)

Hope this helps!

Answer 2

you given URL in HttpWebRequest is looking wrong comparing with this URL https://bdm.insee.fr/series/sdmx/data/SERIES_BDM/000008630?startPeriod=2019

try this

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(@"https://bdm.insee.fr/series/sdmx/data/SERIES_BDM/000008630?startPeriod=2019");
            request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11.0) like Gecko";
            request.Accept = "application/xml";
            HttpWebResponse response = null;
            try
            {
                response = (HttpWebResponse)
                request.GetResponse(); //it will return OK.
            }
            catch (WebException e)
            {
                Console.WriteLine("\r\nWebException Raised. The following error occured : {0}", e.Status);
            }