Get http 500 in a Azure worker role http request

Question

I have an azure web service that is running 1 worker role and 1 web role, the roles are set to be in an azure private network.

The worker role performs some sync work with our database that is downloading reports files from many APIs and servers and than sync them to our database, From some reason 1 (out of 10) API http request is returning HTTP error 500 in the cloud, but when i debug it on my machine it works, and also i have the same code running on EC2 (AWS) aspx website that is doing the same thing with a database that is running on AWS, that is working good.

Here is the code that performs that task (the one that fails):

public void GenerateReport(DateTime start, DateTime end)
        {
            string URL = this._cj_url;
            URL = string.Format(URL, start.ToString("yyyy-MM-dd"), end.ToString("yyyy-MM-dd"));
            RequestState myRequestState = new RequestState();
            myRequestState.request = (HttpWebRequest)WebRequest.Create(URL);
            myRequestState.from = start;
            myRequestState.to = end;
            myRequestState.prefix = "CJ";
            myRequestState.ext = "xml";

            try
            {
                myRequestState.request.Headers.Add("authorization:" + _devKey);
                myRequestState.request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1";

                HttpWebRequest myHttpWebRequest = myRequestState.request;
                myRequestState.response = (HttpWebResponse)myHttpWebRequest.GetResponse();

                // Read the response into a Stream object.
                Stream responseStream = myRequestState.response.GetResponseStream();
                myRequestState.streamResponse = responseStream;

                //' Get response  
                myRequestState.response = (HttpWebResponse)myRequestState.request.GetResponse();


                //' Get the response stream into a reader  
                StreamReader reader = new StreamReader(myRequestState.response.GetResponseStream());
                this.allFileRows = reader.ReadToEnd();

            }
            catch (Exception e1)
            {
                this.monitor.PushToErrors("CJ - error in downloading the file - " , e1,true);
            }

        }

Here is the RequestState class :

public class RequestState
    {
        // This class stores the State of the request.
        const int BUFFER_SIZE = 1024;
        public StringBuilder requestData;
        public byte[] BufferRead;
        public HttpWebRequest request;
        public HttpWebResponse response;
        public Stream streamResponse;
        public DateTime from;
        public DateTime to;
        public string prefix;
        public string ext;

        public RequestState()
        {
            BufferRead = new byte[BUFFER_SIZE];
            requestData = new StringBuilder("");
            request = null;
            streamResponse = null;
        }
    }

And this is the Url that the request calls to :

this._cj_url = "https://commission-detail.api.cj.com/v3/commissions?date-type=posting&start-date={0}&end-date={1}";

EDIT :

For some reason after i enabled the remote control to the service instances it got fixed. I am really wondering how come i had this problem...

Answer 1

One thought is that when you run your code in the cloud, you are running on their servers, meaning you dates could all be out of whack. I have seen it before where you assume DateTime.Now (or whatever you are doing) is your time, but its actually the server time of the VM you are running on in the cloud. The reason it could be a date issue is that I see you are passing in date parameters to the URL, perhaps they are formatted wrong when you are running in the cloud?