Delphi Upload a text file using HTTP

Question

I'm trying to upload a multi-line text file to be GeoCoded by BING.

1,en-US,,16630 Redmond Way,WA,USA,,,Redmond,98052,,,,,,,,,,,,,,,,,,,,,,
2,en-US,,16552 NE 74th St,WA,,,,Redmond,,,High,,,,,,,,,,,,,,,,,,,,,,
3,en-US,Seattle Space Needle,,,,,,,,,,,,,,,,,,,,,,,,,,,,
4,en-US,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
5,en-US,,W Jefferson Blvd,CA,,,,Los Angeles,90007,,,,,,,,,,,,,,,,,,,,,,,,
6,en-US,,,CA,,,,Los angeles,,,,,,,,,,,,,,,,,,,,,,,,,
7,en-ca,Montreal Canada,,,,,,,,,,,,,,,,,,,,,,,,,,,,
8,en-CA,,444 Kirkpatrick Cres NW,AB,Canada,,,Edmonton,,,,,,,,,,,,,,,,,,,,,,,,
9,en-gb,BD4 9JB,,,,,,,,,,,,,,,,,,,,,,,,,,,,
10,en-us,,,,,,,,,,,,,,,,,,,,,,,,,,,,47.673099,-122.11871
11,en-ca,,,,,,,,,,,,,,,,,,,,,,,,,,,,53.48021728,-113.4030925
12,en-gb,,,,,,,,,,,,,,,,,,,,,,,,,,,,53.77848387,-1.719561517

I'm using TIdHTTP at the moment, but it seems that the line breaks in the text file are removed when transmitting the file as they (Bing) only ever geocode the first line of the file and the rest is ignored.

lHTTP := TIdHTTP.Create(nil);
...

//Build the http request string
HTTPRequest := 'http://spatial.virtualearth.net/REST/v1/Dataflows/Geocode?input=csv&output=xml&key=' + BingKey;

//Setup the HTTP request parameters
lHTTP.Request.ContentType := 'text/plain';
lHTTP.Request.Method := 'POST';

//Send the request to Bing. Result in XMLResult StringStream
lHTTP.Post(HTTPRequest, lParamList, XMLResult);

The following example is from Microsoft but I'm not enough of an expert to adapt the HTTPWebRequest part using the stream to Delphi. Can anybody perhaps help?

//Parameters:
//   dataFilePath: The path to the file that contains the spatial data to geocode.
//   dataFormat: The format of the input data. Possible values are xml, csv, tab and pipe.
//   key: The Bing Maps Key to use for this job. The same key is used to get job status and download results.
//   description: Text that is used to describe the geocode dataflow job.
//Return value : A URL that defines the location of the geocode dataflow job that was created.
static string CreateJob(string dataFilePath, string dataFormat, string key, string description)
{
    //Define parameters for the HTTP request
    //
    // The 'Content-Type' header of the HTTP Request must be "text/plain" or "application/xml"
    // depending on the input data format.
    //
    string contentType = "text/plain";
    if (dataFormat.Equals("xml", StringComparison.OrdinalIgnoreCase))
        contentType = "application/xml";

    StringBuilder queryStringBuilder = new StringBuilder();

    //
    // The 'input'(input format) and 'key' (Bing Maps Key) parameters are required.
    //
    queryStringBuilder.Append("input=").Append(Uri.EscapeUriString(dataFormat));
    queryStringBuilder.Append("&");
    queryStringBuilder.Append("key=").Append(Uri.EscapeUriString(key));

    if (!String.IsNullOrEmpty(description))
    {
        //
        // The 'description' parameter is optional.
        //
        queryStringBuilder.Append("&");
        queryStringBuilder.Append("description=").Append(Uri.EscapeUriString(description));
    }

    //Build the HTTP URI that will upload and create the geocode dataflow job
    UriBuilder uriBuilder = new UriBuilder("http://spatial.virtualearth.net");
    uriBuilder.Path = "/REST/v1/dataflows/geocode";
    uriBuilder.Query = queryStringBuilder.ToString();

    //Include the data to geocode in the HTTP request
    using (FileStream dataStream = File.OpenRead(dataFilePath))
    {
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uriBuilder.Uri);

        //
        // The HTTP method must be 'POST'.
        //
        request.Method = "POST";
        request.ContentType = contentType;

        using (Stream requestStream = request.GetRequestStream())
        {
            byte[] buffer = new byte[16384];
            int bytesRead = dataStream.Read(buffer, 0, buffer.Length);
            while (bytesRead > 0)
            {
                requestStream.Write(buffer, 0, bytesRead);

                bytesRead = dataStream.Read(buffer, 0, buffer.Length);
            }
        }

        //Submit the HTTP request and check if the job was created successfully.
        using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
        {
            //
            // If the job was created successfully, the status code should be
            // 201 (Created) and the 'Location' header should contain a URL
            // that defines the location of the new dataflow job. You use this
            // URL with the Bing Maps Key to query the status of your job.
            //
            if (response.StatusCode != HttpStatusCode.Created)
                throw new Exception ("An HTTP error status code was encountered when creating the geocode job.");

            string dataflowJobLocation = response.GetResponseHeader("Location");
            if (String.IsNullOrEmpty(dataflowJobLocation))
                throw new Exception ("The 'Location' header is missing from the HTTP response when creating a goecode job.");

            return dataflowJobLocation;
        }
    }
}

Answer 1

2 main points in comparing the code. First you send your code as text/plain not application/xml and secondly they escape their data, you just send the XML. Have you tried it as escaped data?

Answer 2

Probably TStringList is changing the content of the input file. At least some of the bytes. It would be better to post the data as pure stream, either as memory or file stream.

Indy has the following overloaded methods for posting:

function Post(AURL: string; ASource: TStrings): string; overload;
function Post(AURL: string; ASource: TStream): string; overload;
function Post(AURL: string; ASource: TIdMultiPartFormDataStream): string; overload;
procedure Post(AURL: string; ASource: TIdMultiPartFormDataStream; AResponseContent: TStream); overload;
procedure Post(AURL: string; ASource: TStrings; AResponseContent: TStream); overload;

{Post data provided by a stream, this is for submitting data to a server}
procedure Post(AURL: string; ASource, AResponseContent: TStream); overload;

Several of them are appropriate for posting raw streams.