Missing chunk size in chunked HTTP response using libcurl

Question

I tried to use curl library to build a HTTP proxy. The problem I faced now is chunked HTTP response is not correct. By checking the file saved HTTP response, I found that the chunk size at begining and end of HTTP response is missing. The HTTP response got from curl callback function as show below.

size_t httpClient::write_data(char *ptr, size_t size, size_t nmemb, void* userdata)
{   

    // get total size   
    size_t realsize = size*nmemb;
    // get object
    struct MemoryStruct* mem = (struct MemoryStruct*)userdata;

    // size of memory block pointerd by the mem->memory is changed to the new bytes
    mem->memory = (char*)realloc(mem->memory, (mem->size + realsize));

    if(mem->memory == NULL)
    {
        printf("not enough memory (realloc returned NULL)\n");
            exit(1);
    }

    // copy to memory structure
    memcpy(&(mem->memory[mem->size]), ptr, realsize);
    mem->size += realsize;
    //mem->memory[mem->size] = 0;

    cout<<"Write ["<<realsize<<"] bytes Data to chunk"<<endl;
    return realsize;

}

Hope someone can point out the mistake I made for this callback function. Thanks :D

The expected chunked HTTP response is shown below:

HTTP/1.1 200 OK^M
Date: Wed, 17 Oct 2012 07:44:54 GMT^M
Expires: -1^M
Cache-Control: private, max-age=0^M
Content-Type: text/html; charset=UTF-8^M
Set-Cookie: PREF=ID=58da9f2271ea2d2b:FF=0:TM=1350459894:LM=1350459894:S=EnJS1hQo2d6_AnPM; expires=Fri, 17-Oct-2014 07:44:54 GMT; path=/; domain=.google.com^M
Set-Cookie: NID=65=RW0txpQSNA4NwlRhp0y1I6iF3L0xfugw8Bv4GMsB1yE1qu7iGoBO_2ZxqS0-DSeS4tJKnV26JlfVZmsnjxnjdUaHTDj3-AFREsvyMiE8wSKyabwYG8x-e18Pj8smdxUs; expires=Thu, 18-Apr-2013 07:44:54 GMT; path=/; domain=.google.com; HttpOnly^M
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."^M
Content-Encoding: gzip^M
Transfer-Encoding: chunked^M
Server: gws^M
X-XSS-Protection: 1; mode=block^M
X-Frame-Options: SAMEORIGIN^M
^M
907a^M

The number 907a in hex format is the chunk size missed in my HTTP response file.

Answer 1

I can propound to you next steps:

1. Into your part of code, that setup callback function, create struct MemoryStruct on stack.
2. Write argument into prototype your callback function as struct MemoryStruct *userdata in place of void *userdata .

3. Your code will be look as:

    MemoryStruct data; curl_easy_setopt(curl, CURLOPT_WRITEDATA, &data); curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, write_data); --------------------------------------------------------------------- // if you'll used std::string instead char* --------------------------------------------------------------------- size_t write_data(char *data, size_t size, size_t count, MemoryStruct *userdata) { int result = 0; if(userdata->memory != NULL) { userdata->memory->append(data, size*count); result = size*count; userdata->size += result; } return result; } 

After call function of curl library curl_easy_perform(CURL *handler) you can get right response.

PS. If i'm wrong understood your question, show your code where calls callback function