c++: memory not freed by free()

Question

I'm using libcurl and copying content from my http server. I need to fetch the segments and save the fetched segment in a new file.

The first segment is fetched correctly, but the second segment has the contents of the first segment appended to it, which it should not.

The memory should be freed after first segment is copied to my file but it does not happen.

Part of code is attached:

//code

char *m_pBuffer = NULL;
size_t m_Size = 0;


void* Realloc(void* ptr, size_t size)
{
    if(ptr)
        return realloc(ptr, size);
    else
        return malloc(size);
};

// Callback must be declared static, otherwise it won't link...
size_t WriteMemoryCallback(char* ptr, size_t size, size_t nmemb)
{
    // Calculate the real size of the incoming buffer
    size_t realsize = size * nmemb;

    // (Re)Allocate memory for the buffer
    m_pBuffer = (char*) realloc(m_pBuffer, m_Size + realsize);

    // Test if Buffer is initialized correctly & copy memory
    if (m_pBuffer == NULL) {
        realsize = 0;
    }

    memcpy(&(m_pBuffer[m_Size]), ptr, realsize);
    m_Size += realsize;


    // return the real size of the buffer...
    return realsize;
};

namespace ahs
{

    /* Construction / Initialisation */
    {
        return 0;
    }
    {
        curlpp::Cleanup cleaner;
        curlpp::Easy request;

        // Set the writer callback to enable cURL
        // to write result in a memory area
        curlpp::types::WriteFunctionFunctor functor(WriteMemoryCallback);
        curlpp::options::WriteFunction *test = new curlpp::options::WriteFunction(functor);
        request.setOpt(test);

        // Setting the URL to retrive.
        string abc = "http://192.168.0.34/ahs-1.0/example/ingest";
        string kk= abc + tobeused;
        request.setOpt(new curlpp::options::Url(kk));
        request.setOpt(new curlpp::options::Verbose(true));

        request.perform();

        print();
    }




    catch ( curlpp::LogicError & e )
    {
        std::cout << e.what() << std::endl;
    }
    catch ( curlpp::RuntimeError & e )
    {
        std::cout << e.what() << std::endl;
    }


    FILE *fp;
    if((fp = fopen(pp, "wb")) == NULL) {
        printf("Cannot open file.\n");
        exit(1);
    }

    if( fwrite(m_pBuffer, (long)m_Size, 1, fp) != 1) {
        printf("Write Error.\n");
        exit(1);
    }

    fclose(fp);
    free(m_pBuffer);

Answer 1

First, you are using C++. Use a std::vector to handle your buffers and then this problem goes away.

Secondly, you have a function called Realloc but are calling realloc , is this intentional?

Answer 2

This is a very dangerous call:

   // (Re)Allocate memory for the buffer
    m_pBuffer = (char*) realloc(m_pBuffer, m_Size + realsize);

If realloc returns NULL the data originally pointed to by m_pBuffer is lost: you have nothing pointing to it and you can no longer free it.

Rather than doing reallocs, by the way, you could use std::vector with appends, and if you don't require a contiguous buffer you can use std::deque with appends which is more efficient particularly for large buffers, and you can also use std::string, of course, even if you are writing some null bytes.

The issue with realloc the way you are doing it is it will do the reallocation every single time and move all your memory and it will end up O(N^2).

You should be able to get the estimated size. By the way, if all you want to do is write it to persistence can you not do this a buffer at a time, and not load it all into memory first?

(I think that may be what you are trying to do but are not).

Answer 3

You have a memcpy in there which is doing the appending...

memcpy(&(m_pBuffer[m_Size]), ptr, realsize);

So you reallocate the buffer (expand), and that does not cleanup the buffer as you expect, it leaves the content there, and the into the reallocated space, you copy in the new content... are you surprised? Did you copy the code from somewhere?

Answer 4

As far as I can tell you're getting what you asked for. You allocate some memory (the size of the first block) and populate it with the first block.

You then increase the size of this memory to be the size of the first and second block and populate it with the second block right after the first so you memory contains the first and second block.

This is fine if you write out this memory once at the end. If you write it out after each block then you eith need to remember where in the memory block you are or just allocate your memory for each block, populate it, write it out and then free it.

Answer 5

You just needed to put m_Size=0; after every fetched segment.