Libcurl progress callback not working with multi

Question

I'm trying to manage the progress of a download with libcurl in C++. I have managed to do this with curl_easy, but the issue with curl_easy is that it blocks the program until the request has been made.

I need to use curl_mutli so the http request is asynchronous, but when I try changing to curl_multi, my progress function stops working.

I have the following curl_easy request code:

int progressFunc(void* p, double TotalToDownload, double NowDownloaded, double TotalToUpload, double NowUploaded) {
    std::cout << TotalToDownload << ", " << NowDownloaded << std::endl;
    return 0;
}

FILE* file = std::fopen(filePath.c_str(), "wb");

curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, false);

curl_easy_setopt(curl, CURLOPT_XFERINFOFUNCTION, progressFunc);

curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeData);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, file);

CURLcode res = curl_easy_perform(curl);

which works perfectly and prints to the console the progress of the download.

However, when trying to modify this code to use curl_multi instead, the file does not download correctly (shows 0 bytes) and the download progress callback function shows only 0, 0 .

FILE* file = std::fopen(filePath.c_str(), "wb");

curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, false);

curl_easy_setopt(curl, CURLOPT_XFERINFOFUNCTION, progressFunc);

curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeData);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, file);
curl_multi_add_handle(curlm, curl);

int runningHandles;
CURLMcode res = curl_multi_perform(curlm, &runningHandles);

Answer 1

TL; DR : you are supposed to call curl_multi_perform in loop. If you don't use event loop and poll/epoll, you should probably stick with using curl_easy in separate thread.

---

The whole point of curl_multi API is not blocking: instead of magically downloading entire file in single call, you can use epoll or similar means to monitor curl's non-blocking sockets and invoke curl_multi_perform each time some data arrives from network. When you use it's multi-mode, curl itself does not start any internal threads and does not monitor it's sockets — you are expected to do it yourself. This allows writing highly performant event loops, that run multiple simultaneous curl transfers in the same thread. People, who need that, usually already have the necessary harness or can easily write it themselves.

The first time you invoke curl_multi_perform it will most likely return before the DNS resolution completes and/or before the TCP connection is accepted by remote side. So the amount of payload data transferred in first call will indeed be 0. Depending on server configuration, second call might not transfer any payload either. By "payload" I mean actual application data (as opposed to DNS requests, SSL negotiation, HTTP headers and HTTP2 frame metadata).

To actually complete a transfer you have to repeatedly invoke epoll_wait, curl_multi_perform and number of other functions until you are done. Curl's corresponding example stops after completing one transfer, but in practice it is more beneficial to create a permanently running thread, that handles all HTTP transfers for application's lifetime.