How to read compressed files c++

Question

My Question is closed so I have to update this.

1- My purpose is to send my compressed file google cloud storage URL.

2- To do that I have generated a postman request. I have stored my file to my google cloud storage by using the postman tool and the tool has generated the following code.

CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
  curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
  curl_easy_setopt(curl, CURLOPT_URL, "my URL");
  curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
  curl_easy_setopt(curl, CURLOPT_DEFAULT_PROTOCOL, "https");
  struct curl_slist *headers = NULL;
  headers = curl_slist_append(headers, "Content-Type: 
      application/octet-stream");
  curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
  curl_easy_setopt(curl,CURLOPT_POSTFIELDS,"<file contents here>");
  res = curl_easy_perform(curl);
}
curl_easy_cleanup(curl);

3- Then I have copied the code above into my c++ project to send my compressed file to the URL.

4- To create CURLOPT_POSTFIELDS content I did implement the following code;

std::ifstream ifs;
ifs.open ("./compressed.gz", std::ios::binary | 
    std::ios::ate);

PRINT("Size ->", ifs.tellg());

curl_easy_setopt(curl, CURLOPT_POSTFIELDS, &ifs);

And when I compile and run my code, the request returns me the 200 success response. But when I checked the google storage dashboard, it just contains 6 bytes of data. Actually, the size of my ifstream data is 1090 .

So my problem is that why my request uploads all bytes of the compressed file to cloud storage? Whats wrong in my code?

Answer 1

How to read compressed files c++

Compressed files are generally binary formats, they are not null terminated text. You cannot use strlen to get their length because that requires a null terminated text as input.

You can use any UnformattedInputFunction to read binary data. Don't forget to open any stream in binary mode.

I need to put the compressed file in a char pointer to put it to the server by using libcurl.

You don't need to read the file in order to do that. You can let libcurl take care of reading the file by passing a FILE* to CURLOPT_READDATA.

This way it won't be necessary to store the entire file in memory. Reading the entire file could be a problem if the file is very large.

Answer 2

You can do something like that when you want to read binary data:

std::ifstream file("./compressed.gz", std::ios::binary);
if (!file.is_open())
    ERROR();

std::vector<unsigned char> buffer(std::istreambuf_iterator<char>(file), {});

Then you will have it in the vector buffer . Access the size via buffer.size() and to get raw data use buffer.data() Note that since buffer is an std::vector if it goes outside of scope it will be destructed so the data will be deleted.