如何使用cURL（C / C ++）从网页中获取特定行数

Question

I am a newbie in cURL and trying to implement some application, which could allow user to fetch specific data from an HTML page (dynamic) and save it to .txt

Application is c/c++ based and so far i am able to fetch the whole contant of HTML page.

This is the code i am refering:-

#include "stdafx.h" 
#pragma comment(lib, "curllib_static.lib") 
#include "curl/curl.h" 
#pragma comment(lib, "wldap32.lib") 
#pragma comment(lib, "ws2_32.lib") 
#pragma comment(lib, "winmm.lib")
#pragma comment(lib, "ssleay32.lib") 
#pragma comment(lib, "openldap.lib") 
#pragma comment(lib, "libeay32.lib")

void get_page(const char* url, const char* file_name)
{
  CURL* easyhandle = curl_easy_init();
 // time = 100;
  curl_easy_setopt( easyhandle, CURLOPT_URL, url ) ;

  curl_easy_setopt (easyhandle, CURLOPT_CONNECTTIMEOUT, .29);

  FILE* file = fopen( "my.txt", "a+");

  curl_easy_setopt( easyhandle, CURLOPT_WRITEDATA, file) ;
//  curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);

  curl_easy_perform( easyhandle );

  curl_easy_cleanup( easyhandle );
  fclose(file);

}

int main()
{
  get_page( "http:couldbeanything.com", "style.css" ) ;

  return 0;
}

So, this code fetches whole page and i just want to fetch some specific number of lines using it (for example - 5)

I searched and came across something called "PHP dom parser" and is there some way to implement this fetching in C/C++ ??

Thanks in advance

Answer 1

It's an unusual requirement and no DOM parser is going to help you. Instead you should use some slightly more advanced curl options, instead of using CURLOPT_WRITEDATA use CURLOPT_WRITEFUNCTION . Like this

curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, my_function);

...

size_t my_function(char *ptr, size_t size, size_t nmemb, void *userdata)
{
    ...
}

my_function is a callback, when some data is available from the webpage then my_function will be called by curl with ptr pointing to the available data, size telling you how many data elements are available, and nmemb telling you the size of each data element. You can then do what you want with this data. Presumably in your case this would mean extracting the first few lines.

But there is no guarantee that you will get the first few lines in a nice convenient block (the internet doesn't work like that) so you have some work to do. Check the docs for more information.

Basically since the internet isn't 'line based' there's no simple way to do what you want and I wonder if you should rethink your requirements.