从同一网站的多个页面中提取内容

Question

I have this script to extract data from multiple pages of the same website. There are some 120 pages.

Here is the code I'm using to get for a single page.

$html = file_get_contents('https://www.example.com/product?page=1');

$dom = new DOMDocument;

@$dom->loadHTML($html);

$links = $dom->getElementsByTagName('div');

foreach ($links as $link){
    file_put_contents('products.txt', $link->getAttribute('data-product-name') .PHP_EOL, FILE_APPEND);
}

How can I do it for multiple pages? The links for that specific pages are incremental like the next page will be https://www.example.com/product?page=2 and so on. How can I do it without creating different files for each link?

Answer 1

What about this :

function extractContent($page)
{
    $html = file_get_contents('https://www.example.com/product?page='.$page);
    $dom = new DOMDocument;
    @$dom->loadHTML($html);
    $links = $dom->getElementsByTagName('div');

    foreach ($links as $link) {
        // skip empty attributes
        if (empty($link->getAttribute('data-product-name'))) {
            continue;
        }
        file_put_contents('products.txt', $link->getAttribute('data-product-name') .PHP_EOL, FILE_APPEND);
    }
}

for ($i=1; $i<=120; $i++) {
    extractContent($i);
}