how to fix or optimize code to ignore memory size error in PHP

Question

I wrote a image crawler with simple_html_dom library, I used this code to fetch all images in a web site ;

include 'simple_html_dom.php';
$img_array = array();
if (isset($_POST['url'])) {
    $url = $_POST['url'];
    $html = file_get_html($url);
    echo $html->getElementByTagName('title')->innertext();
    foreach ($html->find('a') as $a) {

        if (strpos($a->href, $url) !== FALSE) // only images from this site
        {
            //
            //    find_images($a->href);
            $imgs = file_get_html($a->href);
            foreach ($imgs->find('img') as $img) {
                if(!in_array($img->src, $img_array))
                {
                    echo '<img src="' .$img->src. '" class="thumb">';
                    $img_array[] = $img->src;
                }
            }
            echo '<hr>';
        }
    }
}

but whene i execute this code i get Fatal error: Allowed memory size of 209715200 bytes exhausted (tried to allocate 71 bytes) in /home/iphotosh/public_html/test/simple_html_dom.php on line 122

test and demo : test.iphotoshop.ir

how to fix this error or how to opti,ize this code for fetch all images from a web site?

Answer 1

您是否尝试使用ini_set（）增加内存：

ini_set("memory_limit","256M");

Answer 2

You should do two things at the same time: Set your memory limit very high:

in php.ini:

memory_limit = 512M

or/and in php file:

ini_set("memory_limit","512M");

At the same time you should delete big variables to free up some memory, usually via:

unset($var);

By the way you can check the amount of used memory via

echo memory_get_usage();

I would try a demo run and check for memory usage in EACH line of your code, so you can see what happening here, and where to start your fixing

Answer 3

Seems like you're trying to allocate way too much memory. You could try increasing available memory in your php.ini (look for memory_limit= directive). Yet, you could still exceed it if you are allocating A LOT. You can check dynamically what's available and how much is used:

function get_available_memory() {
    $ini_mem = ini_get('memory_limit');
    $m = substr($ini_mem, strlen($ini_mem) - 1;
    if($m == 'k' || $m == 'K') {
        $max_mem = 1024 * substr($ini_mem, 0, strlen($ini_mem) - 1);
    }
    elseif($m == 'm' || $m == 'M') {
        $max_mem = 1024 * 1024 * substr($ini_mem, 0, strlen($ini_mem) - 1);
    }
    elseif($m == 'g' || $m == 'M') {
        $max_mem = 1024 * 1024 * 1024 * substr($ini_mem, 0, strlen($ini_mem) - 1);
    }
    else {
        $max_mem = $ini_mem;
    }

    $used_mem = memory_get_usage(true);

    return $max_mem - $used_mem;
}

Now you can do

$available_memory = get_available_memory();

and, if not enough is available, do not try to allocate it and gracefully close your process.