如何从内存中删除Perl对象

Question

I'm having some issues with the memory usage of a perl script I wrote (code below). The script initiates some variables, fills them with data, and then undefines them again. However, the memory usage of the script after deleting everything is still way to high to contain no data.

Accoring to ps the script uses 1.027 Mb memory (RSS) during the first 39 seconds (so everything before the foreach loop). Then, memory usage starts rising and ends up fluctuating between 204.391 Mb and 172.410 Mb. However, even in the last 10 seconds of the script (where all data is supposed to be removed), memory usage never goes below 172.410 Mb.

Is there a way to permanently delete a variable and all data in it in perl (in order to reduce the memory usage of the script)? If so, how should I do it?

use strict;
use warnings;

sleep(30);

my $ELEMENTS = 1_000_000;
my $MAX_ELEMENT = 1_000_000_000;
my $if_condition = 1;

sleep(5);

my %hash = (1 => {}, 2 => {}, 3 => {}, 4 => {});

foreach my $key (keys %hash){
    if( $if_condition ){
        my $arrref1 = [ (rand($MAX_ELEMENT)) x $ELEMENTS ];
        my $arrref2 = [ (rand($MAX_ELEMENT)) x $ELEMENTS ];
        my $arrref3 = [ (rand($MAX_ELEMENT)) x $ELEMENTS ];

        sleep(2);

        if(!defined($hash{$key}->{'amplification'})){
            $hash{$key}->{'amplification'} = [];
        }

        push(@{$hash{$key}->{'amplification'}},@{$arrref1});
        undef($arrref1);
        push(@{$hash{$key}->{'amplification'}},@{$arrref2});
        undef($arrref2);
        push(@{$hash{$key}->{'amplification'}},@{$arrref3});
        undef($arrref3);


        sleep(3);

        delete($hash{$key});

        sleep(5);
    }
}

sleep(10);

Answer 1

Perl FAQ 3 - How can I free an array or hash so my program shrinks?

You usually can't. Memory allocated to lexicals (ie my() variables) cannot be reclaimed or reused even if they go out of scope. It is reserved in case the variables come back into scope. Memory allocated to global variables can be reused (within your program) by using undef() and/or delete().

On most operating systems, memory allocated to a program can never be returned to the system. That's why long-running programs sometimes re- exec themselves. Some operating systems (notably, systems that use mmap(2) for allocating large chunks of memory) can reclaim memory that is no longer used, but on such systems, perl must be configured and compiled to use the OS's malloc, not perl's.

In general, memory allocation and de-allocation isn't something you can or should be worrying about much in Perl.

See also "How can I make my Perl program take less memory?"

Answer 2

In general, perl won't release memory back to the system. It keeps its own pool of memory in case it is required for another purpose. This happens a lot because lexical data is often used in a loop, for instance your $arrref1 variables refer to a million-element array. If the memory for those arrays was returned to the system and reallocated every time around the loop there would be an enormous speed penalty

As I wrote, 170MB isn't a lot, but you can reduce the footprint by dropping your big temporary arrays and adding the list directly to the hash element. As it stands you are unnecessarily keeping two copies of each array

It would look like this

use strict;
use warnings 'all';

sleep 30;

use constant ELEMENTS    => 1_000_000;
use constant MAX_ELEMENT => 1_000_000_000;

my $if_condition = 1;

sleep 5;

my %hash = ( 1 => {}, 2 => {}, 3 => {}, 4 => {} );

foreach my $key ( keys %hash ) {

    next unless $if_condition;


    sleep 2;

    push @{ $hash{$key}{amplification} }, (rand MAX_ELEMENT) x ELEMENTS;
    push @{ $hash{$key}{amplification} }, (rand MAX_ELEMENT) x ELEMENTS;
    push @{ $hash{$key}{amplification} }, (rand MAX_ELEMENT) x ELEMENTS;


    sleep 3;

    delete $hash{$key};

    sleep 5;
}

sleep 10;