简体   繁体   中英

Perl Parallel::ForkManager , takes long time with fork condition changed

I have two questions with Parallel::ForkManager , in modifing the pid condition.

Question 1 : with a simple example,

use strict;
use warnings;
use Parallel::ForkManager;
use IO::Socket ;

my $np = 32 ;
my $pm = Parallel::ForkManager->new($np);
$| = 1 ;

sub do_something{ #an example for 3 second delay
    my $port = shift @_ ;
    my $tgt = 192.168.0.1 ;
    my $sock = IO::Socket::INET->new(PeerAddr=>"$tgt:$port",Proto='tcp',Timeout=>3) ;
    $sock ? return $sock : return 0 ;
}

foreach (1..64) {
    $pm->start and next ;
    do_something($_) ;
    $pm->finish ;
}
$pm->wait_all_children ;

The above script works , with

time perl simple_script.pl 

real    0m7.109s
user    0m0.237s
sys     0m0.080s

while modifying the

$pm->start and next ; 
do_something($_) ;

to

$pm->start and next or do_something($_) ;

or

!($pm->start) or do_something($_) ; 

does seem to work also but not as intended.

time perl modified_script.pl 

real     3m12.307s
user     0m0.237s
sys      0m0.080s

seems like the other two modified version run a single process at a time.

Question 2:

Is there any alternate way to do the same without using the next inside the foreach loop? That way the script can be adapted for map instead of a foreach loop.

PS: Forking and Object creation are not my strong points.

Edit: removed & as per suggestion .

If you change this loop

foreach (1..64) {
  $pm->start and next;
  do_something($_);
  $pm->finish;
}

to this

foreach (1..64) {
  !($pm->start) or do_something($_);
}

then you have removed the $pm->finish call which terminates each child. That means the child processes will continue to execute the for loop, trying to start their own children, which will fail (because the child processes need their own Parallel::ForkManager object - they cannot use the parent's) and so they will do_something each time around the loop. Furthermore, because the children are delayed while they execute the subroutine many times, it will take much longer to get all 64 children started because you have a limit of 32 running at any one time.

You could write

for ( 1 .. 64 ) {
  my $pid = $pm->start;
  if ( $pid == 0 ) {
    do_something($_);
    $pm->finish;
  }
}

$pm->wait_all_children;

but as I said in my comment, if this is just so that you can misuse map instead of using a for then please don't do it

The changes you claimed you made would not cause the process to take longer. As such, I'm going to concentrate on your question: How to convert foreach loop into a map loop.

for (LIST) BLOCK can be written as map BLOCK LIST . The catch is that you can't use next since each pass of the loop is expected to return something (if only the empty list).

my $pm = ...;
map {
   if (!$pm->start) {
      do_something($_);
      $pm->finish;
   }
   ()
} 1..64;
$pm->wait_all_children();

or

sub child {
   my ($job) = @_;
   ...
   exit();
}

{
   my $pm = ...;
   map { $pm->start or child($_) } 1..64;
   $pm->wait_all_children();
}

I have no idea why you'd want to do that. It just makes the code harder to read.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM