How to extract data from Parallel::ForkManager in perl

Question

I am using google just for explaining my issue.
I have calculated Latency time for search request on google and i am printing it.
Now, i need data like max time, min time, average time. So i am trying to push data to array which can be manipulated later on. Array is coming out to be blank. Seems this is limitation of parallel-forkmanager. Please suggest some work around.

My code:

use Parallel::ForkManager;
use WWW::Mechanize;
use LWP::UserAgent;
use Time::HiRes qw/gettimeofday/;
use Time::Format qw/%time/;
use POSIX       qw( strftime );
use Time::HiRes qw( gettimeofday );
$count=5;
@arr=();
$pm = new Parallel::ForkManager($count);   
for(1..$count)  
{     
  $pm->start and next;      
  $m = WWW::Mechanize->new();
  $m->get( "http://www.google.com" );
  $s1=gettimeofday;
  my ($secs, $microsecs) = gettimeofday();
  print strftime("%H:%M:%S", localtime($secs)) . sprintf(".%04d",

$microsecs/10);
  $m->submit_form(
  form_number => 1,
  fields    => {q=>'abcd'},
  );

  print " ";
  print $m->title;
  print " ";
  $s2=gettimeofday;
  my ($secs, $microsecs) = gettimeofday();
  print strftime("%H:%M:%S", localtime($secs)) . sprintf(".%04d",   

$microsecs/10);
  $s3=$s2-$s1;
  $s3=$s3*1000;
  print "  $s3\n";
  push(@arr,$s3);
  $pm->finish   }  

$pm->wait_all_children; ## wait for the child processes
foreach(@arr)
{
print "$_\n";
}

Answer 1

You need to add run_on_finish callback to the manager instance аnd call finish method with 2 arguments into the child. So you can get second parameter of the finish call as sixth argument of the run_on_finish callback.

# parent
my @arr;
$pm->run_on_finish(sub {
    push(@arr, ${$_[5]});
});

# child
my $val = 42;
$pm->finish(0, \$val);

# parent
$pm->wait_all_children();
print($_, "\n") for @arr;

Answer 2

It's not so much a limitation of Parallel::ForkManager as a general limit of parallel code. fork spawns a new process that is a copy of the existing one. It has it's own memory space, and anything that changes between child and parent is ... well, changed.

Threading has much the same problem.

There's quite a big chapter on how to work with interprocess communications - perlipc

Parallel::ForkManager in particular has a couple of mechanisms for returning data to the parent process. Personally - I don't really get along iwth it, because whilst it works, it's also trying to be rather a general solution to the problem.

I'd be thinking rather something like using pipes to transfer the information back to the parent process.

use strict;
use warnings;
use Data::Dumper;

use Parallel::ForkManager;
use IO::Pipe;

my $pm = Parallel::ForkManager->new(2);

for ( 1 .. 4 ) {
    my $pipe = IO::Pipe->new();
    my $pid  = $pm->start;
    if ($pid) {

        #parent
        $pipe->reader->autoflush;
        while (<$pipe>) {
            print "Got data from $pid: $_\n";
        }
        close ( $pipe );
    }
    else {
        #is child
        $pipe->writer->autoflush;
        print {$pipe} "Child $$ says hello!";
        close($pipe);
        print "Child $$ exiting\n";
    }
    $pm->finish;
}