How to write parallel programs in Perl?

Question

I have to do several tasks in one Perl script. Better not use fork or thread .

1. task A: make HTTP request to a server every 5 seconds. Infinity and should not be blocked. And if gets:

   1. 'TASK', task A will call a time consuming sub process, so how to avoid the assignment execution block task A's loop?
   2. 'PAUSE', pause the request
   3. 'UPDATE', tell task B to do something
   4. make POST request to server when receive data from task B or task C

2. task B: make mysql request every 1 minute, and depend on results, will tell task A to do POST request

3. task C: accept socket connection and tell task A or B to do something.

Three parallel infinitive loop processes, and will communication with each other. How can I do that?

Answer 1

That design makes no sense whatsoever, and the claim that it's better not to use threads or child processes makes even less sense.

You have three sources of requests:

• Request Source A: Make web request to a server every 5 seconds.
• Request Source B: Make database request to a server every 60 seconds.
• Request Source C: Accept requests from a socket.

Create a thread for each Request Source. Their job is solely to monitor each Request Source in order to ensure that the Sources are checked when they should be checked. As such, none of these threads should do any real work. If a task must be performed, they delegate the work to a worker thread. They don't POST anything. They don't write to the database.

The actual tasks (including sending POSTs and writing to the database) are performed by one or more worker threads (your choice). The worker threads receive requests from a singular Thread::Queue queue populated by the three Request Sources.

---

So the code would look like:

use threads;
use Thread::Queue qw( );

use constant NUM_WORKERS => 5;  # Tweak this. Can be as low as 1.

sub poll_web {
   my ($request_q) = @_;
   ... init ...
   while (1) {
      ...
      $request_q->enqueue([post => ...]);
      ...
   }
}

sub poll_db { ... }              # Just like poll_web

sub accept_connections { ... }   # Just like poll_web

sub post_handler { ... }         # Receives args passed to enqueue

{
   my $request_q = Thread::Queue->new();

   my %job_handlers = (
      post => \&post_handler,
      ...
   );

   for (1..NUM_WORKERS) {
      async {
         while (1) {
            my $job = $request_q->dequeue();
            my ($job_type, @args) = @$job;
            my $handler = $job_handlers{$job_type};
               or do { warn("Unknown job type $job_type"); next };
            $handler->(@args);
         }
      };
   }

   async { poll_web($request_q); };
   async { poll_db($request_q); };
   accept_connections($request_q);
}

---

If you want to use processes instead of threads, change

use threads;

to

use forks;

but keep using Thread::Queue.