perl File::Tail not reading lines from file after certain period

Question

I am having issues understanding why File::Tail FAILS to read the lines from an always updating file with 1000s of transactions, and automatic rollover .

It read to a certain extend correctly, but then later slows down, and for a long period even not able to read the lines in the logs. I can confirm that the actual log are being populated as when file::tail shows nothing.

my $file = File::Tail->new(name=>"$name",
                           tail => 1000,
                           maxinterval=>1,
                           interval=>1,
                           adjustafter=>5,resetafter=>1, 
                           ignore_nonexistant=>1,
                           maxbuf=>32768);

while (defined(my $line=$file->read)) {
    #my $line=$file->read;
    my $xml_string  = "";

    #### Read only one event per line and deal with the XML.
    #### If the log entry is not a SLIM log, I will ignore it.
    if ($line =~ /(\<slim\-log\>.*\<\/slim\-log\>)/) {
        do_something
    } else {
        not_working for some reason
    }

Can someone please help me understand this. Know that this log file is updated at almost 10MB per second or 1000 events per second for an approximation.

Should I be handing the filehandle or the File::Tail results some other more efficient way?

Answer 1

Seems like there's limitations in File::Tail. There's some suggestions around other more direct options (a pipe, a fork, a thread, seeking to the end of the file in perl) discussed in http://www.perlmonks.org/?node_id=387706 .

My favorite pick is the blocking read from a pipe:

open(TAIL, "tail -F $name|") or die "TAIL : $!";
while (<TAIL>) {
        test_and_do_something
}