ActiveState Perl crashes on IIS under heavy load

Question

I have a problem running ActiveState Perl 5.16.2 (32-bit) on a Windows 2008 Server (64-bit) with IIS 7.5. Perl is run using PerlIS.dll with an ISAPI filter in IIS.

There seems to be a problem related with heavy load and/or concurrent connections that generate seemingly random error messages.

Here are a few sample errors:

*** 'C:\inetmli\wwwroot\ldope\index.cgi' error message at: 2013/02/28 13:01:07
Variable "%Config" is not imported at C:/Perl/lib/Config.pm line 53.
Variable "%Config" is not imported at C:/Perl/lib/Config.pm line 94.
Global symbol "%Config" requires explicit package name at C:/Perl/lib/Config.pm line 53.
Global symbol "%Config" requires explicit package name at C:/Perl/lib/Config.pm line 94.
Compilation failed in require at C:/Perl/lib/Errno.pm line 8.
BEGIN failed--compilation aborted at C:/Perl/lib/Errno.pm line 8.
Compilation failed in require at C:/Perl/lib/IO/Socket.pm line 18.
BEGIN failed--compilation aborted at C:/Perl/lib/IO/Socket.pm line 18.
Compilation failed in require at C:/Perl/site/lib/Net/LDAP.pm line 9.
BEGIN failed--compilation aborted at C:/Perl/site/lib/Net/LDAP.pm line 9.
Compilation failed in require at C:\inetmli\wwwroot\test\index.cgi line 4.
BEGIN failed--compilation aborted at C:\inetmli\wwwroot\test\index.cgi line 4.

*** 'C:\inetmli\wwwroot\ldope\index.cgi' error message at: 2013/02/28 13:01:07
String found where operator expected at (eval 4) line 2, near "croak 'usage: $io->getline()'"
    (Do you need to predeclare croak?)
String found where operator expected at (eval 4) line 8, near "croak 'usage: $io->getlines()'"
    (Do you need to predeclare croak?)
String found where operator expected at (eval 4) line 10, near "croak 'Can\'t call $io->getlines in a scalar context, use $io->getline'"
    (Do you need to predeclare croak?)
syntax error at (eval 4) line 2, near "croak 'usage: $io->getline()'"
syntax error at (eval 4) line 8, near "croak 'usage: $io->getlines()'"
syntax error at (eval 4) line 10, near "croak 'Can\'t call $io->getlines in a scalar context, use $io->getline'"
Compilation failed in require at C:/Perl/lib/IO/Socket.pm line 12.
BEGIN failed--compilation aborted at C:/Perl/lib/IO/Socket.pm line 12.
Compilation failed in require at C:/Perl/site/lib/Net/LDAP.pm line 9.
BEGIN failed--compilation aborted at C:/Perl/site/lib/Net/LDAP.pm line 9.
Compilation failed in require at C:\inetmli\wwwroot\test\index.cgi line 4.
BEGIN failed--compilation aborted at C:\inetmli\wwwroot\test\index.cgi line 4.

*** 'C:\inetmli\wwwroot\ldope\index.cgi' error message at: 2013/02/28 13:01:07
Can't locate loadable object for module main in @INC (@INC contains: C:/Perl/site/lib C:/Perl/lib .) at C:/Perl/lib/DynaLoader.pm line 99, <DATA> line 635.
    DynaLoader::croak('Can\'t locate loadable object for module main in @INC (@INC c...') called at C:/Perl/lib/DynaLoader.pm line 161
    DynaLoader::bootstrap('main') called at C:/Perl/lib/DynaLoader.pm line 106
    DynaLoader::bootstrap_inherit('main') called at C:/Perl/lib/POSIX.pm line 0
    require POSIX.pm called at C:/Perl/lib/POSIX.pm line 0
    PDF::API2::Util::BEGIN() called at C:/Perl/lib/POSIX.pm line 0
    eval {...} called at C:/Perl/lib/POSIX.pm line 0
    require PDF/API2/Util.pm called at C:/Perl/lib/POSIX.pm line 0
    PDF::API2::BEGIN() called at C:/Perl/lib/POSIX.pm line 0
    eval {...} called at C:/Perl/lib/POSIX.pm line 0
    require PDF/API2.pm called at C:\inetmli\wwwroot\test\index.cgi line 13
Compilation failed in require at C:/Perl/site/lib/PDF/API2/Util.pm line 31, <DATA> line 635.
BEGIN failed--compilation aborted at C:/Perl/site/lib/PDF/API2/Util.pm line 31, <DATA> line 635.
Compilation failed in require at C:/Perl/site/lib/PDF/API2.pm line 9, <DATA> line 635.
BEGIN failed--compilation aborted at C:/Perl/site/lib/PDF/API2.pm line 9, <DATA> line 635.
Compilation failed in require at C:\inetmli\wwwroot\test\index.cgi line 13, <DATA> line 635.

The script being executed is a very simple test script using many libraries:

use strict;
use Net::LDAP;
require Archive::Zip;
require CAM::PDF;
require DBI;
require Excel::Writer::XLSX;
require HTTP::Request;
require LWP::UserAgent;
require MIME::Base64;
require MIME::QuotedPrint;
require PDF::API2;
require PHP::Serialization;
require Socket;
require Win32::Process;

print "HTTP/1.1 200 OK\nContent-Type: text/plain\n\nI'm OK!";

When requesting this script multiple times I can generate the error quite often, but there is no apparent pattern to when or what causes it. I even managed to get the error using just the single last line as a script:

print "HTTP/1.1 200 OK\nContent-Type: text/plain\n\nI'm OK!";

The error from that was:

*** 'C:\inetmli\wwwroot\ldope\index.cgi' error message at: 2013/02/28 12:26:25
syntax error at C:\inetmli\wwwroot\test\index.cgi line 1, near "K!"
Can't find string terminator '"' anywhere before EOF at C:\inetmli\wwwroot\test\index.cgi line 26, <DATA> line 635.

I must run the script thousands of time from multiple threads from my PC to generate this error and it only triggers a few times. I believe it to be triggered by concurrent access.

From that error it seems it did not read the whole file when it started to parse it?

This bug is annoying me to insanity! On a heavy load server it generates errors quite often and sometimes the whole application pool in IIS crashes and has to be restarted.

Has anyone seen this before? Is there a know work around? Preferably I would like to stay with ISAPI since the other options (PerlEX and FastCGI is untested with the current code).

I would be very thankfull for any help or tip to help me solve my problem. Since I haven't found anything on the web maybe it could also help someone else who is struggeling with these seemingly random error messages.

So far I've tried:

• splitting applications up in different application pools (only positive effect is that a crashed application now is limited to the current application pool)
• adding multiple worker processes to a single application pool (made the errors rarer but not dissapear)
• setting processor affinity to true
• setting application pool recycling both very high and very low to no apparent effect
• using all kinds of "classic"/"pipeline" and ASP.NET settings despite knowing they probably have nothing to do with it
• Perl 5.14 does NOT have this problem, only Perl 5.16

Anyone got any more ideas?

Answer 1

就我而言（MacOS），当线程数大于120时，STDOUT-> flush（）中会随机出现错误。我在线程环境中将其删除，问题消失了。