CPAN.pm takes ages to install modules

Question

cpan[1]> install Crypt::SSLeay
Reading '/root/.cpan/sources/authors/01mailrc.txt.gz'
............................................................................DONE
Reading '/root/.cpan/sources/modules/02packages.details.txt.gz'

Database was generated on Mon, 06 Aug 2012 09:19:03 GMT
............................................................................DONE

Reading '/root/.cpan/sources/modules/03modlist.data.gz'

................................................................

Running install for module 'Crypt::SSLeay'
Running make for N/NA/NANIS/Crypt-SSLeay-0.64.tar.gz
Fetching with LWP:
http://mirror.cogentco.com/pub/CPAN/authors/id/N/NA/NANIS/Crypt-SSLeay-0.64.tar.gz

Fetching with LWP:
http://mirror.cogentco.com/pub/CPAN/authors/id/N/NA/NANIS/CHECKSUMS

CPAN.pm takes like 15 minutes at these two steps. I have added more URLs to urllist . Most of them use HTTP.

Why is this taking so long and how can I make it faster?

Answer 1

To make CPAN indexes work faster, install CPAN::SQLite and and configure use_sqlite to 1 .

Alternatively, install App-cpanminus and run the cpanm client which does not need a local index.

Answer 2

If you do a lot of stuff with CPAN, configure and use a local MiniCPAN . Let it update automatically when you aren't paying attention to it, then install quickly when you're ready for it. The first time you set it up it will take forever, but it's a forever you don't have to care about because you don't have to watch it.

daxim recommended sqlite and cpanm , which you can use, but if your network is the slow part, you'll still have to use the network with either of those.

What you're seeing in that output snippet is your CPAN client downloading a couple of huge text files that are the indices into what is current on CPAN. The packages file has over 125,000 lines, but compressed its only about 1.5Mb. However, that file is updated hourly and CPAN.pm always wants the latest one. It's one of the huge design warts on the system, but that's what you get with the oldest and longest used CPAN client that has ever existed.

cpanm gets around that by not paying attention to the index files. It goes right to the CPAN search site to figure out what's the latest version and then goes with it. That might work better for you.

You said that you added more URLs. This is another problem for the CPAN client. Once it finds a URL that returns a good status, it goes with it, even if you've chosen something in the middle of Niger that has a 56k modem uplink through a laggy satellite where the operator is typing in the ones and zeroes by hand.

In my latest version of the cpan command line tool , which comes with CPAN.pm (but is usually a little behind, which is my fault), I've added a mirror pinging feature that will find the best mirrors for you based on ping times. That's only checking latency though, and throughput can still be a problem.

All of which is why I use a MiniCPAN .