effective Perl programming

A Perl CPANorama

by Joseph N. Hall
<joseph@5sigma.com>

Joseph N. Hall is the author of Effective Perl Programming (Addison-Wesley, 1998). He teaches Perl classes, consults, and plays a lot of golf in his spare time.

One of the most obviously useful features of Perl and the Perl community is CPAN, the Comprehensive Perl Archive Network. CPAN is an archive of "all things Perl" that is replicated on FTP and HTTP servers all around the world. CPAN is a common repository for the Perl code base, documentation, and contributed software — and, notably, Perl modules that can be used to extend the capabilities of your Perl installation. The aptly named CPAN module greatly facilitates the process of building, testing, and installing Perl modules from CPAN.

In this column I'll discuss the process of installing modules from CPAN and how to use the CPAN module to automate the process.

Finding a CPAN

There are CPAN sites all around the world. Some provide access by FTP, others HTTP, and some both. An easy way to find a CPAN somewhere near you is via the CPAN multiplexer at <http://www.perl.com/CPAN>.

This will take you to some CPAN that the multiplexer thinks is "near" you. Once there, if you like, you can inspect the MIRRORED.BY file at the top level of the CPAN directory structure. It contains a list of the current official CPAN mirrors. If the mirror that the multiplexer found for you isn't ideal, pick a different one from the list of mirror sites.

Finding Modules Manually

The CPAN is a large and, at first glance (or perhaps always!), somewhat confusing archive. Aimlessly wandering up and down the CPAN directory tree may be entertaining, but such a tactic rarely succeeds as a means of finding a particular module or piece of documentation. The easiest way to begin is usually with the so-called long module list, found in modules/00modlist.long.html. The long module list contains a list of all of the modules currently in or formally proposed for CPAN. The list of modules is fairly long, and the information in it is coded for succinctness. Here is an example entry (this one is for my Sort::Fields module):

Sort::
::Fields bdpf sort text lines by alpha or numeric fields JNH

Sort::Fields is the name of the module (broken across two lines to show the hierarchy). JNH is the author's CPAN handle. (You can see a list of handles and the corresponding authors at <authors/00whois.html>.) bdpf is the so-called DSLI code for the module: Development Stage/Support Level/Language Used/Interface Style. In this case, the module is a (b)eta release, supported by the (d)eveloper, written in (p)erl only, with a (f)unction style interface.

Each module is linked into the CPAN several different ways. For example, Sort::Fields can be found in all the following places:

authors/Joseph_N_Hall/Sort-Fields-0.90.tar.gz
authors/id/JNH/Sort-Fields-0.90.tar.gz
modules/by-category/06_Data_Type_Utilities/Sort/
                                  Sort- Fields-0.90.tar.gz
modules/by-module/Sort/Sort-Fields-0.90.tar.gz

Once you have located a module that you want, it's time to install it.

Manually Installing a Module into Your Main Perl Tree

This section assumes that you have permission to modify the contents of your main Perl tree (that is, the place where Perl is installed on your system). If you don't have permission, read this section anyway, then proceed to "Installing a Module into a Nonstandard Place," below.

Because nearly all Perl modules have a standard organization, the process of building, testing, and installing modules is pretty much the same regardless of what the modules do. To install a Perl module, first obtain a copy of the compressed distribution (the .tar.gz file). Unpack the module into a "build" directory (which may be temporary, if you like), for example:

# mkdir /tmp/perlbuild
# cd /tmp/perlbuild
# gunzip Sort-Fields-0.90.tar.gz
# tar xvf Sort-Fields-0.90.tar
Sort-Fields-0.90/
Sort-Fields-0.90/Makefile.PL
Sort-Fields-0.90/Changes
Sort-Fields-0.90/test.pl
Sort-Fields-0.90/Fields.pm
Sort-Fields-0.90/README
Sort-Fields-0.90/MANIFEST

Move down into the newly created directory, then create a makefile for the module by running the Makefile.PL script:

# cd Sort-Fields-0.90
# perl Makefile.PL
Checking if your kit is complete...
Looks good

Writing Makefile for Sort::Fields

You have just created a makefile configured to install the module in the "standard" place in your Perl tree. You can now build and test the module:

# make
mkdir blib
mkdir blib/lib
(... etc.)
# make test
PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib
-I/usr/local/lib/perl5/5.00502/powerpc-machten
-I/usr/local/lib/perl5/5.00502 test.pl
1..38
ok 1
ok 2
(... etc.)

If the tests were successful, you now install the module:

# make install
Installing /usr/local/lib/perl5/site_perl/5.005/Sort/Fields.pm
Installing /usr/local/lib/perl5/5.00502/man/man3/Sort::Fields.3
(... etc.)

That's all there is to it! It's pretty simple, really.

Installing a Module into a Nonstandard Place

If you don't have permission to install a module in your system's Perl tree, or if you want to install one for your private use only, you have to alter the installation process. The module's makefile defines the location where the module and its supporting files are installed. To change that location, you need to change the makefile. You do this by providing some additional arguments to the MakeMaker script that creates the makefile. As an example, here's how I would create a makefile that installs a module into the directories /home/joseph/perllib and /home/joseph/perletc.

% perl Makefile.PL LIB=/home/joseph/perllib \
  PREFIX=/home/joseph/perletc \
  INSTALLMAN1DIR=/home/joseph/perletc/man/man1 \
  INSTALLMAN3DIR=/home/joseph/perletc/man/man3

The attribute LIB specifies where the module's Perl source code and shared objects (if applicable) will be rooted. The remaining attributes specify locations for the supporting files. (You don't have to specify INSTALLMAN1DIR in most cases, but I did so here for the sake of completeness.) After creating an appropriate makefile, build, test, and install the module as normal. The module will be installed in the location that you specified:

% make install
Installing /home/joseph/perllib/Sort/Fields.pm
Installing /home/joseph/perletc/man/man3/Sort::Fields.3
(... etc.)

Using a Module in a Nonstandard Place

Because the module in the example above was not installed into its "normal" place in your Perl tree, you have to tell Perl how to find it. The simplest way to do this is with the PERL5LIB environment variable. PERL5LIB is a colon-separated list of directories that should be searched for Perl libraries. For example:

% setenv PERL5LIB /home/joseph/perllib
% perl -MSort::Fields -e 'print "loaded successfully!\n"'
loaded successfully!

You can also use the -I command-line option:

% unsetenv PERL5LIB
% perl -I/home/joseph/perllib -MSort::Fields \
  -e 'print "loaded successfully!\n"'
loaded successfully!

Or the lib pragma:

% perl -Mlib=home/joseph/perllib -MSort::Fields \
  -e 'print "loaded successfully!\n"'
loaded successfully!

The lib pragma can be used within scripts (although this isn't particularly conducive to portability):

#!/usr/bin/perl
use lib '/home/joseph/perllib';
use Sort::Fields;

Building and Installing with the CPAN Module

The manual procedure for installing modules isn't that difficult, but, like everything else, it becomes tedious if repeated often enough. If things have gotten to that point for you, or if you just generally prefer to avoid busywork, you can use the CPAN module to almost completely automate the process of building, testing, and installing modules.

The CPAN module is generally used in its "shell" mode. Start it up with the following command:

# perl -MCPAN -e shell

If this is the first time you have used the CPAN module on this machine, you will get what I like to refer to as "the inquisition," a series of CPAN configuration questions. If you get the inquisition, answer the questions and stick to the defaults unless you have a reason to do otherwise. You will be asked to choose one or more CPAN sites. You should do this with some care — try to put a fast, high-availability CPAN site at the head of your list. In any event, at some point you will enter the CPAN shell and see a message similar to the following:

cpan shell -- CPAN exploration and modules installation (v1.48)
ReadLine support available (try "install Bundle::CPAN")
cpan>

You can type ? or help at the prompt for a list of CPAN shell commands. One of the most useful is the i command, used to obtain information about modules:

cpan> i /Fields/
Going to read /usr/local/CPAN/authors/01mailrc.txt.gz
Going to read /usr/local/CPAN/modules/02packages.details.txt.gz
(... etc.)
Distribution JNH/Sort-Fields-0.90.tar.gz
Module Sort::Fields (JNH/Sort-Fields-0.90.tar.gz)
Module fields (GSAR/perl5.005_02.tar.gz)
cpan>

Building and installing modules with the CPAN module is extremely straightforward. In most cases you can just use the install command:

cpan> install Sort::Fields

This will automatically download, make, and test the module. If it tests successfully, the module will be installed.

You don't have to use the CPAN shell if you don't want. You can use the CPAN module directly from the command line:

# perl -MCPAN -e 'install Sort::Fields'

This does the same thing as the CPAN shell install above.

Installing in Nonstandard Places with the CPAN Module

The CPAN module is perfectly capable of installing modules in nonstandard locations. The mechanism is the same as when you do a manual install — you must change the way module makefiles are built — but to do this with the CPAN module, you have to change the configuration file that the CPAN module uses to issue the command that creates the makefile. There are a couple of ways to go about this, and because of the vagaries of Perl installations and changes in the CPAN module of late, it's impossible for me to say exactly what will be the best procedure for you. However, I'll outline some general approaches.

The CPAN module allows for user-specific private configuration files. These have to be at a specific location — this is hardwired by the module:

$HOME/.cpan/CPAN/MyConfig.pm

where $HOME is the user's home directory. You can create your own MyConfig.pm by copying it from your Perl tree:

% cp /usr/local/lib/perl5/5.00502/CPAN/Config.pm \
~/.cpan/CPAN/MyConfig.pm

You can make all the changes you need by editing this file, or you can make some later by using the CPAN shell's o conf command (see below). However, you will have to make at least one change before running the CPAN shell. The cpan_home parameter in MyConfig.pm needs to be changed to point to your home directory. If it isn't changed, the CPAN module will probably not be able to create the lockfile that it depends on, and the module will fail to start up. To make the change, open the file and look for the line defining the cpan_home parameter:

'cpan_home' => q[/usr/local/build/cpan],

Change this to whatever is appropriate, for example:

'cpan_home' => q[/home/joseph/.cpan],

You also need to make some other parameter changes. The build_dir and keep_source_where directories should be changed to point to your local .cpan directory as well:

'build_dir' => q[/home/joseph/.cpan/build],
'keep_source_where' => q[/home/joseph/.cpan/sources],

To build makefiles that install in nonstandard locations, you also need to change the makepl_arg parameter:

'makepl_arg' => q[LIB=/home/joseph/perllib \
PREFIX=/home/joseph/perletc \
INSTALLMAN1DIR=/home/joseph/perletc/man/man1 \
INSTALLMAN3DIR=/home/joseph/perletc/man/man3],

After you have made these changes, save the file and use the CPAN module as usual. Your modules will now build and install — automatically! — in the locations you have specified.

If you want to change parameter values without hacking on the MyConfig.pm file itself, you can use the o conf command within the CPAN shell. Use the o conf commit command to save your changes:

cpan> o conf urllist
urllist [q[file:///usr/local/CPAN/]]

cpan> o conf urllist push ftp://ftp.perl.org/pub/CPAN/

cpan> o conf urllist
urllist [q[file:///usr/local/CPAN/],
                                  q[ftp://ftp.perl.org/pub/CPAN/]]

cpan> o conf commit
commit: wrote /home/joseph/.cpan/CPAN/MyConfig.pm

What's Next?

That's a quick overview of the basics of using the CPAN and the CPAN module. For more information about CPAN, read the various files referred to in the CPAN index.html. For more about the CPAN module and the process of making and installing Perl modules, read perldoc CPAN and (shudder) perldoc ExtUtils::MakeMaker.

In the next column I will show you how to do some neat (and timesaving!) tricks with the CPAN module. Until then, keep reading, coding, and honing your Perl skills!