Knit One, Perl 5.005

by Joseph N. Hall
<joseph@5sigma.com>

Joseph N. Hall is the author of Effective Perl Programming (Addison-Wesley, 1998). He teaches Perl classes, consults, and plays a lot of golf in his spare time.


This column refers to the most recent version available as of this writing, Perl 5.005_02. There may have been additional releases by the time you read this article, but you shouldn't expect any radical advances or changes in the immediate future.

There's a New Perl on the Block

The latest "major" revision of Perl, version 5.005, is now available. I'll discuss some of its new features, and some of the less stable experimental additions to the language.

Keep Your Feet Dry

Although the release of a major revision is exciting, you shouldn't immediately download Perl 5.005 and install it on your system. In a production environment where users depend on the reliable operation of a large set of Perl modules and programs, this would be a disaster. For one thing, Perl 5.005 is binary incompatible with previous versions of Perl. This means that any C- or C++-based modules that are installed in your Perl tree must be rebuilt for Perl 5.005. Some of those modules may not even compile against Perl 5.005 because the Perl API has changed; therefore, you may have to wait for new 5.005-compatible versions of those modules to be released. Another issue is that while maintainers have generally taken great pains to ensure backward compatibility with existing Perl scripts, incompatibilities due to changes and/or bugs no doubt do exist.

If you are running Perl on a machine on which you are the only user, or where it is not being used for critical applications, you can convert to Perl 5.005 by taking an inventory of what modules are installed on your system (perhaps with the help of perldoc perllocal or the CPAN module's autobundle command), building Perl 5.005, then reinstalling all of the modules that you want to keep.

On the other hand, in a production environment where a stable Perl is a critical part of daily operations, you will have to create an independent Perl 5.005 installation, reinstall the necessary modules there, then have your users test their applications against this separate installation. Applications that won't run under 5.005 can be fixed, or if that isn't practical, "hardwired" to use an earlier version of Perl kept around for backward compatibility. Only after everything is tested and stable -- a process that might take weeks or months -- should you make the newer version of Perl your new standard.

It's just a suggestion, but if I were contemplating moving a production environment to a newer version of Perl, I would wait a few months and hope for more maintenance releases of Perl 5.005 in the interim.

What about Threads?

Perl 5.005 was originally supposed to include threads support. To a certain extent, it does, but as of 5.005_02, threads are still classified as an experimental feature. I'd like to discuss threads programming in Perl, but the time is not yet right.

Okay, What about the Compiler?

Development on the Perl compiler (also known as B) continues. While the compiler itself is still considered an experimental feature, there are a number of compiler-related modules that are already useful to developers. It's also instructive to get a quick overview of the compiler.

The Perl compiler works by parsing Perl source code into an intermediate representation. Then, with a suitable "backend," it transforms the intermediate representation into something else. The CC backend, for example, creates optimized C code. The resulting C code must then be compiled with a C compiler to produce an executable program. For example, here's a short program, which we'll call cc-test.pl:

print "Welcome to compiled Perl!\n";
for $i (1..10) {
 print "$i times $i is ", $i * $i, "\n";
}

You can produce a corresponding C program, cc-test.c, with the somewhat unintuitive command:

% perl -MO=CC,-occ-test.c cc-test.pl

From there, you need to compile and link this C program. You can do it directly using the cc_harness program, but the process is simpler if you use another program called perlcc. perlcc takes care of the intervening steps automatically. For example:

% perlcc cc-test.pl

will first run the Perl compiler to produce C source code, then compile and link it into an executable called cc-test. When you run cc-test, it works just as if you were running the original Perl source in cc-test.pl.

This is nice as far as it works, but as the documentation states repeatedly, the Perl compiler is buggy and still experimental. However, there are some compiler-related features that you may find interesting. For example, there is another backend called Xref that can be used to produce a (somewhat cryptic) cross-reference listing of some of the contents of the Perl source:

% perl -MO=Xref cc-test.pl
cc-test.pl syntax OK
File cc-test.pl
	Subroutine (definitions)
		Package UNIVERSAL
		&VERSION			s0
		&can			s0
		&isa			s0
	Subroutine (main)
		Package main
			$i	5, 5, 5, 5
			*i	4

You might also find the Lint and Deparse backends useful. For more information about the compiler, use the perldoc command to read the documentation for B, B::CC, O, perlcc, and follow the references to other modules and programs as necessary.

Not So Regular Expressions

There have been quite a few internal changes to Perl regular expressions in Perl 5.005. Regular expressions are now more efficient, a number of bugs have been fixed, and the regular expression code in Perl itself has been somewhat cleaned up. There are a fair number of user-visible changes as well. First, there are some new regular expression atoms. For example, there are now atoms for pattern lookbehind:

@foos = /\bfoo.*?\b/g;	# find all words beginning with foo
@nofoos = /(?<=\bfoo).*?\b/g;	# find all words beginning with
	# foo, but leave off the foo

There are also atoms for "conditional" expressions, atoms that affect backtracking, and even atoms for executing arbitrary Perl code as a regular expression is matched.

Another addition is the re pragma module, which controls various aspects of regular expression behavior. One use for re is debugging output:

% perl -Mre=debug -e '/".*?"/'
compiling RE '".*?"'
size 8 first at 1
 1: EXACT <">(3)
 3: MINMOD(4)
 4: STAR(6)
 5: ANY(0)
 6: EXACT <">(8)
 8: END(0)

The qr// (regular expression quoting) feature greatly simplifies the efficient use of regular expressions that aren't known until run time. The contents of qr// are a regular expression "string" (the same as in an ordinary match operator m//). The result of qr// is a scalar that can be used standalone like a match operator, or incorporated efficiently into other regular expressions:

print "Field to match: ";	# For example, 'subject'
chomp($field = <>);
$re = qr/$field/i;	# /i means ignore case
print "Type in a line: ";	# For example, 'Subject: some
	# perl stuff'
print <> =~ /$re: (.*)/, "\n";	# prints 'some perl stuff'

The new features are documented in the perlre man page, the re module's documentation, and the qr// entry in the perlop man page.

Lots More New Stuff

There are many more small changes and improvements in Perl 5.005. Here are some of the more stable (nonexperimental) ones.

• foreach (1..1000000) is now optimized so that it does not construct a temporary list one million elements long.
• You can now use foreach as a statement modifier. For example:
  print "The number is $_\n" foreach (1..10);
  # prints 10 numbers
• The fields and base pragmas allow you to add more compile-time semantics and checking to your Perl classes.
• Perl is more tolerant of carriage returns in source files. DOS-formatted (CR-LF) Perl source used to cause fatal errors; this is no longer the case.
• Tied arrays and handles are better supported.

For More Information

For more about what's new in Perl 5.005, check the documentation. You can build Perl 5.005 and read the perldelta man page, or you can inspect it online at any CPAN mirror site, for example, <http://www.perl.com/CPAN-local/doc/manual/html/pod/perldelta.html>.