source code UNIX for PCs

by Robert Gray
<bob@cs.colorado.edu>

Bob Gray is a consultant in the digital video industry. Designing architectures for performance has been his focus since he built an image processing system on UNIX in the late 1970s. He has a PhD in computer science from the University of Colorado.

The More Things Change,
the More They Remain the Same

The late 1970s were exciting times for computer technologists because educational institutions and government organizations had source code for AT&T's UNIX for PDP-11 computers. These were the first popular computers to break out of the centralized data-processing centers into the hands of end users in significant numbers.

The 32-bit VAX super minicomputer had just been introduced, and the Computer Systems Research Group at Berkeley, in cooperation with the ARPANET community, was defining, implementing, and releasing a UNIX operating system with rich, new capabilities. Lots of people contributed their time and software to the effort, resulting in VAX/UNIX being the platform of choice for many research facilities in the world.

A major change in computing took place around 1983. Up until that time, most computing had been time-shared computing ­ the length of time that your program took to run depended upon how many other people were using the system at the same time. People often worked during the "off hours" to avoid the slow system performance caused by the many users on time sharing systems. Then Sun introduced workstations ­ hot new platforms with a 1000x1000 bit map display, 1MB memory, and 1 MIPS processor dedicated to a person on the desktop and attached to a network. This was the first time we could count on getting the whole computer dedicated to our work every time we used it.

Sun's operating system, SunOS, was based on the source code developed by Berkeley and the ARPANET community, but few users had access to it. (If educational institutions jumped through enough hoops or a commercial institution paid zillions to AT&T, they could get access to parts of SunOS.) No longer could we dig in to see what was happening, fix things, or make local customizations. One consolation to computer users was that throughout the 1980s, computers steadily increased in performance with the introduction of new Sun, DEC, SGI, and other workstations. But on these systems, source code was still not generally available.

By the early 1990s, the PCs started offering performance comparable to workstations. I, for one, believed that the x86 architecture would be eclipsed by the RISC CPUs, but Intel proved me wrong. The cost advantage of a PC balanced out its ugly packaging and clunky integration. But PCs were stuck running primitive DOS software. Who would want to run a system that crashed frequently and didn't offer memory protection, multitasking, or virtual memory? It was looking bleak for progress.

A key enabling event happened in the early 1990s. The folks at Berkeley prepared and released an unencumbered source code UNIX to the public. AT&T (and its UNIX successors, USL and Novell) claimed that the code could not be publicly released because it contained material originating from AT&T. It turned into a multi-year legal battle, finally resulting in the decision that the fruits of 20 years of cooperative Internet (ARPANET) work, now known as 4.4BSD, could be released to the public. The importance of this event to the source code UNIX community is huge. None of BSDI, FreeBSD, NetBSD, or OpenBSD would have been possible if Berkeley and friends hadn't made the effort to get their work released.

We've come full circle with source code UNIX; now ­ like 20 years ago ­ you can have the full-featured operating system, complete with source code for all of it. But today you don't have to timeshare the system; you can have source code UNIX on your own inexpensive computer.

Over a number of months, this column will explore the benefits of running source code UNIX on a commodity PC computer. When concrete examples are necessary, I will often draw from my experience with FreeBSD, but most of my content will apply (often exactly) to other versions such as BSDI, OpenBSD, Linux and NetBSD. I will not debate which version is "best," because all versions excel in most areas and it is not worth arguing over trivial details. Upcoming columns will deal with choosing hardware, performance of the I/O system, building a Web server, building a firewall, publicly available[1] software and customizing source code UNIX for embedded products. I am most willing to listen to feedback and suggestions, but please, no comments on which version is best.

Why Would You Want To Set Up A Source Code UNIX PC?

I imagine a number of the readers are quite content with their traditional UNIX workstations. You probably have a Sun SPARC running Solaris, an HP with HPUX, an alpha with Digital UNIX, an SGI with IRIX, or an IBM with AIX, etc. All of these are completely functional systems; however, they all lack source code. Do you have custom hardware for which you need an operating system? Are you curious how a hardware device works? Do you want to look at the virtual memory system? Do you frequently wish you could slightly change the way some of your vendor-supplied applications work? Do you need to fix something that your vendor won't get to for six months? Do you need to write an application that could be leveraged from an existing application? Does the documentation hint of a possible method to solve your problem, but the details are missing? If you could only glance at the code, the solution might be clear. Well, you're generally out of luck with the above vendor combinations.

Source code UNIX offered me the most expeditious method to deliver a sophisticated, feature-embellished product based on commodity PC hardware. At my former employer, I was in charge of designing and implementing the software for a realtime product that among other things, played and recorded digital video at 30MB/s to our RAID-3 subsystem. The box consisted of a Pentium single-board computer, a custom CCIR601 video board, and a custom mother board that contained 20 SCSI busses, some fast memory with an XOR engine, and an Ethernet chip. We also had a bunch of miscellaneous devices such as temperature sensors, fan monitors, and LED indicators for which we wrote "driverlets." We considered some of the commercial realtime operating systems, but their source code wasn't available. Our naive hardware engineers pushed for an NT-based solution, but they didn't understand what it takes to get a system running without source code. If we had gone with NT, our little company would still be begging Microsoft for modifications, patches, and cooperation. Instead, we controlled our own destiny, and within days of the hardware functioning, UNIX controlled the product. Using UNIX made so much sense because we required most of its functionality. The product needed networking (TCP/IP), a filesystem, field update capability (CVSUP), Graphical User Interface (Tcl/Tk), realtime process management, virtual memory, and more. With six software engineers, it would have taken much longer than 12months to re-implement all of this work.

Source code UNIX systems are for engineers who need (or want) to have more control over their system than the vendors allow. It is also the best way to learn about how operating systems in general and UNIX specifically work. Those of us with binary vendor releases improve our environment by downloading publicly available application source code, compiling it, and running it on our platforms. This often vastly improves the workstation situation by supplementing the standard vendor packages. One can spend enormous amounts of time obtaining and configuring publicly available packages to one's platform.

Source code UNIX systems have various methods for configuring third-party applications. For instance, there is the FreeBSD ports tree and the Redhat Package Manager (rpm) for Linux. These facilities make it simple to install the latest versions of thousand of applications that come with source code. Some examples are audio tools, mail systems, Web utilities, databases, and graphics packages. I'll dedicate a column in the future to reviewing ports and packages.

How About a Windows Server?

Ironically, source code UNIX makes an excellent disk, mail, and print server for Microsoft Windows 95 (Win95) clients. After you install Samba,[2] Win95 PCs can easily access mail, Web pages and files that reside on an inexpensive, old, low-end PC. In this environment, you can set up a powerful backup capability for the clients. You have an alternative to the NT server; something that is much less expensive and performs better in many situations. Users who have Win95 will be happy, and you, the administrator, will be happy, too. (The Netatalk package gives these capabilities to Apple Macintosh clients.)

Are There Economic Reasons?

There is an economic reason to consider source code UNIX. Even though the traditional vendors have rapidly dropped their workstation's prices, commodity PC hardware is still "dirt cheap." The technology has been and will most likely continue to be one step behind with PCs, but it's really hard to beat the price/performance ratio. If you are willing to be careful when shopping and avoid certain problematic hardware, you can put together a solid, high-performance system for a small amount of money ($1,000-$2,000). Of course, it is reasonable that traditional UNIX vendors get a premium for their carefully integrated and balanced systems. They also get the premium for support that most importantly includes technical support ­ so you don't need a guru around. If you can't support your system or you don't want to do any of the integration work, then source code UNIX may not be for you.

Instead of buying a new PC, a scenario for running source code UNIX is to find "obsolete" PC hardware ­ the stuff that no longer can efficiently run Win95 or NT. Many consider a 16MB 486 underpowered to run Office97. Certainly, NT will require at least a husky Pentium with 32MB or more. But source code UNIX runs on these "discards." If you don't need an X11 system, an 8MB 486 system can be peppy for certain applications. You don't need much disk space for source code UNIX. A small, couple hundred MB disk is big enough for a substantial development system.

Keep in mind too that the cost of running Windows on your PC is not trivial. After you buy the operating system for a couple of hundred dollars, you still have to buy everything else. Plan on spending up to $500 for a C development system. Spreadsheets and word processors cost a few hundred dollars. Then you might need a mail system, virus checker, Web server, security software, and a backup system ­ every one costing a lot of money. By the time you are done, you could easily spend $1,000-$2,000. All of these utilities are included in the source code UNIX systems. The free versions might not be as feature laden, but you can certainly accomplish the same tasks efficiently. You also have the option of purchasing commercial software such as spreadsheets or word processors for your UNIX system.

Finally, you may have an existing system that is already running MS Windows. It is straightforward to split the disk into a Windows system and a UNIX system. It takes a minute or two to switch between the operating systems with a reboot. I run one of my systems with 500MB for Windows and 1.5GB for source code UNIX. My source code UNIX system mounts the Windows partition, facilitating file transfer between the operating systems.

What Would a Modest New PC Cost?

If you haven't looked recently, you will be amazed how fast PC prices have dropped. A very respectable new system without monitor costs under $700 (February 1998). It consists of the following:

• Pentium
  16MB memory
  2GB disk
  16x CD-ROM
  56K modem

Monitors range in price from $200 for a cheap 14-inch unit to over $1,600 for a high quality 21-inch monitor. A nice 17-inch color display costs between $500 and $600.

What If I Want the High-End Stuff?

A hot 300MHz Pentium II MMX with 64MB of memory costs about $1,400 February 1998). For that price, you would also get an 8GB disk, a DVD CD-ROM, a 56K modem, and a 4MB video board, but no monitor.

What If I'm on a Tight Budget?

With just a little hunting, you should be able to find a complete used system for $100-$500. A used Pentium 90MHz with 16MB memory, 1GB disk, and monitor should be easy to find. The 486 systems should be almost free, but with monitor, disk, and printer, maybe $100 or $200. Many companies mothball these systems because they cannot run the current Windows software. You can exploit this fact to gather cheap or free hardware that makes good firewalls or fileservers. But for development, I'd want a bit more pep than a 486.

What Source Code UNIX Should I Run?

(I'm going to try hard to avoid religious wars!) First, a disclaimer: I think all of the source code UNIX systems are viable ­ no need to fuss. My history is with FreeBSD, which will show when I give concrete examples. I wish the various source code UNIX groups worked more closely together. I think Jordan Hubbard, a core member of FreeBSD, sums it up well when talking about Linux and FreeBSD:

• now, within a very short space of time, we're almost spoiled for choice in having machines several times more powerful than the first multiuser VAX machines and available for under $2,000, and we've got not one but several perfectly reasonable free operating systems to chose from. We are in a comparative paradise, and what are some of us doing? Complaining about it! I suppose too much is never enough, eh?
• to which is "best," I have only one standard reply: try them both, see for yourself, think for yourself. Both groups have given you something for free, at considerable personal effort, and the least you can do is give them the benefit of exerting enough effort to try out what they're offering before passing judgment (or worse, blindly accepting someone else's!).
• you run, you're getting a great deal ­ enjoy!

The source code UNIX systems have a tremendous amount of overlap. They share lots of their innovations. Most of the publicly available applications run fine on every system because the system call interfaces are compatible. The differences are relatively minor and difficult to sum up in a few words. Linux is the most widespread, with probably several million installations worldwide. Linux is often described as Sys V-like and has the reputation of supporting some of the more obscure PC hardware. BSDI offers its commercially supported version that is available with source code. FreeBSD has spent a lot of time tuning its I/O and VM subsystems. They claim to have the busiest public FTP server in the world: <ftp.wcarchive.com>. NetBSD's mission is to run on many platforms: PC's, SPARCs, alphas, etc. See <http://www.netbsd.org/Ports/index.html> for the full list. OpenBSD is a recent split from NetBSD. They run on multiple platforms and are committed to fixing security bugs. See <http://www.openbsd.org/goals.html>.

How Do I Get Started?

There is a wealth of information available for source code UNIX. I suggest you become familiar with the Web pages for the version you select (sites given in the sidebar on page 56). These home pages are the definitive source for the various systems and may supersede information provided in this article.

Subscribe to some of the USENET groups listed below. Get the Frequently Asked Questions document:

ftp rtfm.mit.edu
cd pub/usenet/news.answers/386bsd-faq
mget part*

There are a number of ways to load your system with source code UNIX. You might be able to clone someone else's disk. If you have good Internet connectivity, you might

download the system over the wire. (And once you have it installed, you may want to incrementally update the code base with programs such as CVSUP, IIRC, SUP or Anon-CVS. More on this in future columns.) But for most of you, I recommend spending $30-$50 for a CD distribution set. (Note, you are paying for the convenience of having your own CD disks. Feel free to borrow someone's CDs or lend yours.) From these disks, installation is usually quick and easy. Next month I'll go through the typical steps for loading a system. I'll also spend some time discussing various hardware options. For now, spend some time on the Web, get familiar with some of your options, then choose a starting source code UNIX system and order its CD-ROM set.

I'd like to thank the following reviewers: Ken Merry, Steve Gaede, Mike Durian, and Joel Rem.

Books and Articles

For those who want to understand the organization of BSD kernels, you will want Kirk McKusick et al., The Design and Implementation of the 4.4BSD Operating Systems (Addison Wesley, 1996.)

O'Reilly & Associates has dozens of UNIX-related books. <http://www.ora.com>.

For UNIX administration, I highly recommend: Evi Nemeth et al., UNIX System Administration Handbook, 2nd ed. (Prentice Hall, 1995).

For a technical description of porting UNIX to 386, see these four articles in Dr Dobb's Journal: William Jolitz and Lynne Greer Jolitz, "Porting UNIX to the 386: A Practical Approach," January 1991; "Porting UNIX to the 386: Three Initial PC Utilities," February 1991; "Porting UNIX to the 386: The Standalone System," March 1991; "Porting UNIX to the 386: A Stripped-Down Kernel," July 1991

Notes

[1] I define publicly available software as software with freely redistributable source code.

[2] Samba, <http://samba.anu.edu.au/samba>. Also see Jeremy Allison, "The Samba File and Print Server" ;login:, November 1997, pp. 12-18.

Resources on the Net

FreeBSD

Main Web site: <http://www.freebsd.org>
USENET NEWS: <comp.unix.bsd.freebsd.announce>, etc.

You will want to become familiar with the FreeBSD Handbook (see the main Web site). Also get familiar with the FreeBSD Hypertext Man Pages and Frequently Asked Questions for FreeBSD. The site's search engine can be invaluable for finding information. Mailing lists, a la majordomo, are used by most of the source code UNIX organizations. There is a higher signal-to-noise ratio than with news groups. Check the respective Web pages for details.

LINUX

Main Web site: <http://www.linux.org>
USENET NEWS: <comp.os.linux.announce>, etc.

For a mostly complete list of Linux distributions, see <http://www.linux.org/dist/index.html>.

A complete list of HOWTOs and Mini-HOWTOs is available in the file HOWTO.INDEX in the docs/HOWTO directory at the FTP sites, or on the Web at <http://sunsite.unc.edu/pub/Linux/HOWTO/HOWTO-INDEX.html>.

NetBSD

Main Web site: <http://www.netbsd.org>
USENET NEWS: <comp.unix.bsd.netbsd.misc>, etc

OpenBSD

Main Web site: <http://www.openbsd.org>
USENET NEWS: <comp.unix.bsd.openbsd.announce>, etc

BSDI

Main Web site: <http://www.bsdi.com>
USENET NEWS: <comp.unix.bsd.bsdi.announce>, etc