11th Systems Administration Conference (LISA '97)
Implementing a Generalized Tool for Network Monitoring
Marcus J. Ranum, Kent Landfield, Mike Stolarchuk, Mark Sienkiewicz,
Andrew Lambeth, and Eric Wal
Network Flight Recorder, Inc.
Determining how you were attacked is essential to developing a
response or countermeasure. Usually, a system or network manager
presented with a successful intrusion has very little information with
which to work: a possibly corrupted system log, a firewall log, and
perhaps some tcpdump output.
When hackers come up with a new technique for cracking a network,
it often takes the security community a while to determine the method
being used. In aviation, an aircraft's ``black box'' is used to
analyze the details of a crash. We believe a similar capability is
needed for networks. Being able to quickly learn how an attack works
will shorten the effective useful lifetime of the
attack. Additionally, the recovered attack records may be helpful in
tracking or prosecuting the attacker. Since we've developed a general
purpose statistics-gathering system, we believe it will be useful for
more than just security. For example, a network manager may desire an
historical record of the usage growth of certain applications, or
details about the breakdown of types of traffic at different times of
day. Such records will provide useful information for network managers
in diagnosing performance problems or planning growth.
This paper describes an architecture and toolkit for building
network traffic analysis and statistical event records: The Network
Flight Recorder. The NFR uses a promiscuous packet interface to pass
visible traffic into an internally meta-programmed decision engine
which routes information about packets and their contents into
statistical or logging backends. In addition to packet analysis and
collection, the NFR's internal architecture permits network managers
to sample interesting portions of network traffic for logging or
statistical analysis. The NFR programming language is simple, but
powerful enough that you can perform reasonable analysis on traffic
before choosing to record it. For example, you might analyze SMTP
transactions but only choose to record those relating to a user who is
sending spam or abusive E-mail. The analysis language includes a
capability for generating alert messages which the rest of the system
queues, multiplexes, and delivers. A simplified hyper-query interface
allows extensive browsing of the NFR's stored datasets and statistics
from any Java-enabled browser. The NFR is currently being deployed at
a number of ISPs and commercial sites, and is available for download
in source code form from www.nfr.net.
- View the full text of this paper in
HTML form and
- If you need the latest Adobe Acrobat Reader, you can download it from Adobe's site.
- To become a USENIX Member, please see our Membership Information.