USENIX Technical Program - Paper - Proceedings of the 12th Systems Administration Conference (LISA '98)
An NFS Configuration Management System and its Underlying Object-Oriented Model
This paper describes an NFS configuration and management system for large and heterogeneous computer environments. It also shows how this system can be extended to address other services in the network. The solution is composed of a process that describes service configuration and management life-cycle, a modular architecture and an objected oriented model. The system supports multiple features, including: automatic host and service installation, service dependency inference and analysis, performance analysis, configuration optimization as well as service functioning monitoring and problem correction.
The installation and configuration of hosts and services are two very common tasks in the administration of any computer network. Even small and homogeneous networks offer several different services that must be consistently configured on server and client computers. Furthermore, configuration management becomes a very complex problem when large and heterogeneous systems, together with their Internet connection, are taken into account. According to several authors [2,3], some of the reasons for this increase in complexity are :
The configuration of hosts and services have dramatic influence on network performance, resilience and safety, and therefore are among the most critical tasks in system administration. For instance, it is widely known that a large percentage of security problems on networks connected to the Internet are due to bad Internet services (like HTTP and FTP) configuration.
Therefore, more systematic and structured processes of service configuration are necessary for the administration of today's networks and, in particular, those that are connected to the Internet. Furthermore, to achieve high levels of consistency and safety, any such process must be supported by automated tools which must possess three fundamental properties:
This article presents a service and configuration management system that supports NFS configuration and monitoring on large and heterogeneous networks. The system is based on a formal model that describes hosts, network components and services in a generic and abstract way. This model allows the system to be extended to support several different services in a variety of platforms.
The next section will present the service configuration problem, the related work and how our solution extends the state of the art. Subsequently, we will show the process that describes an NFS service configuration and management life-cycle. The next section will present the system architecture. Then we will demonstrate the generalization of our solution for the other network services. Finally, we will present our conclusions.
Service Configuration and Related Work
Typically, the configuration of a service is composed of several interconnected tasks: planning, consistency checking, deployment, and management. In each task, a number of critical issues must be addressed, including:
These are only a few of the issues that must be addressed in the process of host and service configuration. Hardware vendor's specific tools, like the AdminTool from Sun Microsystems Inc. and Smit from IBM Inc., do not provide satisfactory solutions to deal with heterogeneous system.
According to Evard , "the general approach taken by the administrative community over this time period has been to develop a host cloning process and then to distribute updates directly to hosts from a central repository." In a large and heterogeneous network, this cloning process is not satisfactory because each machine has its own characteristics.
In most solutions, the central repository is a collection of ASCII files, like in LCFG , GeNUAdmin , Syslogd  and Fisk's system . This leads to the problem of keeping the consistency and the integrity of the information. Some tools have changed this approach by using DBMSs, like UHA , Aurora , Finke's system .
The swatch  and pong  systems are designed to monitor the network and some services. However, neither addresses the complete life-cycle of service configuration and monitoring, and, therefore, does not support integrated service management.
None of the above cited approaches are based on a formal and generic conceptual model of the network and its services. Such a model is the basis of commercial tools like TME10(  and Unicenter TNG( , and are essential to support the inclusion of new services and to keep the overall integration of the system. These two market leaders offer extensive features to manage heterogeneous networks, but do not offer built in facilities for high level configuration management.
In this paper we present a solution dedicated to address the problem of configuring and managing network services in heterogeneous environments. This system presents the following features:
An NFS Configuration and Management Process
The configuration and management of the NFS service follows a process that is common to most services, and is graphically depicted in Figure 1.
Figure 1: The Process.
Following this process, the administrator should perform a number of tasks in a coherent and consistent form:
The Architecture of the System
The NFS Configuration and Management System was implemented according to the architecture showed in Figure 2.
Figure 2: The Architecture.
This architecture was presented by Franklin in .
The architecture implements the process showed in Figure 1, which is based in the following stages: planning, deployment, monitoring, diagnosis and actuation.
The architecture has four components: the DBMS, the Agents Society, the Manager and the Interface.
The DBMS is the central element of the architecture and it is partitioned into two components:
The Agents Society, based on the intelligent agent paradigm described in , is composed of three types of agents, with complementary functionalities:
The agents have been implemented in Java, using the IBM Aglets Workbench API . This API supports the construction of mobile multi-agents systems.
The Manager controls the agents and interaction process between the architecture modules, activating, deactivating and creating the agents when necessary. When a set of agents is activated, it is not necessary to wait for all agents to finish their tasks to record information on the database. The manager records incomplete information and controls the time-out interval (defined by the system administrator) of all running agents. Incomplete information is dealt with by the reasoning agents. The manager has been implemented using the same technology as the agents.
The Interface allows the interaction between the system and the administrators. It provides the following functionalities: the DBMS front-end, visualization of all management processes, the NFS status in the network and visualization of the service planning.
The system interface has been developed using HTML embedded into ORACLE PL/SQL stored procedures. The use of HTML pages allows greater portability and easy of access through any web browser. Furthermore, the use of stored procedures facilitates the interaction with the database. However, it makes the interface dependent on the ORACLE database. A more portable solution using JAVA and ServLets to communicate with the DBMS is under construction.
The planning and verification stages are also implemented in PL/SQL and accessed directly through the interface. The remaining stages are implemented by the agents society, as described above.
A set of communication protocols are needed: between the agents of the same category, between agents of distinct categories, between the agents and the DBMS, between the agents and the network components. The latter implements the propagation and activation phases of the process described in Section III and is the subject of another paper .
The Use of the System
The navigational model follows the cycle defined in Figure 1. Each stage has been implemented as a separate subsystem, and are reached through the first page of the interface, as shown in Figure 3.
Figure 3: NFS configuration management systems interface.
Figure 4: Planning subsystem.
Planning and Validation Subsystems
This subsystem allows the definition of NFS clients and servers. Only hosts that can be servers appear in the list box of Figure 4 (e.g., those that have disks). Similarly for clients.
The following step is to configure the selected server and client hosts. As shown in Figure 5, once a host running Solaris 2.5.1 is selected, only the platform specific parameters for this version of the operating system appear in the configuration interface.
Figure 5: Planning subsystem - second interface.
Once the NFS planning is finished, that is, all servers and clients are selected and configured, it is necessary to check and validate the configuration. This checking is perform by the validation subsystem, which is a PL/SQL procedure that either requests the system administrator to modify any inconsistencies found in the configuration or stores the configuration in the database if it is correct.
Deployment and Actuation Subsystems
To deploy or modify an NFS configuration, the manager verifies what needs to be updated (a deployment or modification may only affects a subset of the hosts) and schedules the process. The start of the process can be immediate or in a pre-schedule time defined by the administrator, as shown in Figure 6.
Figure 6: Deployment/actuation subsystem.
At the deployment time, the agents and the configuration protocol are activated.
In this subsystem, the manager queries the monitoring database looking for monitoring information. The system administrator tells the manager, through the interface, which agents need to be activated, which hosts to monitor and the time intervals of the monitoring processes, as shown in Figure 7.
Figure 7: Monitoring subsystem.
At the defined time, the manager activates the monitoring agents, which travel to the target hosts and start the monitoring scripts, as discussed above. Once the monitoring process is finished the manager inserts the information in the monitoring database.
The diagnosis is also started through the manager, by a system administrator's request. However, it can also be automatically triggered every time new information from the monitoring process is stored in the database.
The manager starts the diagnosis agents which will compare the configuration and monitoring database, looking for possible inconsistencies, and will notify the manager with two possible results:
The manager is responsible to format the result and send messages to the system administrator.
Reverse Engineering Subsystem
This subsystem is implemented by the monitoring agents already defined and by specific agents capable of collecting more detailed information about the network hosts. The reverse engineering is similar to the monitoring process, but in this case the collected information about the network will be stored in the configuration database. This process has to be authorized by the system administrator, since it changes the network configuration.
The generalization depends on a conceptual model, specially designed to support all the necessary components and characteristics to configure and manage services in an heterogeneous network. This model, described in , uses object oriented paradigm that allows abstract descriptions of hosts, platform, services and configuration parameters, and dependencies among them. In this model, it is possible to construct a platform independent specification of a given service, which when combined with the specification of a given platform, produces the instantiation of the service for that platform. The inclusion of a new service is easily performed only by class instantiation.
Figure 8: The conceptual model.
The model showed in Figure 8 has seven classes, described as follows.
The class Service contains the description of the service (e.g., filesystem sharing services). This class is useful when there are several products that implements the same service. In this case, each implementation is modeled as a separate product and all versions belong to the same service.
The class Product contains product specific information, like, for instance, version and supported platforms (e.g., SunOS 4.1 version of NFS). This class has a self relationship permitting the dependency representation among services.
Each instance of the class Parameter represents a configuration parameter (e.g., filesystem access type). New parameters can be added through class instantiation. Each instance of the class Product refers to one or more instances of the class Parameter, therefore representing the information that is necessary to install product of a given service on the network hosts.
Each configuration parameter associated with a service has a set of restrictions. Different implementations of a service for distinct platforms have different set of restrictions. The restrictions are captured by the class Parameter Restriction, where each parameter connected to a service and a platform has well defined rules for its correct use during the configuration process (e.g., the filesystem access type parameter can only assume the values ro - read-only or rw - read-write).
The class Configured Value holds the actual configuration values for each parameter of a given host (e.g., on host host1, the filesystem access type parameter has value ro to filesystem1). The class Host represents all hosts of a computer network and each instance of this class represents an actual host. (e.g., host1).
The class Platform captures the heterogeneous features of the hardware and operating system platforms on the networks (e.g., host IBM RS/6000 running AIX 4.1). A new platform can be easily added through class instantiation.
This model is independent of any implementation decision. So, it can be implemented in ASCII files or in any desired DBMS. In our case, we used ORACLE DBMS.
This object-oriented conceptual model specifies the database that holds network configuration information, it represents the DBMS component of the architecture showed on Figure 2. From this database, the configuration information is distributed to every host on the network using a distribution protocol. This protocol is described in .
Some features offered by this model are listed as follows:
The service configuration and management system presented in this article has several important and desirable features:
The solution presented in this article, uses several technologies from the design and implementation of a complex system, capable of support the planning, configuration and pro-active management of services in heterogeneous networks. The NFS prototype was used to validate the process, architecture and conceptual model.
The possibility of supporting other services, coherently integrated in the framework, is a clear advantage of this solution when compared to managing the services isolated and by hand. Currently, HTTP, NIS and DNS services are being included in the system. This will allow not only the configuration management of these systems separately, but also the correlation of information among all supported services.
The FLASH project is co-funded by the Brazilian Government agency CNPq, through the ProTeM-CC Program (Phase III) and by the Center for Advanced Studies and Systems at Recife (CESAR).
The prototype implementation, together with a configuration distribution system , has been used to assist the administration of laboratories of the Department of Informatics, UFPE. However, the system will only be available for distribution after the inclusion of the above mentioned services.
Fabio Q. B. da Silva is an Associated Professor of the Department of Informatics at the Federal University of Pernambuco, Brazil, where he coordinates the FLASH Project. He holds a Ph.D. in Computer Science from the University of Edinburg, Scotland. He is also the Finance Director of the Center for Advanced Studies and Systems at Recife (CESAR), a not-for-profit organization dedicated to promote Industry/University interaction (https://www.cesar.org.br). Reach him electronically at <firstname.lastname@example.org>.
Juliana Silva da Cunha is a Phd Student of the Department of Informatics at the Federal University of Pernambuco, Brazil and a member of The FLASH Team. She holds a master degree in Computer Science at Federal University of Ceará, Brazil. You can reach Juliana electronically at <email@example.com>.
Danielle M. Franklin is currently a member of The FLASH Team and a system management consultant at CESAR. She holds a master degree in Computer Science. You can reach Danielle at <firstname.lastname@example.org>.
Luciana S. Varejao is currently a master student in Federal University of Pernambuco's Infomatics Department and a member of The FLASH Team. She holds a bachelors degree in Electronic Engineering. You can reach Luciana at <email@example.com>.
Rosalie Belian is a member of The FLASH Team. She holds a Master Degree in Computer Science. You can reach Rosalie electronically at <firstname.lastname@example.org>.
 Paul Anderson, "Towards a High-Level Machine Configuration System," USENIX Systems Administration (LISA VIII) Conference Proceedings, 1994.
 Magnus Harlander, "Heterogeneous Unix Environment: GeNUAdmin," USENIX Systems Administration (LISA VIII) Conference Proceedings, 1994.
 John Rouillard and Richard Martim, "Config: A Mechanism for Installing and Tracking System Configurations," USENIX Systems Administration (LISA VIII) Conference Proceedings, 1994.
 Rex Walters, "Tracking Hardware Configuration in a Heterogeneous Network with syslogd," USENIX Systems Administration (LISA IX) Conference Proceedings, 1995.
 Michael Fisk, "Automating the Administration of Heterogeneous LANs," USENIX Systems Administration (LISA X) Conference Proceedings, 1996.
 Grady Booch, Object-Oriented Analysis and Design With Applications, Second Edition, Addison-Wesley, 1994.
 FLASH Project, https://www.di.ufpe.br/~flash.
 Hal Stern, Managing NIS and NFS, O'Reilly & Associates Inc., 1991.
 Glêdson E. da Silveira and Fabio Q. B. da Silva, "A Configuration Distributed System for Heterogeneous Networks," USENIX Systems Administration (LISA XII) Conference, 1998.
 Juliana Silva da Cunha, Glêdson E. da Silveira, Fabio Q. B. da Silva, J. Neuman de Souza, "An Object-Oriented Service Configuration Management System," International Conference on Telecommunication (ICT-98), Chalkidiki, Greece, June 1998.
 Danielle Franklin. I-DREAM: an Intranet baseD REsource and Application Monitoring system. Master Degree Thesis, Federal University of Pernambuco, 1997.
 Michel Wooldridge, Nicholas R. Jennings, "Intelligent Agents: Theory and Practice," Knowledge Engineering Review, Cambridge University Press, 1995.
 Jon Finke, "Automation of Site Configuration Management," USENIX Systems Administration (LISA XI) Conference Proceedings, 1997.
 Gregory Thomas, James Schroeder, Merilee Orcutt, Desiree Johnson, Jeffrey Simmelink, John Moore, "UNIX Host Administration in a Heterogeneous Distributed Computing Environment," USENIX Systems Administration (LISA X) Conference Proceedings, 1996.
 Helen Harrison, Mike Mitchell, Michael Shaddock, "Pong: A flexible network services monitoring systems," USENIX Systems Administration (LISA VIII) Conference Proceedings, 1994.
 Stephen Hansen, E. Todd Atkins, "Automated system monitoring and notification with swatch," USENIX Systems Administration (LISA VII) Conference Proceedings, 1993.
 Tivoli Systems Inc., https://www.tivoli.com.
 Computer Associates Inc., https://www.cai.com.
 Xev Gittler, W. Moore, J. Rambhaskar, "Morgan Stanley's Aurora System: Design a Next Generation Global Production Unix Environment," USENIX Systems Administration (LISA IX) Conference Proceedings, 1995.
 Rémy Evard, "An Analysis of UNIX System Configuration," USENIX Systems Administration (LISA XI) Conference Proceedings, 1997.
 Luciana S. Varejao. Sistema de Monitoraçao para Redes Heterogêneas (A Monitoring System for Heterogeneous Networks). Master Degree Thesis (under development), Federal University of Pernambuco, 1998.
 Agent Building Environment, https://www.networking. ibm.com/iag/iagsoft.htm.
 Knowledge Interchange Format (KIF) https://logic.stanford.edu/kif/.
 IBM Aglets Workbench Homepage https://www.trl.ibm.co.jp/aglets/.
This paper was originally published in the
Proceedings of the 12th Systems Administration Conference (LISA '98), December 6-11, 1998, Boston, Massachusetts, USA
Last changed: 3 April 2002 ml