| ||||||||||||||||||||||||||||||||||||||||||||||||||||
Third USENIX Conference on Object-Oriented Technologies (COOTS), 1997
[Technical Program]
Obtuse, a scripting language for migratory applications Robert P. Cook www.cs.olemiss.edu/~bobcook; bobcook@cs.olemiss.edu
Abstract This paper discusses the design and implementation of Obtuse, a scripting language for migratory applications. The paper reviews the pertinent ActiveX technology that provides the runtime object infrastructure. Then we discuss the Obtuse object model and present an overview of the language. Next, several sample programs are used to illustrate the concepts. Finally, we review some of the problems with DCOM, based on our experience. Keywords: scripting language, Obtuse, migratory applications, Obliq, distributed systems 1. Introduction Obtuse was designed by the author, inspired by Cardelli’s Obliq [1] and Bharat’s Visual Obliq[2] systems, and implemented as part of a summer research appointment at Microsoft Corporation. The goal was to explore the potential of several core ActiveX technologies [3,4,5], including COM (Component Object Model), Automation, and DCOM (Distributed Component Object Model). Obtuse is unique in two respects; first, in its synergistic use of ActiveX technology and second, in its ability to transfer the state of a Visual Basic form from one machine to another. A migratory application is one that can transfer program state (including the user interface) to different Internet locations under program control. Other terms used in the literature are mobile or transportable agents. Obtuse is also a scripting language; that is, it defines sentences capable of being executed as fine-grained code fragments. As an example of transferring UI state from one machine to another, consider the Visual Basic (VB) form in Figure 1, which consists of an edit control and a button. The form is used in a simple, routing-slip application. The user can type a command line, such as "obtuse poll m1 m2 m3" to initiate execution. The list (a routing slip) represents a sequence of computers to visit. The DCOM infrastructure is utilized by the Obtuse runtime to implement the remote activation that is necessary to support the routing-slip application. The form is circulated to the machines in the order listed. The accumulated comments are available to each recipient and the completed form, with all comments, is returned to the source machine. When one user clicks the OK button, the form is moved to the screen of the next computer in the list.
Figure 1. User Interface for a Roving Poller We refer to programs, such as the routing-slip example, as in-your-face applications. When one user clicks the routing slip’s OK button, the document appears instantaneously on the screen of the next recipient. Most word processors also support routing slips by using e-mail as the transport mechanism. However, users are only notified if an e-mail client is executing at a site and if they decide to read their mail. In the routing-slip application, the code, together with its execution state, can also migrate from one machine to another with the form. Obtuse implements program migration by exposing threads and contexts (a program’s global variables) as COM objects. Figure 2 lists a simple Obtuse program that moves itself from one machine to another. In Obtuse, a running program is a collection of COM objects, which support an Automation interface. The sample program creates a thread and a context object on a remote machine. Next, the Fork method of the running-thread object is invoked to clone the program’s state. At this point, there are two threads executing, one locally and one remotely. However, since they have duplicate contexts, any object references in one are duplicated in the other. As a result, the remote thread can access objects on the parent machine in a location-opaque fashion. // Note that variables are "typed" at runtime var me, thread, context, where;
end; Figure 2. A Sample Obtuse Program Obtuse is unique in that the mechanisms to support the runtime (threads, contexts, stacks) are all COM objects. Another unique aspect is that Obtuse uses Visual Basic forms to implement user interfaces. These forms can also be marshaled in order to transport their state from one machine to another. Since Obtuse is based on COM, it can be used to manipulate any COM Automation object, which includes all Office applications and ActiveX controls. The DCOM infrastructure supports the remote location and activation of COM objects. Other features of Obtuse include support for script-based execution, support for multiple threads, runtime strict typing, and a machine-invariant program representation. An Obtuse program can consist of a sequence of expressions with no variables, a series of statements on a set of global variables, or a collection of procedures. Furthermore, an Obtuse program can invoke an Automation object’s methods and access its properties at runtime; it is not necessary to "import" or "include" interfaces. Obtuse does not support compile-time type binding. The type checking in expressions and procedure calls is performed at runtime. However, Obtuse is "strict"; that is, types must match exactly on operations such as comparison or multiplication. Variables are bound to a type on runtime assignment. From that point on, until another assignment occurs, that variable must be type compatible with every operator that is applied to it. Programs are UNICODE-based and compile to a machine-invariant representation that encodes the source program. That is, the object code can be inverted to recover the original source, including comments. The paper first presents an overview of ActiveX technology. Then we discuss the Obtuse object model and present an overview of the language. Next, several sample programs are introduced to illustrate the concepts. Also, we present some performance measurements for Obtuse/ActiveX. Finally, we review some of the problems with DCOM based on our experience. 2. ActiveX—COM and DCOM The two most important aspects of ActiveX for scripting support are its implementation of dynamic method binding and invocation, as well as self-describing types. Dynamic method binding is the technology (Automation) that enables Visual Basic applications to manipulate Office documents, such as spreadsheets or slide presentations. It also enables HTML scripting support (VBScript) in Microsoft’s Internet Explorer 3.0. The dynamic, or late, binding technology enables an object to expose methods and properties for use by other objects. The technology also supports the lookup of method and property names and a mechanism to build and execute a procedure call at runtime. It is a separate set of code from COM. Self-describing types (or variants as they are termed in COM), are the key, underlying representation for data types in the Visual Basic language common to VBScript and VB. In the next sections, we present an overview of COM and DCOM. 2.1 COM – Component Object Model An "object" in COM typically has a document type, such as .xls, .ppt, .doc. Each document type can have a registered server. For example, winword.exe is the server for *.doc objects. Objects also have a registered application name (e.g. "Microsoft Word Document") and a globally-unique identification number, called a class id (CLSID). The association between a class and its server is maintained in a persistent store called the registry. There is one registry per machine and there is currently no "yellow pages" server to support object lookup for distributed services, although one is reputed to be available shortly. The COM model is language independent; it may have a concrete implementation in a particular language, such as C++, but the relationship between COM and different languages is orthogonal. For example, Obtuse is implemented in C++ but it uses COM objects, which are implemented in Visual Basic, to define its user interface. A COM object is defined by its support for a collection of interfaces, each is which is tagged by a 128-bit globally unique interface identifier (IID). The interfaces that an object supports can vary over time; and the interfaces need not have any other relationship (such as inheritance). There is only one requirement i.e. EVERY COM INTERFACE MUST INHERIT FROM THE IUnknown INTERFACE, which is listed in Figure 3. virtual HRESULT QueryInterface (InterfaceID & riid, LPVOID * ppvObj)=0; virtual HRESULT AddRef(void) = 0; virtual HRESULT Release(void) = 0; Figure 3. COM IUnknown Interface
The power of COM derives from several of the requirements satisfied by the IUnknown implementation. First, QueryInterface must be used to obtain an object handle for an instance variable x (as in x.Queryinterface) that supports a particular interface (identified by the InterfaceID argument). If the object x does not support the interface, the HRESULT returned indicates an error. In C++, a COM object handle is a "pointer to a pointer to a vTable". The vTable is generated in C++ because the interface is "pure virtual", as are all COM interfaces. Since every interface is required to inherit from IUnknown, any object handle can be used to retrieve a handle for any interface that the object supports by calling QueryInterface at any time. Further, the object handles are reference counted. QueryInterface increments an object’s reference count and so does AddRef. A Release call decrements an object’s reference count. 2.2 DCOM – Distributed COM DCOM extends COM in a number of ways. First, objects can be remotely activated and a handle returned to the activating site. The returned object handle can be used by a program in a location-opaque fashion; that is, the programmer need not be aware of the object’s location. Second, DCOM imposes location, security and identity restrictions on COM objects. Each site has total control over who can activate an object, how objects are activated, and with what permissions object servers can execute. Third, DCOM implements reference counting across machine boundaries and garbage collection. Finally, DCOM automatically remotes calls to COM interfaces that are supported by remote objects. For user-defined interfaces, an IDL compiler must be used to generate proxy stubs for the client/server sides. Typically, both stubs are included in a single DLL. 2.2.1 Remote activation The DCOM method to activate (cause its server to be loaded) a remote object is CoCreateInstanceEx. The arguments to the method are the object’s class id, a machine name, and a list of interface ids. Machines are identified using the naming scheme of the network transport layer. By default, all UNC (\\chairpc) and DNS names ("chair.com" or "135.9.19.33") are supported. Object search is restricted to a single machine at present. DCOM has no notion of distributed scope or of distributed search paths. To optimize network performance, the CoCreate call may specify a list of interface ids. Thus, N object handles can be retrieved in a single round-trip to the server site. Conceptually, this is analogous at runtime to the "import java.lang.*" convention in Java, which can be used to import all of the classes in a package at compile-time. 2.2.2 Access control Access to objects can be regulated under program control using the NT security API; however for most Obtuse users, the utility program dcomcnfg is the point of control. This program lists the application objects that are "registered" on a particular machine. The Location, Security, and Identity of each object can be separately controlled. The Location options are "run here" or "run there". The latter option supports forwarding a requested activation from one computer to another. The Security option supports editing the access control lists (ACLs) for activation, access and configuration. NT provides very fine-grained access control so that individual users, or groups, can be specified. The Identity option designates the protection domain in which a server is executed. The choices are the domain of the interactive user, the launching user, a particular user, or a system service. For example, the "particular user" option can be used to solve the "game accounting" problem, which is to let a user run a game program that can write its list of winners to a file that is not accessible to one of the players. The appropriate protection domains can be created by having one DCOM object (player’s domain) to play the game communicating with another DCOM object (game’s domain) to record the scores. 2.2.3 Reference counting As we discussed in Section 2.1, COM defines a mechanism to reference count object handles. If a program fails, any cross-process links must be broken to properly release objects. Similarly for DCOM, the system must account for inter-machine links and must break links when processes terminate or fail. For distributed systems, there are also the possibilities of node crashes and communication outages. The DCOM implementation addresses these problems. 2.2.4 Remote procedure call DCOM automatically remotes inter-node calls on COM interfaces. The arguments are marshaled through the normal remote procedure call (RPC) mechanism. RPC on user-defined interfaces requires the use of the IDL compiler to generate client- and server-side proxy stubs. The automation interface (IDispatch) can be used to "late bind" a procedure call; that is, a program can build a procedure call at runtime. The automation interface provides the object-access infrastructure for any scripting language, such as Obtuse, VBScript, JavaScript or AppleScript. COM supports the registration of type libraries that describe an object’s properties and methods (also argument lists and return values). As a property example, a button object might have BackgroundColor and Text properties, which could be accessed or modified remotely using Obtuse. The IDispatch interface includes methods to "query" for the id of a method or property name and then to "invoke" that method or "access" that property. The Automation runtime builds the argument list in a format that is compatible with the target language and handles the call/return processing. 2.2.5 Variant data Another aspect of the automation solution is a "universal" data type termed a variant. A variant is a "union" of about 40 different base types that also includes arrays of those types, and arrays of arrays. An array can be created with homogeneous elements of a particular type or with variant-type elements, each of which can be of any type. IDispatch and IUnknown object handles are two of the possible variant-record base types. Since IDispatch is one of the "builtin" COM interfaces, it is remoted automatically by DCOM. In Obtuse, all argument lists to procedures, return values, and property values are encoded as variant data. 3. Obtuse Language Overview The Obtuse system consists of a compiler and an interpreter, and a collection of COM objects. The compiler’s output is a UNICODE text string that encodes the source program, including comments. The executable can be inverted to produce the original source program. Thus, after a program is initially compiled, there is only one representation, which can be used for both execution and symbolic debugging. Obtuse supports only one data type (variant), so a variable declaration is just a list of identifiers. The type is implicit. A form of type checking is supported based on the notion of assignment-typing. Basically, every assignment statement binds a new type to an identifier as well as a new value. Expression evaluation is type checked at runtime. There is no implicit conversion as in VB; that is, type checking is "strict". In Obtuse, the "object" built-in function maps a registered object name at a particular machine to an object reference. For example, the function call object("Bob.Thread", "foo.univ.edu") would check the registry on the designated machine and then load the server if necessary. Once an object reference is obtained, the program can manipulate the properties of that object or invoke its methods. Assignment of object references copies the reference, not the value, even if the assignment crosses machine boundaries. The DCOM reference counting infrastructure tracks each copy. For assignment of other variant values, including arrays, Obtuse copies the value. The rule is simple: sharing can only be accomplished through COM objects. Obtuse implements a common set of statements such as if, loop, for, case, in addition to variable and method declarations. Pointer, structure and class declarations are not supported. A qualified reference can be used to access an object’s properties. Since an object’s methods are dynamically bound using the IDispatch automation interface, the compiler cannot perform checking for undefined names or mismatched argument lists. To facilitate some checking, calls to Obtuse procedures are delineated with the traditional "( )" and calls to an object’s methods use "{ }". As mentioned earlier, Obtuse programs are encoded as UNICODE strings. Sufficient information is retained in the encoding to invert the object code to the source. The opcodes were designed to use a character encoding so that program fragments could be embedded in documents, sent as mail messages, or be applied as drag-and-drop operators on user-interface objects. Figure 4 lists several example encodings. The blank, tab, and new-line opcodes are no-operations. The " opcode designates a constant. Constants are translated at runtime so the opcode includes a type designator, the length of the string, and the text constant. This is not very efficient but it does avoid representation issues, such as for floating-point numbers. Small integers are encoded as individual opcodes. The opcode design also took into account the requirements for the next version of Obtuse in which type modules, such as Complex numbers, could be called upon to parse their own constant representation.
Figure 4. Program Encoding Example The Y opcodes encode the syntax of the source program. Even the comments in the source are encoded, but the comment opcode is treated as a no-op at runtime. The compiler attempts to generate code for branch instructions so that syntax markers are not included in loops. 4. Obtuse Object Model The initial Obtuse implementation supports FORM, FILE, MUTEX, THREAD, CONTEXT, and STACK objects. FORM objects are Visual Basic forms, which can contain any VB control. Each FORM object represents one VB form. Since there are hundreds of different VB controls, the Obtuse user interface model has a broad range of capabilities. As a result, forms can be constructed as the user interface for almost any application. The FORM interface is implemented as a Visual Basic program. VB supports the creation of programs that support COM interfaces (particularly IDispatch, the Application Automation interface). As a result, these programs, can be activated remotely using DCOM. We implemented (in VB) a form-server object that supports Form, Item, Save and Restore methods. The Form and Item functions return object references to a form or to any of the controls on that form. Once an object reference is obtained, Obtuse can set or retrieve the properties of a form or control. For example, the "value" property of a scroll bar is a numeric quantity that can be used to get/set the thumb position. In the current prototype, a programmer constructs a user interface with a VB program called GenForm, which is part of the Obtuse system. When GenForm is executed, it writes a file that contains a text array constant that "defines" a form. The array constant is then inserted into an Obtuse program as a "resource". The Restore method causes the VB form-server object to display the previously-saved "look". A VB form is encoded/decoded by Save/Restore as a text array. Figure 5 illustrates the encoding of the Roving Poller form that was displayed in Figure 1. [12345, 12, 3, 15, 4, 5535, 1, 2145,5, 16776960, 2, "Roving Poller", 23456, Figure 5. Array Constant for a VB Form To save space when creating a new form, the GenForm program only saves the differences between a canonical set of control values and those specified by the user. For example, the "top" and "left" properties are almost always changed; the "visible" property is rarely changed. Since VB has a large number of properties for each control, this convention saves considerable space. Obtuse has the unusual property that the mechanisms of the language implementation are objects, in fact DCOM objects. Remember that a DCOM object can be activated on any machine. The Obtuse interpreter is only required to run Obtuse code, not remote objects. This is one of the main differences with Obliq and other distributed application systems, which require an instance of their interpreter at each node. A FILE object supports I/O on files and directories anywhere in the Internet. Interestingly, DCOM can pass a file handle from one machine to another and the handle retains its validity. This is not possible with NT, the host operating system. Since activating a FILE object is necessary to access files and since DCOM implements per machine and per object access controls, the user has full control over the safety of the system. A MUTEX object is used to implement critical section synchronization for shared variables or resources. The supported methods are Enter and Leave. The Obtuse object model takes unique advantage of DCOM’s capabilities. First, thread and context (a program’s global variables) objects can be created on any DCOM machine on the Internet so that a thread on one machine can opaquely access variables on any other machine’s context. Figure 6 lists the attributes of the three Obtuse program objects – Thread, Context, and Stack. A context can be shared among any number of threads, local or remote. For example, a master debug console can easily be constructed to monitor the modification of contexts located all over the world. Finally, a thread can migrate by simply forking its state to a new machine and killing the parent thread. Since a context is one of the arguments to the Fork method, the new thread can be created with its own copy of the parent’s context or it can share the parent’s context. All object references are marshaled properly by DCOM on inter-machine transfers so that programs remain completely location opaque. STACK objects are always co-located with their thread objects; however, they still are DCOM objects. There is no requirement that execution be "procedure based". The interpreter can evaluate formulas with only a stack and a code string. When a thread is marshaled to be transferred to another machine, the stack content, including return addresses, is converted to a portable format. In the COM model, objects are normally created by server front-ends called class factories. The separation of request and creation on a per-type basis provides a way to create many different types of servers for each object type. Remember that in COM an object is defined by the interfaces that it supports, not by its data structures or algorithms. 4.1 The thread object In the current implementation, a THREAD object contains a code string, an IDispatch object handle to a context object, two object handles to a stack object, and type information (used by IDispatch, described later). The IOperator interface defines methods such as Add and Subtract; the IProcedure interface defines methods such as Frame and Return (used for procedure call/return). The latter interface may be omitted for calculations that do not involve procedure calls.
Figure 6. Obtuse Program Objects When a thread starts execution, it binds to an IObject handle. For efficiency (since a context holds global variables), the IObject interface was compiled by the IDL compiler to generate proxy stubs. As a result, references to a remote context, must pass through a proxy DLL, which must be registered at that site. Accessing global variables using the IObject interface is much faster than using IDispatch. Figure 7 lists the IThread interface, which contains methods for creating a thread, marshaling its state, and controlling its execution. Migrating a thread or object depends on support for marshaling its state. In theory, any system, such as C++ or Java, could support migration. 4.2 The context object A context object is a vector of global variables. Since all variables in Obtuse are represented using the variant data type, a context is just an array of variants. Further, since an array of variants is also a variant type, a context is marshaled automatically by DCOM when passed from one machine to another.
Figure 7. The IThread Interface The IObject interface, which is listed in Figure 8, describes the methods to store and retrieve Obtuse variables. The Get/Put methods access simple variables; GetIndex/PutIndex access arrays. Since all Obtuse variable locations are the same size, variable addresses are just indices (e.g. 0,1,2 etc.). Array access is implemented by passing the entire subscript list as an argument. This approach is more efficient for remote access than evaluating one subscript at a time. In Obtuse, array assignment is "by value". The only way to generate a "reference" in Obtuse is by creating a COM object. The context class interface contains a number of helper functions (save, restore, persist, sweep) that are intended only for local use. The "save" and "restore" functions are used to clone a context. The "persist" helper function toggles a flag that indicates whether a context should be retained after its thread terminates. This option is useful for debugging and also for writing programs that inter-operate by passing contexts back and forth. Used in this way, a context is somewhat like a COMMON block in FORTRAN.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Technical Program