a Tcl story

Guy Bobenrieth is working as a software engineer for JWay, in Luxemburg, building solutions based on SGML/XML for desktop and Web applications.


One of our customers wanted a solution that would achieve batch processing of SGML documents having a database updated to trace the documents' status.

Like other word processors, WordPerfect can do a lot for you in text handling, and it can also process SGML documents -- from creation to printed outputs according to style sheets.

WordPerfect integrates a macrolanguage that can be used to customize its behavior (even to build dialog boxes). But in this case, our needs weren't limited to document processing: they included the ability to access an Informix database in order to know which documents to handle and the ability to update it.

WP's macrolanguage is not aimed at addressing such tasks without extension. Rather than investing in building an extension to a proprietary language (which could be used only within WordPerfect), we took a look at alternative solutions.

Because of WordPerfect's OLE server capabilities, it was obvious to us that our solution could be based on another language that would better meet our needs. Tcl seemed to be a good option. One attractive aspect of Tcl is its high integration capability with other applications. Furthermore, we use it to process SGML documents, activating the Web pages, etc. Our glue was found.

To gain the benefits of speed and better Win platform integration, we chose Tcl 8.1a1.

After designing our application's basis, we looked at the two things that were not built into Tcl : database connection and WordPerfect handling.

To remain independent of the database, we decided to work with ODBC. After searching the Web for database tools and testing several ODBC extensions for Tcl, we chose TclODBC15 from Roy Nurmi <Roy.Nurmi@iki.fi>: retrieving, inserting, and updating data worked fine. It's a great piece of work.

Unfortunately, a problem showed up. In Europe, we use a lot of characters like the ones with accents, and strings with such characters coming out of the database were not string compatible with Tcl's strings. After we got in touch with Roy about that, he experienced these problems, too. TclODBC15 was designed and tested on Tcl 7.6 and not on Tcl 8.1, which has special string handling based on UNICODE.

Roy decided to rewrite/update TclODBC to meet Tcl 8.1 needs, i.e., supporting the UNICODE handling. In the meantime, we found a way to work around this problem. I spent some time exploring the Tcl 8.1 source, especially the parts on string handling. I found that the scan instruction could convert our strings from ODBC to the Tcl 8.1 format, making the string comparison work well: the DB solution was up and ready to work.

Now it was time to work on the WordPerfect side. At first we thought that DDE calls would be powerful enough to have our Tcl applications deal directly with WordPerfect. But DDE calls were suffering some lack of power and stability. We decided to see if it was possible to bind Tcl and WordPerfect with OLE.

After looking at the different extensions available for Tcl, we realized that nothing was available to do what we needed to do. There were extensions for visual objects like widgets, but nothing that would allow us to control WordPerfect. Was it time for us to try to build an extension for Tcl on a Win platform and do it from scratch? We gave ourselves a week to try to get it done!

The first thing was to see just how to build an extension on a Win platform for Tcl. This was relatively easy because there are a lot of examples on the net. We decided to build it like a new instruction with several available options.

The principle was simple: after being loaded, the extension creates a new instruction that is registered by the Tcl core. Then each time the instruction is used a special procedure tests the option, seen as parameters to the instruction, to see what secondary procedures should be called.

Then we needed to build an interface for WordPerfect's OLE server. This was our main problem. Generally, working with OLE servers is achieved by using a C++ interface that hides the complexity of OLE technology. But we wanted to have access to these inside mechanisms: this drove us to spend the largest part of our efforts on this.

The binding is rather primitive: we instantiate our object and then have the ability to call its methods by using the Tcl instruction's options. Only the most needed methods are built inside the extension (e.g., opening a file, printing it, starting a WordPerfect macro). But the others are also accessible, through a generic method caller.

One of the problems we encountered then was to map the Tcl types to WordPerfect's methods parameters. Fortunately, these methods were using only two types of parameters: string and long int. When WordPerfect is called from Tcl, it acts in a proprietary way. For example, you may not quit the application from inside its menu; only Tcl script that called it can do that. For the best results, you would have to use the extension under the Tcl graphical interpreter WISH. This is needed so that the Win events are parsed correctly by Windows.

After testing our new extension and getting WordPerfect to open documents, run some macros, and then printing them (all driven by Tcl and logged in our database), we needed to conduct some more tests to establish whether it was stable and usable.

At that moment, we thought it could become an Open Software to be contributed to the Tcl community and thus gain sufficient feedback to initiate refinement cycles if so requested. To do this, we have made the extension available on the net under GNU license and opened a discussion group on one of our Web servers.

After the announcement in the Tcl newsgroup, the extension was downloaded about ten times, but we were only partially successful in collecting feedback. Nevertheless, there was a common request for a more general extension based on the OLE principle that would open Tcl to a great number of applications based on OLE.

This would be a nice feature to be added in the future, but for a proper implementation more people should be involved. If there are volunteers, we would be glad to coordinate the work and participate actively. Co-workers are welcome to contact us (email: <gb@mail.jway.lu>).

In conclusion, Roy Nurmi has recently completed his new version of TclODBC2.0, completely integrated with Tcl8.1. It should be available soon on the net <Roy.Nurmi@iki.fi>.