Extending W3 Clients

Alan Falconer Slater,
Department of Computing and Electrical Engineering,
Heriot-Watt University,
Edinburgh, Scotland.

1. Introduction

World-Wide-Web (W3) browsers, such as Mosaic, are powerful tools which have been proven to be directly applicable in a large number of domains. However, although browsers can support localised customisation of the mechanisms used to display particular types of data through the use of facilities such as MIME, the internal structures and processes of systems are typically not exposed in a manner that supports more radical modifications.

This paper firstly addresses what benefits may be gained through the provision of mechanisms for the extension of W3 clients before examining how these additional software components may be realised.

2. Examples of Client Extensions

This section examines some possible applications of client extensions. Almost all of the examples discussed have corresponding demonstrations which have been produced by modifying the distributed Mosaic source code to provide access to internal variables and to invoke the external programs which implement these examples. The problems with this approach to client extensions are discussed later.

The set of examples provided does not attempt to completely explore the possibilities provided by client extensions, it merely attempts to demonstrate that there are tangible benefits to be gained from the provision of client extension mechanisms.

2.1 URL Processing

When a user indicates to her W3 browser that she wishes to follow the link from the anchor object to the associated resource the client usually attempt to access this resource in a manner which directly conforms to that specified in the document. One area where extensions may be included is to allow the processing of the supplied URL before it is resolved by the browser.

2.1.1 Redirection

A standard technique in the distribution of resources is to mirror a central set of resources at a number of widely separated sites, with each site acting as a local cache for some geographical area. Users do not usually have much control over the strategies used to access such cached resources. Through local processing of URLs, client extensions provide users with direct control over how resources are accessed.

Server & Path Replacement

If it is known that some expensive resource is mirrored at some convenient location then a simple string substitution can redirect all links. For example, if I access many documents that contain images from the collection at Stanford, then I would probably find it better to redirect links to these images to a mirror site which is closer to Edinburgh:

process_url(http://www-pcd.stanford.edu/gifs/logo.S.gif) ->
	http://www.eee.strath.ac.uk/images/stanford/logo.S.gif

HTTP to Local File

In some situations it may be possible to retrieve a considerable amount of information from files that can be directly opened by the client. For example, a reference to the author’s home page could be translated directly to the appropriate file path:

process_url(http://www.cee.hw.ac.uk/~afs/) ->
	file:/u1/staff/afs/www/index.html

This particular type of mapping could be used where W3 documents have been downloaded to an appropriate storage medium, such as CD-ROM, and used on a machine which does not have suitable net access . This technique would also have the advantages of not requiring any modifications to the original documents and possibly negating the requirement to have a W3 server running on an isolated machine.

2.1.2 Monitoring

As well as providing access to equivalent resources from a different source, URL processing may be used to significantly alter the behaviour implied by the hypertext link.

Warnings

A particular user may decide that she would like to be warned before accessing URLs which match certain criteria. Possible causes for concern could be shell scripts, links to certain servers, or URLs which contain port numbers which indicate that they are probably not controlled by system administrators. The URL processor must create a new document, which contains both a suitable warning and a link to the original document, and return the URL for this new document rather than the original.

Access Denial

In some cases it may be desirable to configure a particular browser so that it cannot access certain resources. This may be simply done by checking the supplied URL and resolving it only when it meets the appropriate criteria, with a URL to a document containing a suitable message being substituted in all other cases. This facility could be used to limit a publicly accessible browser to appropriate material, perhaps limited to a particular organisation or geographical area.

2.2 Restructuring HTML Documents

Once a browser has retrieved an HTML document, it may be desirable to modify the structure of the document before it is displayed to the user. Extensions of this form can be viewed as filtering the received HTML, possibly involving the creation of new local documents containing processed elements of the original.

2.2.1 Outlining

A user may choose to have certain documents, perhaps those which exceed a certain size, split into a number of sub-documents. These sub-documents will be formed by collapsing all the text below a certain heading level into a new document, leaving only the heading as the anchor text for a link to the sub-document. This particular filtering process is comparable to the outline view of documents which most word-processors provide.

2.2.2 Anchor Text Only

Another view of documents that can easily be supported by client extensions is that of only displaying the anchor text for links, with an option to display the URLs too. This could be useful for manually searching a set of documents for a particular link.

2.3 Supporting Alternative User Interfaces

Client extensions can also be use to experiment with the interface between users and W3 browsers. In particular, we have been examined the possibility of using W3 technology within public access information points. These systems are usually expected to exist in environments where operating conditions may be inappropriate for the use of mice or conventional keyboards.

Figure 1. A prototype push-button W3 client interface

Ideally, a W3 client used in such circumstances should not be limited to documents that have been specifically constructed to suit its attributes, but should be capable of displaying any HTML document. It is therefore necessary to process the supplied document into a form suitable for a given client. A typical system may only support a small screen size and a simple set of numbered input keys. To satisfy these constraints it is necessary to modify the original document in the following ways:

Label each link with a conspicuous icon which corresponds to a single input button;
Split large documents into suitably sized 'chunks'. Documents being split when they exceed the size of the screen or when the number of links is greater than the number of input buttons. A prototype of such a system, produced using extensions to Mosaic, is shown in Figure 1.

2.4 Local Document Generation

It is common for CGI scripts to use mechanisms available at the server machine (databases, search engines, image renderers and speech synthesisers being examples) to generate resources which are then delivered to the client. It should also be possible to execute programs in the client's environment to generate comparable results. An example of this is the generation of an HTML document that describes the processes a user has running within the UNIX environment. This example is shown in figure 2.

2.5 Link Generation

In the W3, links are embedded within HTML documents by explicitly associating a resource with an anchor object. Other systems, such as Microcosm [1], adopt a more flexible approach where links are computed by a separate mechanism when requested by the user rather than hard- coded by the author. In the Microcosm system, link information is held separately within linkbases (link databases).

We are currently experimenting with the additional of such facilities to Mosaic through the use of separate ICF programs that use the PostScript-like extension language used by the ICF, the ICL, to compute links from text selected within Mosaic. A stack of dictionaries is used to contain the mappings between objects and URLs - the dictionary stack being searched from the top down to find the most relevant entry. When a matching entry is found the program uses the ICF to send Mosaic to the appropriate URL.

Modifications to the dictionary stack can be made by using the ICF to send messages from Mosaic to the dictionary program to push a dictionary relevant to a new document or to dispose of dictionaries which are now inappropriate. The current implementation of this mechanism relies upon the user to select the link that will send the ICF message to update the dictionary program. Ideally, the dictionary mechanism should monitor the current state of the client and automatically reconfigure its dictionary stack. The greatest problem with this approach appears to be the identification of the domain to which a given document belongs and therefore which dictionaries should be used together with the order in which they should be searched.

2.6 Interfacing with Application Programs

W3 clients may form components of more specialised software systems. In this case it is important to support bi-directional communication between clients and programs running in the same local environment.

Ideally, W3 clients should directly support the following features for communicating with other applications in the local environment:

Messages, comprised of commands and data, should be included directly or as links within HTML documents;
Data from fill-out forms should be made available for processing by local programs;
Other programs should be able to access the state and processes of the client.

Other programs should be able to query the client to determine attributes such as the current document and navigation history. Access to client processes should allow programs to directly control the client.

Most of these functions, with the exception of allowing other programs to examine the state of a client, are currently possible by using the X implementation of Mosaic in conjunction with the ICF library [2, 3].

A simple example of how locally generated documents can be combined with local processing of form-data to provide a user-friendly means of accessing the UNIX 'ps' and 'kill' commands to deal with unwanted processes, as shown in figure 2. Although this example is quite easy to implement using the ICF, it involves the execution of a separate program to generate the process listing and to then process the data in this form to send the appropriate signals.

Figure 2. Listing and killing local Unix processes

2.7 Internal Data Visualisation

W3 clients such as Mosaic, as with most programs, can only internally process and display a fixed set of data types. It would not be practical to build into a client the means the cope with all the possible file types which could possibly be delivered to it. Indeed, the use of MIME types in Mosaic ensures that appropriate external programs can be employed when required and that the mapping between data types and these viewer programs is quite easy for users to customise. However, although the use of external programs provides an elegant software solution it is arguable that this does little for the usability of the resulting system. Browsing a large set of images can all too easily leave a user floundering in a sea of xv windows.

W3 clients should be able to employ extensions, which may still be separate processes, to render images directly into documents, this idea being taken from the Development Priorities workshop at Geneva [4]. External images will, of course, still be preferable in cases such as when the image is particularly large or is otherwise unsuitable for inclusion within a document, but the decision should be made for presentational reasons rather than to allow for the capabilities of a particular client.

3. The Interface to the Client Program

The techniques used to construct the prototype client extensions described in the previous section suffer from a number of serious problems:

The programs run as separate Unix processes and use relatively primitive techniques to communicate with Mosaic (a mixture of temporary files, pipes and signals);
The Mosaic source code had to be modified to permit access to internal data structures;
Large numbers of documents are created in the user's public W3 directory.
The extension programs are not invoked selectively. As these programs are used as filters for data within Mosaic they must explicitly handle those cases where they are not appropriate by simply acting as identity functions and passing their input directly to their output.
As these extensions are implemented as separate programs their existence is all too apparent to the user (i.e., the machine on which the client is running must have the extensions installed, execution paths must be set correctly, etc.). Any attempt to design a standard environment for W3 client extensions must address the underlying causes of these problems by defining standards in the following areas:
The interface between W3 clients, their extensions and other applications;
The environment for executing extensions;
The method used to deliver extensions to clients. The interface definition would effectively define an 'open architecture' for clients that would provide a standard abstract external view of the data-structures and processes used by the client.

4. Portable Extensions

This section provides an overview of a proposed framework for the provision of portable W3 client extensions. Some novel attributes of this proposal include:

The interface between the client and other software should be object-oriented rather than procedural;
Client extensions should be expressed in a form which is portable but which does not restrict programmers to a single scripting language;
The code forming the executable form of client extensions should be delivered to clients using the W3.

These areas are discussed in greater detail below.

4.1 The Client Interface

In their discussion of the factors relating to the design of OO class libraries, Kiczales and Lamping [5] identify two important system attributes, generality and extensibility: generality refers to the ability of one system, without modification, to server in a large range of circumstances, while extensibility refers to the ability of a system to be easily modified to better meet a particular need.

W3 clients can therefore be seen to be tools which already possess generality but which do not have a standard method for providing extensibility. Object-oriented design and programming arguably provides the best basis for the construction of extensible systems as one of its defining features is the intention to build reusable software structures. According to Kiczales and Lamping:

The first task in specifying an extensible system is specifying what the replaceable units are: What are the pieces that can be changed to affect the behaviour of the system as a whole?

In the case of extensible W3 clients, we should be able to specialise existing system-defined classes (such as URLs, HTML documents, images, etc.) and define additional localised behaviour for these new classes. An important element of any such interface is that it should expose sufficient structure of the client to facilitate extensibility but should prevent any extensions that compromise the functioning of the client by prohibiting the redefinition of existing client classes or methods [5].

The Common Lisp Object System (CLOS) [6] incorporates a number of features that make it, certainly in the experience of the author, the most effective environment for developing extensible software. In particular, the following features of CLOS are of particular interest:

Methods, which implement system behaviour, are associated with tuples of classes rather than with a single class. These multi-methods allow methods to specialise on any number of parameters. Neither C++ nor Smalltalk directly support this feature [7, 8].
Invocation of a CLOS generic function results in the computation of all the applicable methods and their invocation in a manner that can result in a number of methods being executed for a single call to a generic function. Additional flexibility is also provided through :before, :after and :around methods.

It is suggested that a standard W3 extension mechanism should provide an object-oriented interface to extensions and that methods that implement the behaviour of the extensions should be invoked through a mechanism that is similar to that used by CLOS. Of course, it is not suggested that this mechanism should incorporate anything like the full functionality of CLOS, merely that the features described above provide an unusually powerful basis for the construction of extensible software.

4.2 Other Proposals for Client Extensions

There are a number of existing proposals for client extension mechanisms, two notable examples being the Mosaic Common Client Interface (CCI) and the W3 Applets API (W3A)[10]. It is interesting to note that these works reflect two different approaches to extending clients. The CCI defines a communication protocol between Mosaic and other programs which appears to be at a level which is comparable to HTTP. The draft CCI envisages using data delivery mechanisms such as TCP/IP, OLE and AppleEvents. The Mosaic CCI does not address issues such as how extensions will be implemented or delivered.

W3A presents a very detailed proposal for the procedural interface between a client and its extensions (termed applets). Specifications for the data structure which must be supported by this interface are also provided. W3A applets are assumed to be modules of object code which can be dynamically linked into the client during execution, implying that extensions must be specific to the architecture of the machine upon which the client is executing.

It is clear that both the W3A and the Mosaic CCI proposals are closer to realisation than the approach outlined in this paper. However, it is arguable that such an important area as client extensions should be approached in a manner that is consistent with the overall approach of the W3, with portability of resources being of paramount importance.

4.3 The Extension Execution Mechanism

The selection of the 'best' programming language for any situation is one that is guaranteed to provoke heated debate. Discussion of the choice of a suitable scripting language for W3 clients has already produced many suggestions with no candidate appearing to be a clear winner. A number of observations can be made regarding the choice of a suitable language:

The extensions should be portable across platforms and between clients ;
Some platforms have a 'natural' language - Visual Basic under MS Windows and AppleScript on the Macintosh;
For almost every programming language there tends to be a sizeable group of vociferous advocates;
Imposition of a single standard language will probably upset large numbers of people, who will rightly object that their favourite language is the best tool for their problems;
Some extensions could conceivably become large application packages and would represent a considerable financial investment. If they ever become marketable items then it is probably unlikely that vendors would wish to distribute their products as source code.

The desire for a portable medium for expressing client extensions can be justified by one of the factors that has contributed to the success of the W3 itself: the portable nature of the HTML. The other observations suggest that the best means of distributing this type of software is not at the level of source code but at a level that is more akin to object code. The Open Software Foundation's Architecture Neutral Distribution Format (ANDF) is very close to what is needed [11]. However, the ANDF is much more ambitious in scope, apparently requires some time to convert from the distributed form to an executable and, unfortunately, is not yet available. It is proposed that a suitable object code be defined for a virtual machine and that each client should incorporate an appropriate interpreter that is capable of simulating this hypothetical machine. Compilers for the desired source language would then be used to produce extension modules.

4.4 Delivering W3 Client Extensions

If extensions are implemented in a form that can be used by a variety of clients, and on a range of platforms, it is then natural to distribute extensions in a similar way to any other resource on the Web - identified by a URL and retrieved via HTTP. Extensions could be retrieved as and when they are required by the client, providing a transparent means of incrementally extending the client as needs dictate.

It could also be possible to distribute the compilers for the virtual machine using this technique - thereby providing support for distribution of extensions as sources in any supported language.

5. Conclusions

This paper examines the benefits to be gained from allowing W3 clients, already general tools, to include mechanisms for extending their functionality. A number of prototype client extensions are presented together with a tentative proposal for a novel generic extension mechanism would allow the software implementing client extensions to be distributed on the W3 in a manner similar to any other resource.

6. Acknowledgements

I would like to thank my colleagues at Heriot-Watt, Cambridge and Strathclyde Universities for contributing much of the background for this work. However, responsibility for factual errors and silly ideas remains totally with me!

I would also like to thank Heriot-Watt University and the SERC/DTI MOBIT project for sharing the costs involved in attending this conference.

7. References

1. Microcosm Project (1994),
Microcosm - The Linking Engine,
http://ilc.ecs.soton.ac.uk/microcos.htm

2. Slater, A.F. (1994),
Controlled by the Web,
Proceedings of the First International Conference on the World-Wide Web,
CERN, Geneva, Switzerland.
http://neptune.cee.hw.ac.uk/~afs/www94/www94.html

3. Smeaton, C. & Slater A.F. (1994),
Integrating Simulations and W3 Courseware,
Paper to be presented at the Second International Conference on the World-Wide Web.
http://neptune.cee.hw.ac.uk/~calum/wwwfall94/wwwfall94.html

4. Berners-Lee T. [Ed.] (1994)
, Development Priorities BOF
First International Conference on the World-Wide Web, CERN, Geneva, Switzerland.
http://info.cern.ch/hypertext/WWW/WorkingNotes/WWW94/DevPriBOF.html

5. Kiczales, G. & Lamping J. (1994),
Issues in the Design and Specification of Class Libraries,
OOPSLA '92.

6. Paepcke, A. [Ed.] (1993),
Object-Oriented Programming - The CLOS Perspective,
MIT Press.

7. DeMichiel, L.G. (1993),
CLOS and C++, in [6].

8. Cointe P. (1993),
CLOS and Smalltalk, in [6].

10. Boss, B. (1994),
An API for WWW Applets,
http://www.let.rug.nl/~bert/W3A/W3A.html

11. List of ANDF papers,
OSF,
http://riwww.osf.org:8001/ri/andf.papers/toc.html

About The Author

Alan Slater is a Research Associate in the Department of Computing and Electrical Engineering at Edinburgh's Heriot-Watt University. He is currently employed on the SERC/DTI MOBIT project which is developing industrial Intelligent Training Systems in collaboration with a number of industrial partners. Alan has also worked on a number of large European projects, funded under the CEC's ESPRIT programme, investigating model-based diagnosis and ITSs. During work on the INTERACT TLTP project, which aimed to produce an environment for delivering educational simulations, he became interested in hypermedia tools and the W3. He is currently in the final stages of preparing a PhD thesis which is based on his work in the field of ITS systems. This thesis identifies a generic architecture for intelligent agents which is used to provide the basis for realising the two main modules of an ITS system: the domain expert and the tutor.

As part of the MOBIT project he is examining the possibility of delivering computer-based training material via W3 tools, the training domains for MOBIT being control of a nuclear power plant (with Scottish Nuclear) and the manufacture of computer systems (with Digital Equipment Scotland Limited).

afs@cee.hw.ac.uk
http://www.cee.hw.ac.uk/~afs/