[[page created automatically from word-processed document; for original see: Postscript version]]
Hypertext: dreams and reality
P.J. Brown
Computing Laboratory
The University
Canterbury
Kent, CT2 7NF
C Copyright P.J. Brown 1989, 1990
The aim of hypertext is simple and practical: to create a better world for readers of on-line documents. There is a danger, in any field of research and development, of the researchers moving into a dream world, far divorced from the real needs of users. This can happen throughout computing research -- and doubtless equally in other disciplines -- even when the topic is severely practical. It happens in relatively mature disciplines, such as software engineering, and is certainly not confined to new and fashionable disciplines like hypertext.
Often the dream world is encouraged by funding agencies, who push grandiose and ambitious projects. When these fail no-one wishes to acknowledge it, least of all the funding agency, and thus further even more grandiose projects are built upon the failure of the earlier projects.
We could all gain by following Ivan Sutherland's (Frenkel, 1989) wisdom: ``It seems to me the secret in research activities is to pick something easy enough to do, ... but a lot of people pick projects that are too hard. I've tried to pick easy ones to work on. And there are plenty of easy ones going begging''.
The purpose of this paper is to act as a counter-weight to the dreams and hype surrounding hypertext. It takes a real project with down-to-earth aims, and examines how the lessons learned from that project throw light on the issues facing hypertext.
Just as you can prove anything with statistics, you can prove anything by choosing the right projects as examples. Too many `research advances' in computing are justified by toy examples which are so small and so over-simplified that all the real issues are avoided.
A more subtle effect comes from a project being the first in the field, as any project that exploits a new research technique will inevitably be. There is extra excitement and commitment from the people involved (including the so-called Hawthorne effect), and funding criteria may be different -- the project may not need to justify itself economically, but instead may be regarded as a research exercise.
The project chosen for this study is the Locator project, undertaken by ICL at Stevenage in collaboration with the University of Kent at Canterbury. It is certainly not a toy project and had strictly economic funding criteria, and competition from other groups. It was, however, a pioneering project and, during development, was blessed with excellent and well-motivated staff. The overall results may therefore be better than a normal project, though, to counter this a little, the project faced, during its application stages, the familiar problems of ``We have done it this way for 20 years. Why should we change to your new approach?''.
The project involved many ICL staff, but four of the leading roles were taken by George Rouse (the overall manager), Brian Brough, Des Meehan (who deserves huge credit for technical and implementation matters) and Mike Thomas. Further details are given in Meehan (1987).
The project is concerned with what ICL call laundering: dealing with hardware fault reports from customers. A team of diagnosticians sit at telephones (they were originally called `launderers' before the grander title of diagnostician was used) and receive calls from customers; their aim is to elicit enough information from the customer so that
Prior to Locator, the laundering system was based on shelves of paper manuals with numerous cross-references (``If it is a printer problem go to Section ... of Volume ...''), plus the diagnosticians' experience and expertise. The aim of Locator was to provide a better way for the diagnosticians to find their way round the information. Locator did not therefore set out as a project to test hypertext techniques; instead it was a project to solve a problem, and hypertext techniques turned out to be (part of) the solution.
The hypertext system used was Guide, in its implementation that runs on UNIX workstations (Brown, 1989a), in this case SUNs. (There is also a widely-used implementation of Guide that runs on IBM PCs and Macintosh machines. This was developed by Office Workstations Ltd. (OWL), based on the original ideas from the University of Kent. It differs in detail from the UNIX implementation -- every hypertext system must fit its environment -- but is similar in principle.)
The standard version of UNIX Guide was specially tailored to meet ICL's needs. For example, the normal menu was cut down and made more specialised to the laundering environment.
With the Locator system the diagnostician works exactly as before, except
that the documentation, instead of being on paper, is now displayed by
Guide on a workstation screen.
When a customer phones to report a fault the diagnostician views the initial screen shown in Figure 1.
The big window on the left-hand side is running Guide.
(We shall discuss the rest of the screen later.)
The Guide window contains replace-buttons
(shown in bold, e.g. Acoustic Hood) corresponding to the types of
fault the customer may report.
Replace-buttons are used in Guide to expand detail: a button is replaced
in-line by more detailed material that relates to that button.
.KF
.rs
\^
Figure 1: the complete screen with the initial Locator window
Assume the customer says the laser
printer is the problem. The diagnostician then selects the LASER Printer
replace-button by pointing at it and clicking the mouse.
This then leads to the screen shown in Figure 2. In fact
the whole of the document seen in Figure 1 is a Guide enquiry: a
collection of replace-buttons with interspersed text/graphics that is treated
as a unit. If any button within the enquiry is selected, the whole
enquiry is replaced by the button-replacement of the selected button.
.KF
.rs
\^
Figure 2: the effect of selecting the LASER printer button
In Guide the document that the user sees appears as a scroll. Selecting replace-buttons within the document causes the content of the scroll to change, and reveals information previously hidden (folded) behind the button. The user continually refines detail by selecting the appropriate replace-button until the scroll contains the desired information. This scroll model is different from the frame-based model exemplified by HyperCard, and we shall discuss the implications of this later. However, if, as in Figure 1, a Guide enquiry occupies the whole screen, then Guide behaves like a frame-based system.
Replace-buttons are used to build a hierarchy into a document. Hierarchies are an important facet of many hypertext systems -- indeed some only offer hierarchies -- and we shall discuss the importance of hierarchies later.
In Figure 2 the screen consists of one line of text (`A.1 LASER Printer')
followed by a further enquiry. The diagnostician need not, of course, be
aware of mechanisms such as enquiries, or, indeed that he/she -- we shall
here assume she -- is using a hypertext system. To her, selecting buttons
causes the appropriate text to appear magically.
.KF
.rs
\^
Figure 3: .. after a further reply from the customer
Figure 3 shows the result of selecting the `Operator Panel SYMBOL FLASHING' button in Figure 2. This in turn leads to an extra line at the top of the screen to give the customer's second response, and a further enquiry. (The extra lines at the top of the screen are provided by Locator to tell the diagnostician how she reached the current position; it is a disciplined use of Guide's facilities; in general a button-replacement can take any form.)
A design aim of Guide is that it should contain a small number of facilities. Reading from a hypertext system will never be quite as simple as reading from paper, but you want to get as close as you can. To the diagnostician Guide contains just four facilities: the menu, two types of button -- one of which we have just covered, and `undoing' (explained later).
The other type of button is the glossary-button. Glossary-buttons
are underlined on the screen, and are used for such purposes as
explaining jargon terms, footnotes, citations, etc. The button-replacement of a
glossary-button does not appear in-line, in place of the button, but
instead appears in a separate sub-window (in OWL's Guide this is an
ephemeral one). In Figure 3
HELP
is a glossary-button, and its
button-replacement is a picture of the laser printer showing where the operator
panel is. Figure 4 shows the effect of selecting it.
.KF
.rs
\^
Figure 4: .. after selecting the `HELP' glossary-button
When you move from the research laboratory to real use you always hit vagaries of human behaviour. In Locator, three such aspects of customer and diagnostician behaviour were of special importance.
Firstly, many of the Locator customers are naive in computer terms and often change their mind during a conversation (``Did I say I had a laser printer? My colleague now tells me it is a daisy-wheel printer.''). Secondly, and more frustrating still, some of them do not know the answers to `simple' questions and have to go off and find out; they then might call back later, perhaps the next day, and expect the diagnostician to resume the conversation where she left off.
Thirdly, ICL maintains its service even if, for example, all the diagnosticians are struck down with food poisoning on the same day. In such cases a new partially trained stand-in diagnostician needs to be brought in.
This last problem is ameliorated by Guide's minimal set of features. A new diagnostician can be using Guide effectively after a quarter of an hour. The same is true of almost every other hypertext system: the user can be presented with a minimal set of facilities so that training is no problem.
We shall now discuss the first problem: the customer changing his mind.
UNIX Guide assumes a mouse with at least two buttons. One is used for `going forwards', i.e. selecting replace-buttons or glossary-buttons, and the other is used for `going backwards', i.e. undoing any previous selection. The exact rule is that if you point at any material on the screen and click the `undo' mouse-button, then Guide undoes the button selection that caused that material to appear. (Thus undoing does not have to be done in the same order as button-selection; it is easy to undo any previous selection.)
The design of the Locator button-replacements, where the first line gives the customer's decision, makes undoing particularly simple. The diagnostician just points at the wrong statement and clicks the `undo' mouse-button.
More generally, if the `undo' mouse-button is held down a moment before
releasing it, a pop-up menu appears. This shows the diagnostician the current position in
the hierarchy, i.e. all the button selections that caused the pointed-at
material to be reached. Figure 5 contains an example of this. The pop-up
menu can be used to undo any of these button selections. As the user
moves up and down the pop-up menu, Guide highlights the
corresponding button-replacement that might be undone; on a monochrome
display, as in our Figures, highlighting is done by inverse-video.
.KF
.rs
\^
Figure 5: the pop-up menu to allow `undoing' of previous decisions
The undoing facility caters well for the customer who changes his mind, and it does not even matter if he changes his original answer after giving several subsidiary answers. It represents the third of the four Guide facilities the diagnostician needs to know about.
Eventually the goal of the customer and diagnostician will be reached and the cause of the fault will
be resolved. This is shown in Figure 6, a Guide screen with no
further questions. If the customer is satisfied with this
conclusion, the diagnostician then selects Resolved from the menu,
and Locator is ready for the next call. The diagnosis of the previous call is
automatically saved so that the next stage of curing the customer's
problem can proceed, e.g. sending out an engineer with the appropriate parts.
.KF
.rs
\^
Figure 6: the fault is diagnosed
We mentioned that calls may need to be `held' while the customer finds out some information; when the customer phones back such held calls are resumed where they left off.
The diagnostician places a call on hold by selecting On-hold from the menu. The call is then stored as a icon representing a Guide session that can be resurrected later. Figure 1, which shows the whole Locator screen, contains three such icons, labelled `Printer switch', `Checking power' and `Consulting his manager'.
If the workstation is closed down overnight and then re-booted the next morning, Guide automatically saves the state of its `on-hold' sessions, and restores them when it starts again the next morning.
As we have said, Locator uses a specially tailored version of Unix Guide. This contains extra mechanisms to cover the `On-hold' and `Resolved' facilities. There are other changes, mostly concerned with removing facilities that are not needed for this application so that they do not confuse the diagnostician.
All these changes are directed towards making the hypertext system sit naturally in its working environment. Certainly the message from the market-place has been that each application has different needs. As a result UNIX Guide now contains much fuller facilities for tailoring of menus and other objects.
The above is a somewhat oversimplified description of the hypertext technology used in Locator. Although we have given the impression of a tree structure the underlying data structure is, in fact, a more elaborate directed graph, since the same goal can be reached by many routes, as we shall discuss. Nevertheless there is nothing special in the hypertext technology used. Instead the interest of the project is to throw light on some of the issues currently facing hypertext, and to gain from some of the excellent skills shown by those concerned with authorship and design.
In the rest of this paper we shall focus on seven issues facing hypertext. These issues are taken from Brown (1988), though no knowledge of that paper is assumed.
No user wants a hypertext system per se. The user wants a solution, and, at best, the hypertext system will only be part of that solution. It is vital therefore that the hypertext system be able to work in conjunction with other existing tools. Such tools may cover:
UNIX Guide uses troff-style mark-up to represent structure (e.g. a .Bu request in a Guide document marks the start of a button). The reason that this, albeit imperfect, form was chosen is that it is a UNIX standard. The UNIX spelling checker, for example, works naturally with this format. The Locator project used a wide range of existing and newly-created UNIX tools and this standard format has been invaluable.
A further facet of integration is that it can be intimate in the sense that the hypertext tool may run simultaneously with other tools in a single seamless package. In Locator, for example, Guide works intimately with some software that logs telephone calls. More ambitiously, hypertext systems would gain greatly from working seamlessly in tandem with drawing programs, editors, expert systems, etc, (Meyrowitz, 1987).
The biggest opportunity arising from hypertext is for creative authors to present material in an entirely novel way, exploiting the new medium rather than using techniques taken from paper documents.
Sadly, however, large hypertext projects rarely start from scratch. Instead there is normally an investment in existing documentation, designed for paper but usually stored inside a computer in some word-processor format. This investment needs to be protected, and, indeed, the cost of re-gathering the material would often be prohibitive. Thus a largish project that really does start from scratch, such as Glasgow Online (Baird and Percival, 1989) is an unusual opportunity.
Capturing existing documentation consists of two stages:
Stage (2) is expensive, but, in the Locator experience, worth a good deal of investment. As Oren (1987) has remarked, if you just take an existing paper-based document and apply some automatic conversion procedure to it you add nothing.
Quality assurance in Locator is multi-faceted. Comparatively easy matters are spelling checks (use an existing tool) and linkage checks (use UNIX Guide's option which, on loading, checks that all glossary-buttons have corresponding definitions and that all cross-references lead somewhere).
More difficult is testing the quality of the material:
Equally difficult is testing the mechanics of usability:
There are, of course, no easy answers to these questions any more than there are easy answers to producing good paper documents. A pre-requisite to achieving anything, however, is to have a method of recording sessions so that the problems can subsequently be analysed.
In Locator, one of the problems that arose was wholly unexpected. When diagnosticians become adept at using Locator, they are often tempted to explore by selecting possible buttons before the customer has answered the question. This sometimes leads to conversations getting out-of-phase. Perhaps it has been made too easy to explore documents.
In the field of programming it took a while to learn that large programs are inherently different from small programs. In a large program complexity dwarfs all other considerations, and unless one's methods successfully tame complexity all is lost. In hypertext we are still exploring the limits.
It is undoubtedly true that in hypertext, as in programming, discipline is vital in taming complexity. The haphazard author, even if blessed with flair and imagination, is a menace.
The Locator hyperbase is of medium-to-large size -- a few megabytes of text, plus some pictures. A key to the success of the project is that authors have used a strict discipline throughout. As we have seen from the above Figures, material follows a prescribed form. During the project a strong house-style evolved. Following Engelbart's (1987) suggested `hypergrammars', this house-style was designed specifically for the Locator project rather than for every hypertext application within ICL. Discipline helps both readers and authors. It is boring, anti-creative, but necessary.
We have explained that the Guide user sees a document as a continuous scroll, which expands and contracts as buttons are selected/undone. This is unlike those hypertext systems that work in units such as cards, pages or frames. Guide's approach is not inherently better or worse than the card-based approach. If the underlying information is naturally divided up into separate card-sized units, then the card-based approach is best. If not, dividing the material into cards adds an extra and unnecessary level of complexity.
The underlying data for Locator is probably equally suitable for either approach. The Guide `scroll' for Locator is in fact never bigger than the screen, so no scrolling takes place. (This is highly unusual in a Guide application, but was a Locator design aim.) On the other hand the Locator display presented by Guide is not based on fixed pages: recall that the top part of the display gives the customer's decisions that caused the current point to be reached; this exploits Guide's in-line replacement facility. The information is dynamic: several different paths can lead to the same point, so the path (i.e. the list of customer decisions) cannot be wired into the destination.
At hypertext conferences half the papers tend to be about navigation, specifically about avoiding getting lost. Yet on the Locator project, and probably on most other real-world hypertext projects, the getting-lost issue is a minor one compared with many of the others we are considering. In Locator getting lost is certainly not a big problem for readers (in this case diagnosticians), and is only a moderate problem for authors.
The biggest reason why navigation problems have been kept in check was good authorship; in this case one of the added benefits of the discipline used by the authors has been that readers and authors are less likely to get lost.
A secondary reason was Guide's relative lack of gotos. This is a more technical point, and we shall spend a little time explaining it.
Links in hypertext are like gotos in programming languages. They represent a low-level feature that causes problems in maintenance and, more specifically, in getting lost. Guide has made modest steps in getting away from gotos by means of its usage-buttons. Usage-buttons are employed in Locator when two or more paths join.
Paths that join are extremely common in Locator. More generally in any information system it should be possible for a user to reach the same piece of information in many different ways. In Locator, assume that the underlying problem is with the power -- the customer may even have forgotten to switch it on. The problem may manifest itself in many different ways, e.g. `Will not boot', `Screen blank', `Disk light not on', etc. Thus many paths through Locator will lead eventually to the section dealing with lack of power. In the old paper manuals on which Locator is based, this information was in one manual -- in this case the manual concerned with disk booting -- and all the other occurrences had cross-references to this: `This is a power problem: see Section ... in Volume ... '.
If converting to hypertext is done without thought the same arrangement will be followed. Users who approach the lack-of-power material via `Will not boot' will see a hierarchy, but for other users the hierarchy will be broken by a goto that leaps into the `Will not boot' material. A hierarchy is a powerful aid to orienting the user -- indeed it is the only higher-level abstraction he has. Breaking it in a case like the above is arbitrary and unnecessary, and Guide's usage-buttons avoid the break.
What a usage-button does is to make a cross-reference appear to the user to be an ordinary hierarchical replace-button. The abstraction is that the referenced material is copied to the position where it is used. Thus in our example, if the diagnostician approaches the lack-of-power fault through the `Screen blank' route, then, when she comes to the cross-reference to the material on lack-of-power, this material is notationally copied into the `Screen blank' hierarchy. To the user it appears to be naturally part of this hierarchy, just as it does to the user who approaches it through a `Will not boot' hierarchy. In summary, usage-buttons enable a host of different hierarchies to be built on the same underlying material, and each user sees their hierarchy as the `true' one.
In Locator the underlying hyperbase is an acyclic directed graph. Perhaps surprisingly, usage-buttons are equally valuable with cyclic directed graphs, e.g. for instantiating grammars (Brown, 1989b).
We have been discussing one abstraction, the hierarchy, but there is a need for much more.
Indeed, abstractions are another area where the needs of hypertext authors are similar to the needs of programmers. We have learned from programming languages that abstractions are a powerful way of understanding programs: instead of talking about bit-patterns one creates abstractions that relate to the current problem. Thus one might have a data structure called `wheel' and a procedure called `reduce size'. For authors to represent and to understand the structure of hypertext we need similar abstractions that are at a higher level than the goto link. The previous Section described how a higher level abstraction, the hierarchy, could encompass gotos.
As Engelbart observed when explaining his hypergrammers, many of the most useful abstractions are ones tied to the problem in hand, rather than general ones that a hypertext system might provide for all users. In Locator one abstraction is the particular style of question that pervades the Locator system -- we call this a Locator-question. A Locator-question is made up of a variable number of lower-level Guide constructs such as enquiries, replace-buttons, usage-buttons, etc., following rules dictated by the house-style adopted for Locator.
To provide these abstractions, ICL have developed a tool that creates Guide source files. This tool is specifically designed for creating Locator-questions and other abstractions, together with the links between them. It has a pleasant visual interface, and does checking on-the-fly, so that the author can easily correct mistakes. The tool has led to a dramatic increase in the productivity of authors, and has enabled ICL to extend the Locator hyperbase to cover six machine ranges. It merits a paper in itself, but is unfortunately subject to commercial secrecy.
Poor information has a huge cost. The potential benefits of hypertext are that information will be better -- the readers will more frequently reach the information they need -- and will be obtained more quickly. The hypertext literature is full of descriptions of `immensely successful' applications, but quantifiable measures of real projects are sadly lacking.
The Locator project was in an environment where budgets were strict, and competing projects were all too keen to show that they could get better results. Thus measurements of effectiveness needed to be made, and the results of these were very pleasing.
In the paper-based system that preceded Locator, 68% of faults were correctly diagnosed as a result of the phone conversation with the customer. The effectiveness of Locator was tested first after two weeks of operation, and again after six weeks, when the diagnosticians were more practised. After two weeks 88% of faults were correctly diagnosed, and after six weeks this had risen to 92%. The overall result was a huge saving in engineers' time.
Although the benefits of Locator have been quantified I cannot, unfortunately, reveal details of costs, other than that ICL management thought the project was cost-effective and, indeed, awarded it a prize. Originally, the biggest concern was authorship costs, but ICL tamed this problem by using the authorship tool we have mentioned earlier.
Locator runs on expensive hardware: graphics workstations. The verdict, albeit a subjective one, is that the extra cost of such hardware is justified.
There are still a large number of problems and research issues facing hypertext. Indeed experience of areas such as software engineering shows that, as the state-of-the-art advances and bigger applications become tractable, the number of current problems increases rather than decreases. The problems facing hypertext are not by their nature problems that will ever be `solved' -- no-one will solve the authorship problem. The scale of the problems will, we hope, be reduced over time, and there is scope for spectacular advances in some areas.
Experience of Locator has shown that these problems can be surmounted, and that there are numerous opportunities to make effective use of hypertext in real applications.
As for researchers, the `easy problems', which will eventually lead to the most glory and the fewest tears, probably lie in the more mundane areas such as testing and abstractions rather than in glorious navigation aids or multi-media extravaganzas.