Temporal properties of interaction have a profound impact upon the usability of human computer interfaces. Delays in response time can lead to frustration and error. The simultaneous presentation of many different pieces of information imposes heavy demands upon the cognitive and perceptual resources of system operators. These problems have been investigated by a number of recent research initiatives. Unfortunately, it has been difficult to replicate the results that have been obtained from experimental investigations. This creates significant problems for designers if these results are to guide the future development of interactive systems. This paper briefly describes how an multi-disciplinary team has addressed this problem during the Temporal Aspects of Usability (TAU) project.
Keywords: time; simulation; empirical evaluation; formal methods.
Temporal properties of interaction can determine the usability of interactive systems. For instance, Kuhmann, Boucsein, Schaefer and Alexander (1987) have shown that delays in system responses lead to frustration and to increased error rates. Wickens (1984) argues that the simultaneous presentation of multiple information sources places heavy burdens upon the cognitive and perceptual resources of system operators. Unfortunately, the empirical work that supports these general conclusions has proven difficult to replicate. For instance, the results obtained by Kuhmann et al are highly specific to their prototype interface. Similarly, the evidence that Wickens cites is hedged with caveats about the experimental set-up and the grouping mechanisms that were used to present the multiple sources of information. These problems make it difficult for designers to gauge the general applicability of findings about temporal problems during interaction.
The TAU project was set up in 1992 to address the problems of replicating and validating empirical results for temporal usability problems. The project has involved a multi-disciplinary team at the University of Glasgow from the Departments of Computing Science (David England, Philip Gray, Chris Johnson and Steve McGowan) and Psychology (Steve Draper and Paddy O'Donnell). The focus of the project was to develop a simulation environment that would support experimental investigations. This system was intended to provide a stable vehicle that could be used by many different research groups. This would support the replication of experimental conditions by avoiding the ad hoc development of many different pieces of software in each of the labs that were investigating this area.
It is important to stress that the aim of the TAU project was to develop a simulation environment rather than a prototyping tool (Johnson, 1995). The distinction lies in the way that the system is intended to be applied. Traditionally, prototyping tools provide means of rapidly mocking up human- computer interfaces. The emphasis is upon providing potential users and clients with some impression of what it would be like to interact with an eventual implementation. Figure 1 illustrates the general approach. A partial implementation may be used to guide formative evaluation up to the point where designers can be relatively confident about the course of potential interaction between a system and its users. Temporal properties are only considered to the extent that they might affect the operation of a final interface.
Figure 1: Using Prototypes For Iterative Interface DevelopmentIn contrast, the TAU simulation tool was developed with the specific intention of supporting experimental investigations. The aim here is not to develop a partial implementation to the point where it might guide the detailed development of a particular interface. Rather, it is intended to support the experimental investigation of temporal problems that affect human computer interaction. Figure 2 illustrates the features of our approach.
This diagram illustrates the three areas that were addressed by our multi- disciplinary team during the life-time of the TAU project: the development of an appropriate notation for describing temporal usability problems; the design and implementation of the simulator itself and the development of appropriate experimental techniques to be used in conjunction with the notation and simulator.
One of the key problems in replicating results in this area is that papers often contain imprecise vernacular terms such as `after' or `in between'. XUAN (eXecutable User Action Notation) was developed to address this problem. This includes means of describing user tasks in terms of pre- and post-conditions, external constraints, necessary user actions and local objects that are affected by the task. In addition, to features for describing pre- and post-conditions, user actions and so on, this notation provides an expressive range of interval and real- time operators. An early result of our analysis was to establish that both classes of temporal descriptions are required in order to capture temporal usability problems. For example, users develop coping strategies, such as multi-tasking, in response to consistent real-time delays in system responses. If it always takes five minutes to load a program then a user might perform another task, such as reading their mail, while waiting for the program to be loaded. This could lead to a usability problem if the coping strategy led to interference between the two tasks. Users might lose track of the loading task while they read their mail. In XUAN this is represented as follows, `duration' represents the real-time delay, the `||' operator represents concurrency:
duration(Load) = 300 and Load || ReadMail
Other usability problems cannot be so easily expressed in terms of real-time. For example, confusion may result if two tasks are order independent. This is especially significant in control applications. In some contexts, users might issue a command to slow equipment before cutting the power. In other contexts, they might be required to cut the power before issuing commands to slow the plant. Remembering correct orderings for particular contexts can impose heavy burdens upon system operators. In XUAN this is represented as follows, sequence is denoted by `,', order independence is denoted by `&':
CutPower, HaltCompressor & HaltCompressor, CutPower
A full description of the XUAN language is provided in Gray, England and McGowan (1994).
The TAU simulator was developed to directly implement interactive programs from the XUAN descriptions illustrated above. The use of a formal language helped to provide a consistent means of representing temporal usability problems. The development of a simulator helped all of the members in the evaluation and development teams to understand the consequences of an XUAN description of a particular usability problem. From a computer science perspective, the simulator used a YACC and Lex preÐprocessor to translate the XUAN descriptions into Ada tasks. This architecture provides a means of executing task descriptions that included concurrency requirements through parallel operators. Figure 3 provides a more complete overview of the software architecture.
This design and implementation of the TAU simulator is described in more detail in McGowan, Gray and England (1994).
During this project Steve Draper and Paddy O'Donnell from the Department of Psychology, University of Glasgow acted as the target users for the XUAN simulator. They provided case studies of the classes of experiments that they wished to perform in order to clarify different sorts of temporal usability problems. Much of this work focused upon a reÐexamination of the Teal and Rudnicky (1992) investigations into system delay and user selection strategies.
The TAU simulator was used to re-produce their experimental setÐup as faithfully as possible from the methodological information that was provided in the original papers. Our series of experiments demonstrated the usefulness of the simulator and of our more general results. It proved to be impossible to replicate the findings that were identified by Teal and Rudnicky. This might be due to the problems in exactly reproducing their experimental setÐup. If so, then future investigations might use the XUAN description language and the TAU simulator to re-create faithfully the system that was used to produce our results. Alternatively, our results might suggest aspects of the problem that were not uncovered during the initial range of experiments. In particular, users may not be exploiting the range of strategies that were identified by Teal and Rudnicky. The simulator is currently being used to develop a further range of experiments to investigate this issue.
This paper has provided an overview of the Temporal Aspects of Usability (TAU) project. Brevity prevents a more complete technical description but we have emphasised the main aim of our investigations: to support the replication and validation of empirical results about temporal aspects of usability. In order to achieve this aim, we have developed an existing task analysis notation to provide a precise and concise means of representing temporal properties of human-computer interfaces. We have developed a simulation tool that can directly execute these XUAN descriptions. Finally, we have used our tools and techniques to replicate an important set of experiments into temporal usability problems. Our inability to reproduce the results described in the literature is indicative of the problems that arise when different research teams are forced to use ad hoc software and imprecise descriptions to present their experimental setÐ up.
There are many directions for further research. In particular, the development of semi-formal descriptions and simulation tools only provides a partial solution to the problems of replicating experimental results. External factors, such as the rewards system used to motivate volunteers, can affect the results of temporal experiments. These details are almost entirely neglected by previous work in this area. Further work is envisaged to identify these factors that may prevent the replication and validation of experimental results in this area.
This work was supported by the UK Joint Councils Initiative in HCI and Cognitive Science, Grant No 9201233.
Gray, P; England, D & McGowan, S. (1994). XUAN: Enhancing UAN to Capture Temporal Relationships Among Actions. Proceedings of HCI'94. Cambridge University Press. pp. 301-312.
Johnson, C.W. (1995). Using Z to Support the Design of Interactive, Safety- Critical Systems. Software Engineering Journal. (10)2:49-60.
Kuhmann, W; Boucsein, W; Schaefer, F & Alexander, J. (1987). Experimental Investigation of Psychophysiological Stress-Reactions Induced by Different Response Times in Human-Computer Interaction. Ergonomics (30)6:933- 943.
McGowan, S; Gray, P & England, D. (1994). XUAN: An executable task description language. Technical report IS-94-01. Department of Computing Science, University of Glasgow.
Teal, S L & A I Rudnicky, A I. Changes in user task strategy due to system response delay. Proceedings of CHI'92. pp 295-305.
Wickens, C.D. Engineering Psychology and Human Performance. Merrill, 1984.