Heuristic Evaluation for Software Visualisations

Introduction

Heuristic Evaluation (Nielsen and Molich, 1990; Nielsen, 1994) is a method of usability evaluation where an analyst finds usability problems by checking the user interface against a set of supplied heuristics or principles.

Heuristics

The following heuristics are supplied. The first ten were proposed and described by Nielsen (Nielsen, 1994), and the last three have been added for software visualisations.

Each heuristic is presented in a structured manner, with one or more of the following elements:

Conformance Question

What the system should do, or users should be able to do, to satisfy the heuristic.

Evidence of Conformance

Things to look for , for example design features or lack of design features that indicate partial satisfaction or breaches of the heuristic.

Motivation

Usability problems that the heuristic tries to avoid.

1. Visibility of System Status

Conformance Question

Are users kept informed about system progress with appropriate feedback within reasonable time?

Evidence of Conformance

Necessary evidence must be identified through analysis of individual tasks.

Motivation

Feedback allows the user to monitor progress towards solution of their task, allows the closure of tasks and reduces user anxiety.

2. Match between system and the real world

Conformance Question

Does the system use concepts and language familiar to the user rather than system-oriented terms. Does the system use real-world conventions and display information in a natural and logical order?

Evidence of Conformance

Necessary evidence must be identified through user studies (or through assumptions about users!), and through the analysis of individual tasks.

Motivation

A good match minimises the extra knowledge required to use the system, simplifying all task action mappings (re-expression of users intuitions into system concepts).

3. User control and freedom

Conformance Question

Can users do what they want when they want?

Evidence of Conformance

Necessary evidence takes the form of a diverse set of design features, for example "undo and redo", clearly marked exits etc.

Motivation

Quite simply, users often choose actions by mistake.

4. Consistency and Standards

Conformance Question

Do design elements such as objects and actions have the same meaning or effect in different situations?

Evidence of Conformance

Necessary evidence must be identified through several analyses (consistency within system, conformance to style guides, consistency across task methods).

Motivation

Consistency minimises user knowledge required to use system by letting users generalise from existing experience of the system or other systems.

5. Error prevention

Conformance Question

Can users make errors which good designs would prevent?

Evidence of Conformance

Necessary evidence must be identified through analysis of individual tasks and of system details (e.g. adjacency of function keys and menu options, discriminability of icons and labels).

Motivation

Errors are the main source of frustration, inefficiency and ineffectiveness during system usage.

6. Recognition rather than recall

Conformance Question

Are design elements such as objects, actions and options visible? Is the user forced to remember information from one part of a system to another.

Evidence of Conformance

Necessary evidence must be identified through analysis of individual tasks.

Motivation

Forcing users to remember details such as command and file names is a major source of error. Recognition minimises user knowledge required to use the system. Summarising available commands or options may allow the user to guess their meaning or purpose.

7. Flexibility and efficiency of use

Conformance Question

Are task methods efficient and can users customise frequent actions or use short cuts?

Evidence of Conformance

Necessary evidence must be identified through analysis of individual tasks, and the presence of design features such as keyboard accelerators etc.

Motivation

Inefficient task methods can reduce user effectiveness and cause frustration.

8. Aesthetic and minimalist design

Conformance Question

Do dialogues contain irrelevant or rarely needed information?

Evidence of Conformance

Necessary evidence must be identified through analysis of individual tasks.

Motivation

Cluttered displays have the effect of reducing search times for commands or users missing features on the screen. Users unfamiliar with a system often have to find an action to meet a particular need ‹ reducing the number of actions available could make the choice easier.

9. Help users recognize, diagnose and recover from errors

Conformance Question

Are error messages expressed in plain language (no codes), do they accurately describe the problem and suggest a solution?

Evidence of Conformance

Necessary evidence must be identified through analysis of error messages.

Motivation

Errors are the main source of frustration, inefficiency and ineffectiveness during system usage.

10. Help and documentation

Conformance Question

Is appropriate help information supplied, and is this information easy to search and focused on the useršs tasks?

Evidence of Conformance

Necessary evidence takes the form of help documentation which should be easy to search, focused on the useršs task and present a short list of actions to perform.

Motivation

Ideally, a system should not require documentation. However, it may be necessary to provide help which users need to access at very short notice.

11. Present necessary data accurately and unambiguously

Conformance Question

Does the visualisation display only the data necessary for a particular task, and is this data presented accurately and unambiguously so that users do not make false inferences?

Evidence of Conformance

Necessary evidence must be identified through analysis of individual tasks, and by comparison of possible interpretations of the data with its intended meaning.

Motivation

A visualisation should only display the data needed for a particular task (Norman, 1991) -  cluttered displays have the effect of reducing search times or increasing confusion. This data should be presented accurately and unambiguously so that users do not make false inferences.

Mackinlay (Mackinlay, 1986) and Norman (Norman, 1991) have suggested matching the characteristics of the data to the way it is displayed so that all the appropriate relationships are displayed and no false relationships can be implied by the user. For example, Mackinlay (Mackinlay, 1986) presents a variant of the following chart which displays car manufacturers and their nationalities. Nationality data has no ordering, but it is being displayed in such a way that ordering is implied to the reader.

mackinlay image

figure 1: A graph of car manufacturers and nationality

Figures 2 and 3 present another example where the structure does not match the display. The tree in figure 2 is being displayed as a treemap (Shneiderman, 1992) using a space filling approach. However, the treemap only displays true hierarchical data, and so it can not present graphs with multiple paths and loops without using keys.

tree image
figure 2: A conventional tree

treemap image
figure 3 :The same tree drawn as a treemap

Casner (Casner, 1991) suggested that different graphical primitives have different bandwidths, i.e. a range of values it can express. For instance, more data values can be expressed by line length than by shading. Data also has a bandwidth : its range of possible values. Casner suggests ensuring that the graphical primitive has a higher bandwidth than the data being represented.

12. Avoid Unwanted Gestalt Effects

Conformance Question

Are there any unwanted gestalt effects of proximity, similarity, closure, continuity and symmetry?

Evidence of Conformance

Necessary evidence must be identified by comparing possible interpretations of symbols and collection of symbols with the intended meaning.

Motivation

The Gestalt psychologists (Koffka, 1935) described how people perceptually interpret symbols and collections of symbols, and infer relationships whether they were intended or not. (Preece, et al., 1994) describe the important principles of proximity, similarity, closure, continuity and symmetry.

Proximity

People tend to interpret objects as groups if they are close together.

Proximity image

figure 4: Gestalt Principle of Proximity

Similarity

People tend to see symbols of the same shape as belonging together.

Similarity image

figure 5: Gestalt Principle of Similarity

Closure

People tend to see the following symbol as whole.

Closure image
figure 6: Gestalt Principle of Closure

Continuity

People perceive the dots as two lines of dots rather than a random collection of dots.

Continuity image
figure 7: Gestalt Principle of Continuity

Symmetry

People perceive the symbols as whole symbols because of symmetry.

Symmetry image
figure 8: Gestalt Principle of Symmetry

13. Make Software Objects Easy to Identify and Discriminate

Conformance Question

Are software objects displayed in a way that users can easily identify them?

Evidence of Conformance

Necessary evidence takes the form of standard or familiar coding schemes for display elements, or must be identified through user studies (or through assumptions about users!)

Motivation

Minimises user knowledge required to use system, and reduces confusion and anxiety about the meanings of display elements.

Acknowledgements

Heuristic Evaluation was originally proposed by Nielsen and Molich (Nielsen and Molich, 1990). The first ten heuristics were proposed and described in (Nielsen, 1994).

This presentation of Heuristic Evaluation has been developed for use on UK EPSRC project no. GR/K82727 ( Extending HCI Design Principles and Task Analysis for Software and Data Visualisation) by Darryn Lavery, Gilbert Cockton and Malcolm Atkinson. A structure has been applied to the heuristics described in (Nielsen, 1994), and to the best of our intentions we have kept the original meanings of the individual heuristics. The structure and any unintended changes to the meanings remain the responsibility of the authors and not Jakob Nielsen.

Re-used with permission by conforming to requirements laid out at http://www.dcs.gla.ac.uk/asp/materials/SVHE_1.0/. The materials must not be copied by anyone else who has not visited the web page and agreed to the conditions of use. The latest versions of the materials can be obtained from http://www.dcs.gla.ac.uk/asp/materials/.

Copyright University of Glasgow 1996

References

Casner, S. M. (1991). "A Task-Analytic Approach to the Automated Design of Graphic Presentations", ACM Transactions on Graphics, 10(2), pp. 111-15 1.

Koffka, K. (1935). Principles of Gestalt Psychology, Harcourt, Brace and Company, New York.

Mackinlay, J. (1986). "Automating the Design of Graphical Presentations of Relational Information", ACM Transactions on Graphics, 5(2), pp. 110-141 .

Nielsen, J. (1994). "Heuristic Evaluation", In Nielsen, J. and Mack, R. L. (Eds.), Usability Inspection Methods. John Wiley and Sons, New York, pp. 25-62.

Nielsen, J. and Molich, R. (1990). "Heuristic Evaluation of User Interfaces ", In Proceedings of ACM CHI'90 Conference on Human Factors in Computing Systems, pp. 249-256.

Norman, D. A. (1991). "Cognitive Artifacts", In Carroll, J. M. (Ed.), Designing Interaction: Psychology at the Human-Computer Interface. Cambridge University Press, Cambridge, UK, pp. 17-38.

Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., and Carey, T. (1994). Human-Computer Interaction, Addison Wesley, Reading, Mass.

Shneiderman, B. (1992). "Tree Visualization with Tree-Maps: 2-d Space Filling-Approach", ACM Transactions on Computer Graphics, 11(1), pp. 92-99.


Created 2-4-96

Last updated 12-6-97

Darryn Lavery