Over the last five years, large numbers of papers have identified a vast array of usability problems with the World Wide Web. These have ranged from the problems of navigation to caching strategies and browser design. Unfortunately, very few of these studies have considered the economic and social aspects of interaction. This paper, therefore, argues that many previous results will have to be revised or abandoned. In particular, I argue that methods of payment have a profound impact upon individual attitudes to retrieval delays. Page layout may be a less important determinant of successful retrieval than the details of a user's telephone bill. Similarly, I argue that most information retrieval tasks are not performed in isolation. Navigation techniques may be a less important determinant of successful retrieval than the development of expertise within groups of co-workers.
In spite of much of the hype that surrounds the World Wide Web, there is still a considerable amount of user dissatisfaction. For example, Robert Kraut's HomeNet project has produced large amounts of high quality, longitudinal data on Internet usage for families in the Pittsburgh area. This data has shown a considerable decrease in Internet usage after an initially high level of activity [6]. A growing number of empirical studies can be used to explain these observations. For example, my previous work has established a clear connection between user dissatisfaction and the increasing retrieval delays that affect popular sites [3,4,5]. For example, Figure 1 shows how subjective ratings for the quality of video clips vary with file size. Clips below two megabytes are criticised for being too short. Clips larger than three megabytes produce negative reactions as retrieval delays increase. Ramsay, Barbesi and Preece have replicated these results [10]. Other authors have used this data to directly informing interface development. Nielson's forthcoming book uses the data in Figure 1 to develop guidelines for the provision of remote, video resources [9].
All of this gives an impression of steady progress. We are gradually increasing our understanding of the usability problems that affect the World Wide Web. I would like to challenge this assumption. Many authors simply describe the symptoms of a usability problem. They focus upon user dissatisfaction with current browser technology or page layouts. A far smaller number go on to identify cures. For example, reducing the number of images on a page reduces negative reactions to high download times. Very few authors have attempted to explain the underlying psychological and, in particular, the social and economic factors that lead to usability problems. In consequence, there is a widespread assumption that papers about the World Wide Web lack both engineering content and academic rigour [2,5].
This paper focuses upon the underlying factors that lead to user dissatisfaction with retrieval delays. Empirical work has shown that this dissatisfaction arises when the costs of accessing a resource are perceived to outweigh the benefits of that resource [10]. However, in order to interpret these results it is important to have a clear understanding of the different economic factors that affect users' perceptions of retrieval delays.
There are two ways of considering the costs that users must pay for each item of information that they retrieve from a remote source. The first simply concerns the financial overheads. These include the fixed costs associated with acquiring appropriate hardware and software. They also include the variable costs that must be met each time users access remote information. These variable and fixed costs differ from one country to another and between service providers within those countries. For example, UK companies typically charge per unit of time for telephone-based access to Internet resources. In contrast, many US providers operate a flat fee tariff for local calls. The impact of this can be illustrated by the following graphs.
Figure 2 a) illustrates the pay per unit model adopted in the UK. Here the fixed costs include hardware and software purchase. The variable costs are determined by the amount of time that the user requires to complete their search tasks. This diagram shows a constant increase in costs for each unit of time that is consumed. 'Frequent-user' reductions may, in reality, reduce the cost of each additional unit of time after some specified threshold. After this point, the slope in Figure 2 a) would be reduced. In contrast, Figure 2 b) illustrates the flat rate approach. Here the fixed costs include the tariff imposed for all local calls. This must be paid irrespective of how much time the user spends in completing their search tasks.
These payment mechanisms have a profound impact upon the strategies that users employ when interacting with Internet resources [3]. For example, customers that use the flat rate model can predict the financial cost of any retrieval task. Customers that choose to pay through the unit charging scheme cannot predict their costs unless they know the total amount of time that will be required to access the remote resources. This is an impossible task given variable network loading and the lack of information that most web pages provide about the size of remote resources. The key point here is not the precise shape of the graphs shown in Figure 2. Instead, it is to point out the lack of published, empirical results into the impact of payment techniques on user behaviour over the Internet [6]. This is an incredible omission. There are large numbers of well developed studies into the impact of billing techniques on cable television, mobile telephones and even hire purchase agreements [3]. These studies pose a significant challenge to the CHI community. For example, they have forced me to completely revise the evaluation techniques that I used to measure the satisfaction ratings shown in Figure 1. File size may be a less important determinant of successful retrieval than the details of a user's telephone bill.
Opportunity costs represent a second form of tariff that users must meet when interacting with distributed, information retrieval systems. These can be thought of as the time that users must sacrifice in order to retrieve remote resources. It might also be thought of as the cost that any delay incurred for the success of the user's task as a whole. This view is a simple extension of March and Simon's empirical work into satisficing [7]. Their experimental approach involved users literally sacrificing some quantity of one good or service in order to gain another. In the World Wide Web, users are trading time for information. Figure 3 illustrates this point. Initially, users have some anticipated or predicted minimum for the amount of time that it will take to complete any transfer. Up to this point, the value of any information retrieved will be relatively high. Documents received before the minimum predicted period can be seen as a bonus from the user's perspective. Eventually, as delays increase the perceived value of the resource will rapidly decline. At some point there will be no time left for the user to complete their task. After this moment, the information will be of almost no value.
Recent experimental work in Econometrics has suggested ways in which March and Simon's pioneering research might be extended to consider individual attitudes to risk taking [8]. For example, Figure 3 characterises someone who is relatively impatient. They expect relatively low returns if the resource is not accessed within the initial moments of a request. Hence, they are more willing to abandon that request and move on. In contrast, Figure 4 represents a more patient approach. The user still expects to receive information with a relatively high value even as they approach the maximum available time for their search task. They will, therefore, be more reluctant to take the risk of abandoning their request and looking elsewhere.
These diagrams have important implications for the future development of information retrieval systems. Browsers can be tailored to support individual strategies. For example, if users frequently abandon requests before downloads are completed then predictive caching can be used. This involves automatically generating requests for pages that are linked to the one the user is currently viewing. This improves performance as they move from one request to another. In contrast, the browsers of more patient users can be configured to avoid the additional space and server overheads that predictive caching involves [4].
Again, the key point is not the exact shape of the curves presented in figures 3 and 4. As with the work on payment mechanisms, the intention is to point out the need to learn from work in related fields such as Econometrics. It is clear that much research remains to be done before we can validate models, such as that shown in the previous diagram. Ramsay, Barbesi and Preece echo this observation when they argue that; "Until the advent of better and cheaper web technology, it is essential to understand the wider psychological effects of slow retrieval times on the web. In so doing, and by turning the knowledge to our advantage, we can move towards greater usability on the web" [10].
The closing words of Ramsay, Barbesi and Preece are worth citing because they emphasise the need to understand the psychological processes that influence subjective responses to distributed, information retrieval systems. It is also possible to take a slightly wider perspective and argue that not only the psychological but also the social processes of interaction play a key role in determining the 'usability' of the World Wide Web. The importance of this approach can be illustrated by Figure 5. In contrast to previous diagrams, this focuses on the expected benefits to be gained from investing time at different points in a search task. For example, the benefits to be gained from investing an extra five minutes into a search may be very high if the search has just begun. If the user has already been looking for several hours then five minutes may make very little difference either way.
Figure 5 a) presents a concave curve. This represents the archetypal novice in the sense that relatively few rewards might be anticipated for time invested in a search until the user has identified relevant sources of information. Bergman confirms this when he reports that between 30 and 50% of novice users' requests result in 'no hits' [1]. Once users have identified appropriate sources of information, the returns will increase for each unit of time that is invested in the search activity. At some point the returns will begin to diminish as the available sources begin to 'dry up'. Figure 5 a) does not reflect this because novice's may continue to expect high rewards from sources that they have yet to find, even though those sources may not exist [12].
In contrast, Figure 5 b) represents an expert's attitude to the returns from any search. Relatively low investments in terms of the expert's time will quickly yield large rewards. Expertise is characterised not only by knowledge itself but by the ability to quickly find and filter new sources of data [11]. Eventually, however, there comes a point at which any additional time will only produce limited returns; if the expert can't find it then it's probably not there.
As in previous sections, the shape of the curves is less important than the challenge that they represent for interface designers. How can we help novices to access the same quality of information as an expert with the same investment in time? Typically, this is done through the design and optimisation of pseudo-natural language querying systems. An alternative approach is to enable users to collaborate as a means of improving the results of their search activities. In this view, information retrieval is a social activity and not just a field for technological innovation. Twidale and Nichols argue that "By acknowledging the importance of other people in the search process, we can develop systems that not only improve help-giving by people but can lead to a more robust search activity, more able to cope with, and indeed exploit, the failures of any intelligent agents used" [12]. Previous sections have criticised the lack of work into the impact of pricing mechanisms and individual attitudes to retrieval delays. There has, however, been some ground breaking research into the collaborative aspects of information retrieval. For example, Rose, Bornstein and Tiene have pioneered the concept of information rendez-vous [11]. This approach provides brokering services that mediate between the producers and consumers of information. For example, some implementations rely upon user models to establish connections between information seekers and domain experts. Instead of looking for isolated pages of information, these systems identify individuals who can help users to find the information that they require. The user's query is matched against a list of interests that has been recorded for the domain expert. Again, however, many questions remain to be answered. With agent based search engines, domain experts could quickly become inundated with users' requests. It remains to be seen whether altruistic co-operation can be sustained in the face of a rising demand for an expert's help in locating information.
Some researchers have already begun to consider the social and economic constraints of collaborative searches over the World Wide Web. For example, Twidale and Nichols have developed a graphical notation that is intended to support users' requests for help. They have provided a visualisation for traces of interaction between a user and an information retrieval system. These representations can stored for later re-use. They can be shown to experts who can use the visualisation to highlight optimisations. Figure 6 illustrates the impact that this has upon the outcome of a search. Initially poor returns for a novice query are quickly boosted through effective collaboration. The higher of the two curves illustrates the effect of expert advice. Diminishing returns set in because experts can advice users when not to expect any further results from a particular search or query.
Recent work in information science has shown that most web-based search activities are collaborative, even without Twidale and Nichols' visualisation techniques [1,2]. Large numbers of users rely upon the advice and guidance of their colleagues when searching for information. These self-help activities ranges from exchanging pages of useful links through to detailed advice about query formation on specific search engines. One reason for the growth of these self-help networks is that economic barriers prevent organisations from employing information retrieval specialists. There are also social factors that encourage collaboration. Most consultations with professional librarians tend to be extremely short. Users have an immediate need. They do not want prolonged consultations that last for several sessions [12]. This makes it difficult for users to communicate the necessary domain details for librarians to formulate appropriate queries [1]. Again, my purpose is not simply to identify recent developments in information science but to point out that the CHI community has provided little or no support for these self-help groups. A trivial example of this is the way in which every member of a community must independently update the URLs on their pages every time another member of that community changes their Web site. This raises deeper questions. Perhaps too much attention is being paid to the development of technological solutions. Too much effort is being paid to improved search engines and caching techniques. Too little attention is being paid to the ways in which groups of people are currently using today's technology. Does this complaint sound familiar?
Great steps have been taken over the last five years in understanding the usability problems of the World Wide Web. These have ranged from the problems of navigation to caching strategies and browser design. I have argued, however, that many of these advances have failed to consider the social and economic factors that influence the day to day use of the Web. For example, recent work on payment mechanisms for cable television and mobile telephones has forced me to revise my previously published work on retrieval delays. Page layout may be a less important determinant of successful retrieval than the details of a user's telephone bill. Similarly, work in information science has emphasised the collaborative nature of information retrieval tasks. From this it follows that we should focus less on individual performance and more on the development of information retrieval expertise within teams of co-workers.
1. C. Borgman, Why are On-Line Catalogues Still So Hard to Use? Journal of the American Association for Information Science, (47)7:493-503, 1996.
2. T. Erickson, Social Interaction on the Net: Virtual Community as Participatory Genre. In R.H. Sprague (ed.), Proceedings of the Thirtieth Hawaii International Conference on System Sciences, VI, IEEE Computer Society Press, 13-21, 1997.
3. C.W. Johnson, The Impact of Time and Utility on Distributed Information Retrieval. In H. Thimbleby, B. O'Conaill and P. Thomas (eds), People and Computers XII: Proceedings of HCI'97, 191-204, Springer Verlag, Berlin, 1997.
4. C.W. Johnson, Electronic Gridlock, Information Saturation and the Unpredictability of Information Retrieval Over the World Wide Web. In F. Paterno and P. Palanque (ed.) Comparative Approaches to Formal Methods, Springer Verlag, Berlin, Germany, 1997.
5. C.W. Johnson, Ten Golden Rules for Video over the Web. In J. Ratner, E. Grosse and C. Forsythe (eds.) Human Factors for World Wide Web Development. Lawrence Erlbaum, New York, United States of America, 1998.
6. R. Kraut, Residential Use of the Internet: Results from the HomeNet Field Trial. In P. Thomas and D. Withers (eds), HCI'97 Conference Companion, 3, British HCI Group, London, 1997.
7. J.G. March and H.A. Simon, Organisations, Wiley, New York, United States of America. 1958.
8. C. Puppe, Distorted Probabilities And Choice Under Risk, Springer Verlag, Lecture Notes In Economics And Mathematical Systems, No 363, Berlin, Germany. 1991.
9. J. Nielson, Effective Web Development, in press.
10. J. Ramsay, A. Barbesi and J. Preece, A Psychological Investigation of Long Retrieval Times on the World Wide Web, Interacting with Computers. Special edition on CHI and Information Retrieval, 1998 in press.
11. D. Rose, J. Bornstein and K. Tiene, MessageWorld: A New Approach to Facilitating Asynchronous Group Communication, Fourth International Conference on Information and Knowledge Management, 266-273. 1995.
12. M.B. Twidale and D.M. Nichols, Designing Interfaces to Support Collaboration in Information Retrieval, Interacting with Computers. Special edition on CHI and Information Retrieval, 1998 in press.