This talk describes one part of my PhD research.
The overall aim of the project was to investigate ways of converting large, linear texts to hypertext. A collection of IR techniques for determining relevance between portions of text were adapted for this purpose. The results consisted of a large number of differently-linked hypertexts with no readily available or reliable way to evaluate them.
This talk concentrates on the problems of evaluating hypertext link structure. I will discuss existing, statistical measures and also some standard experimental methods for determining how well users of a text react to different link structures. The new measures that we employed, based on recall of text structure, are discussed in more detail and some conclusions we plan to draw.
The status of this work is not final, complete or exhaustively tested. Our measures of readability/usability were born of necessity and frustration at the paucity of existing evaluation techniques. Nor, for that matter, are we trained experimental psychologists. I welcome fierce criticism and more general discussion about the problems that we faced during this project.