# Continuous Uncertain Interaction

## Figures

High quality, original versions of the figures from the thesis are available from this page. All original images are copyright John Williamson 2006. You may use these in your own publications, talks etc. as long as appropriate citation is made. You may also modify these figures. If you do so, please make it clear in the caption (or equivalent) that the figure is a modification of the original.

Figures in the thesis which are reproduced from other sources are listed, but are not available from this page -- a reference to the original paper is instead provided.

Images are in png format for bitmap originals, and in eps format for vectors. Most vector images are also available in svg or ai (Adobe Illustrator) versions. The svg images can be opened and edited with Inkscape. A few images are originally in dia format. A few others are in sxd format, which can be opened by OpenOffice. You can convert the eps format images to other vector formats using pstoedit along with Ghostview on most platforms.

Chapter I: Introduction
Chapter II: Theory and Definitions
Chapter III: Dynamic Probabilistic Feedback
Chapter IV: Predictive Uncertain Display
Chapter V: Augmented Dynamics
Chapter VI: Active Selection

Back to the main thesis page.

### Chapter I: Introduction

 1.1 The evolution of the human-computer interface. From the rigid, time delayed interactions of the punch card interface, to the conventional GUI, and beyond to tightly-coupled extensions of the mind. SVG | EPS 1.2 A Morse key; an example of a highly refined physical interaction device. This device has evolved over a period of a century, and is extremely effective at transforming intentions into sequences of Morse code; the weighting, bearings, spring tensions and shape of the device have all been carefully adapted to provide the optimal dynamics to maximise the transfer of information. Building interfaces which have this quality of interaction, but also the flexibility of software control, is a major challenge. PNG 1.3 The effect of delays on control. (a) Very short delay: simple comparison of current state and desired state is sufficient. (b) Medium delay: a model of the system dynamics must be introduced to compensate. (c) Long delay: capacity of the delay exceeds complexity of model and control becomes bursty. All real world systems have some delay in both directions, but the lengths of the delays may be significantly different. SVG | EPS 1.4 An overview of the structure of the thesis. Chapters III and IV discuss feedback from the goals; Chapter V explores feedback from the inference process to the interface dynamics; and Chapter VI describes general techniques for building selection systems. SVG | EPS

### Chapter II: Theory and Definitions

 2.1 Bit-rate curves for different interfaces. Many interface design decisions involve trading off ease-of-learning against ultimate interaction rates. SVG | EPS 2.2 A basic negative feedback control loop. The error signal is fed back to maintain control. SVG | EPS 2.3 The human-computer interaction as a closed-loop control process. The “imaginary path” between the intention of the user and the actions of the system is shown as a dashed line. Real communication must take place through the interface, via the control loop. The circles indicate comparator units. AI | EPS 2.4 The interface divided into the evidence space, the goal space and the state machine. Only the input side of the interface is illustrated, the feedback paths being omitted. SVG | EPS 2.5 The changing entropy H(x) across regions of the goal space, shown here for a three goal system. Entropy is shown as the dotted surface above the simplex – distance from the surface indicates entropy at that point. It reaches its maximum of 1.584 bits at <1/3, 1/3, 1/3> and drops to zero at the vertices. PNG 2.6 The plots at the top show four example densities on a two dimensional space. The contours show the total probability at each point (i.e. sum of all three goal probabilities). The red line is the path of a test trajectory, beginning at the lower right. (a) shows a Gaussian/squared exponential kernel $e^{\frac{-x^2}{\sigma^2}}$, (b) a Cauchy kernel $\frac{1} {\pi \left( 1+ \frac{x}{\sigma} \right) }$, (c) a Laplace/double exponential kernel $\frac{1}{ e^{\sqrt{\frac{x}{\sigma}}} }$, and (d) a raised cosine kernel $1+\cos\left( \pi \frac{\sqrt{x}}{\sigma}\right)$. The lower plots show the goal space trajectories corresponding to the example path on the 2-simplex for each of the p.d.f's. These densities transform spatial (i.e. sensed) measurements into positions in the goal space (distributions over goals). SVG | EPS 2.7 Time-dependent utility of information and the arrival of evidence. Each curve shows the utility of information as a function of time (solid green, fixed for all three cases) and the arrival of evidence for that piece of information (dotted blue). In (a) evidence arrives at nearly the same time as the maximal utility of the information; effective control is possible in this case. (b) shows a situation where the system under control is very predictable; evidence for the state of the system is accumulated well in advance. In (c), information is not reliable until long after the event has occurred, and is no longer useful at that point. Such a system cannot reliably be controlled. SVG | EPS 2.8 Some roughly estimated delays in an interaction, for an audio display. The upper bound time is 2145ms, the lower is 57ms. For a visual display, the transmission time would drop to almost zero and the visual processing delay in the brain would increase by »20ms. SVG | EPS 2.9 The interface as a series of nested control loops. Layers move outward from physiological loops to complex learning behaviours. SVG | EPS 2.10 Some potential sources of uncertainty in the control process. AI | EPS 2.11 Terminal events in conventional interaction devices. In each case, a single button style motion is used to trigger the action. Evidence accumulates throughout the interaction, but is not used until this action occurs, and when it does occur, it is treated as if it were absolutely certain. PNG 2.12 The transfer functions used for mouse control in Windows XP. Taken from the Microsoft website ( http://www.microsoft.com/whdc/device/ input/pointer-bal.mspx , retrieved 10th of June 2006). The mouse control panel has four “speed” settings which select one of these transfer functions. This function is applied after any on-mouse filtering. 2.13 Oscilloscope traces for two different buttons being pushed. Note that the behaviour is quite complex, with noise and ringing around the transient. Debouncing algorithms are needed to clean up these signals for button controls. The top trace in both cases is the raw signal values; the bottom indicates the corresponding logic states. These two images are taken from Jack Ganssle’s debouncing guide ( http://www.ganssle.com/ debouncing.pdf , retrieved 15th April 2006).

### Chapter III: Dynamic Probabilistic Feedback

 3.1 Exposed mechanics. Good feedback should reveal the workings of a system in a way users can model. PNG 3.2 Some postulated visual uncertain displays in a map navigation example. Left to right, top to bottom: Single point, no uncertainty; point cloud (no alpha); point cloud (alpha blended, anti-aliased); density alpha overlay; outline of two standard deviation bound; world convolved with position density. PNG 3.3 The granulation process (shown here for three grains). A number slices are taken from a set of original sounds. Sample positions are chosen by some process, here by sampling from a p.d.f. These slices are then enveloped (to eliminate clicking artifacts), pitch adjusted and spatialized according to the densities for each parameter. All of the slices are summed together to produce an output sound stream. In practice many hundreds of grains are active simultaneously. AI | EPS 3.4 Some possible enveloping windows for granular synthesis. These are applied to waveform sections to eliminate clicking artifacts. The effect of windowing a sine wave is illustrated. EPS 3.5 Multiple models in a recognition task are directly sonified via granular synthesis. AI | EPS 3.5 Image from the mixture of Gaussians demo. A number of Gaussian densities are arranged in a 2d state space, each with its associated waveform. The probability of each source given the current cursor position is sonified. In this image, isoentropy contours are shown. The colours indicate the mixture of probabilities at that source. PNG 3.5 (a) Audio densities are sparse and non-overlapping, leading to a jumpy sound. (b) Densities are adjusted so as to cover the entire inter-interval region. Sound is smooth and clean AI | EPS 3.5 An image from the arc length mapping demo. The dark path indicates the mean positions, while the outer circles indicate the one standard deviation bound. The current position along the path is given by the triangle extending from the cursor; as the mouse moves closer and further the potential positions contract and expand respectively. Audio is produced corresponding to the section of the curve highlighted by the two endpoints. PNG