Alistair Morrison

[ Home ] [ Papers ] [ Replayer ] [ Software and downloads ]

Replayer: Coordinated Visualisation of Video and System Log Data

Replayer is being developed to support the understanding and development of mobile technology. It can be used by computer scientists, social scientists, or interdisciplinary groups engaged in studies into the use of mobile technology.
The work is supported by Equator and the ESS project Integrating Field and Systemic Data in a Visualisation for Collaboration.

Evaluating mobile systems

Replayer is designed primarily to aid in the evaluation of mobile applications. Traditional usabililty-type studies, where users are observed and recorded in controlled settings, are hard to perform with multi-user distributed ubiquitous systems:

Users are mobile - they cannot be recorded by precisely angled static cameras
Capturing user activity on small devices is difficult
The focus of an evaluation might be interaction between users at different physical locations

So videos will generally fail to provide an analyst with a complete record of a user trial. Creating system logs, where user activity is stored by the mobile device, will provide precise timestamped records of all interaction, but not the rich contextual information offered by video recordings.

The solution offered by Replayer is to combine both of these forms of data in one synchronised system. An activity log recorded on a mobile device will reliably note when an action was performed; video data will show environmental or social factors that might influence a user's behaviour. An analyst can view all the recorded information simultaneously, with several video streams and activity logs from multiple users.

Visualisations

Replayer provides a number of information visualisation tools that can visually display the recorded system logs. These tools can be used to provide overviews of the data recorded from a user trial, or to drill down to focus attention on specific incidents.

	Time Series The Time Series component plots logged data over time. In this example, the wifi signal strength is graphed for two users over the course of a 27 minute trial of a mobile application. Two two drop-down lists at the bottom of the list allow the analyst to select which of the recorded variables to graph and which to colour by (in this case, signal strength and user ID respectively).
Event Series Discrete events are visualised on the Event Series. User interactions such as button clicks are shown over time, along with system events such as spotting wireless access points. Events are coloured by user. Clicking on the axis on the left can filter out users in whom the analyst is not currently interested.
	Video Component Replayer handles multiple concurrent video streams. The horizontal bar at the bottom of the image is a timeline of the trial, with the thumb (blue icon) showing the current position. Dragging the thumb along the timeline skips both videos to the corresponding positions.
Map component Spatial distributions of events can also be shown. Here we plot the paths of two users over the coure of a trial. The analyst can choose to colour by user (as here), by different kinds of event, or by any other nominal values recorded during the trial.

Synchronised use

The major benefit offered by Replayer is the ability to coordinate multiple views and heterogenous data sources in a synchronised environment. The various views are coordinated so that the selections made in one component are reflected in all the others.

Examples of use include:

Watching synchronised video playback and seeing events highlighted on the Event Series and map as they occur
Selecting an event of interest on the Event Series and seeing its position on the map, the system state at that moment on the time series and the video recorded at that instant
Selecting a small area of the map (such as a road junction) and seeing all the events that occurred at this location
Selecting a cut-off point or range of values for a system state variable (eg wifi signal strength) and seeing in all the other views the distributions of events where the value is in this range

In the example below, one type of event has been selected by drawing a rectangle in the Event Series. This shows the position on the map where each of these events was performed. The video timeline on the right is also highlighted in green at the times the events occurred. Playback can then skip between only these parts.

Example

Further reading

More details on Replayer can be found in a very short paper presented at a CHI workshop in 2006 or in more depth in a paper from IEEE CMV 2006.

Download

Download Replayer.

NCESS

counter free hit unique web