nature science update
updated at midnight GMT
nature science update home
by subject
send to a friend
printable version
about the site
about us

Graphs test official reports

Text analysis picks apart complex chains of reasoning in inquiry dossiers.
15 August 2003


Visualizing arguments can help build consensus.

A new analytical technique tests the conclusions and distils the arguments of complex documents. Designed to scour accident-inquiry reports, it could probe other long, controversial accounts, such as the UK government's Iraq dossier, or the US government's report on the terrorist attacks on 11 September 2001.

"This is a way to check that things aren't being oversimplified or hidden," says its developer, Chris Johnson, an accident analyst at the University of Glasgow, UK. The approach reveals whether or not a report's content support its conclusions.

The analysis produces diagrams of conclusions, analysis and evidence. If, for example, a report blames an accident on human error, a computer search of the whole report pulls out references to this. A reader then follows the trail of argument leading to each conclusion, and pinpoints the evidence for or against it.

In this way, Johnson has recently found inconsistencies in the 80-page document discussing the breakdown of London's computerized ambulance dispatch system. It blamed technical failure brought on by lack of testing. But Johnson's diagrams pick up references to the system's extensive testing throughout the report.

The technique has a subjective element, Johnson admits. "If two different people analyse the same report using the same technique, their diagrams will differ. But by visualizing the arguments, you can help build a consensus - it's a tool for encouraging agreement."

"It's a very helpful technique," says Peter Ladkin of Bielefeld University, Germany, who also develops computer programs for analysing reports. "Before this, people had to keep all the hypotheses and counterhypotheses in their heads, and most of us aren't good at doing that."

Accident reports sorely need such review, he says. "In about 50% of cases the official reports give misleading indications of what the causes were."

"Major accidents tend to be done very well," says Graham Braithwaite, director of the Safety and Accident Investigation Centre at Cranfield University, UK. "But more companies are looking at day-to-day accidents, and that's where the weaknesses come in. Any tools and guidance they can get is a good thing."

Representing reasoning

The UK government is currently defending the accuracy of the dossier it produced on Iraq's weapons of mass destruction last year. Following the death of government weapons adviser David Kelly, who discussed the report with journalists, a prominent judge, Lord Hutton, is investigating the case.

Good methods of representing reasoning are crucial to understanding any major issue
Peter Ladkin
Bielefeld University

Document-analysis techniques could usefully be applied to the Iraq case, says Ladkin: "Good methods of representing reasoning are absolutely crucial to understanding any major issue."

"It might be the sort of thing that should be brought to Lord Hutton's attention," says computer scientist John McDermid of York University, UK, who studies safety and security systems. But one would need all of the drafts of the report to follow the chain of argument, he says, not just the final version.

The technique can be applied to other types of data. Last week, Johnson presented a comparison of the media coverage of the 2000 Paris Concorde crash and the 2002 official inquiry to the International System Safety Conference in Ottawa, Canada1. He thinks that journalists converged so quickly on the conclusion that the inquiry reached two years later that someone with inside knowledge of the investigation must have been briefing them.

  1. Johnson C. W. Newspaper and online news reporting of major accidents: Concorde AFR 4590 in The Times, The Sun and BBC Online. Presented at International System Safety Conference, 2003|Article|

Nature News Service / Macmillan Magazines Ltd 2003

related stories
Computer program detects author gender
18 July 2003
Greeks were slaves to the beat
22 March 2002
Physicists decode the Bard
21 September 2001
Making sense of sentences
12 November 1999
links out
International System Safety Conference
more news
Flying the nest leaves birds on top
18 August 2003
New microphone detects whispers
18 August 2003
Smoking doubles TB deaths
15 August 2003
Graphs test official reports
15 August 2003
Element 110 to be named tomorrow
15 August 2003