The task should be described in a set of tables, one for the
top-level task and others for the subtasks. You can use a "task
tree" to represent the goal hierarchy graphically, but then use
UAN tables to specify the nodes of the tree in detail.
UAN has four basic columns, viz., user actions, feedback,
system state and application operations. None of your UAN
descriptions made use of this final column. It will have one
entry, recording the actual change in the column width, which is
an operation on the underlying document and not just a user
interface operation.
In specifying column selection, several groups included an
intermediate action, (~[x,y])*, between selection of the diagonal
corners. In UAN you do not have to indicate as separate user
actions all of the intermediate cursor movements. All the user
must do is move to the uper left-hand corner, hold down the mouse,
and then move to the bottom left-hand corner, i.e.,
~[top left corner of col 1] Mv
~[bottom right corner of col 1] M^
The column labelled "System State" (or "UI State") is intended
to include state, not otherwise captured in the user actions
themselves, which can affect the consequences of user actions.
Sometimes these states are referred to as "modes". In this
example, the only significant states are the selected
column, which affects the contents of the subsequent dialogue box,
and (perhaps) the activation of the dialogue box itself. Selecting
a menu by mousing down on its label creates a state which is
indicated by the description of the user action as selecting the
menu item by name; including this in System State isn't wrong, but
is unnecessary.
The UAN analysis can be used as the basis for your cognitive
walkthrough. You don't need a separate analysis called a 'Goal
Hierarchy'.
General Comments on Cognitive Walkthrough
The purpose of a cognitive walkthrough is to identify
potential usability problems. The input to the walkthrough is a
scenario, which will usually be incomplete, especially with
respect to information about what the user already knows. Often,
one won't know for sure, but can hypothesize a possible problem.
For example, we know that Kim has used Macs and Word before, but
we don't know if she has ever created or formatted a table.
Therefore, it is possible that she doesn't know how to select a
column nor the relevant menu item for column resizing. Some of you
caught this potential problem, but several groups did not.
One of the most likely problems to be encountered is the fact
that Kim must be very lucky to pick the right value after only two
tries. There is nothing in the interface to indicate the correct
value and no way of determining it other than by trial and error.
Some of you weren't sure how to handle alternative methods of
performing the task other than that described in the scenario.
Cognitive walkthrough depends on the scenario description. There
may be other ways of carrying out the task, but these are not
handled by the CW technique given here. This is a possible
weakness of this evaluation method.
In writing up your cognitive walkthrough, you should include
(i) the questions you asked and the answers you gave and (ii) a
summary report of problems found. The first section provides the
evidence for the problems in your summary and also indicates what
you believe will not be a proboem (relevant, too). The second
section is important because finding and dealing with these
potential problems is the aim of the walkthrough and a summary
makes the walkthrough much more usable.
Giving a likely level of seriousness for each problem can help
in using the cogntive walkthrough results, e.g., in prioritising
the problems for further investigation.
Comments on Group Solutions
Group A
See general comment 4 on Cognitive Walkthrough.
In your summary of problems, only the last one is a serious
interaction problem. Kim's previous Mac experience makes the other
possible problems rather unlikely to occur. See general comment 5
on Cognitive Walkthrough.
Also, you missed a potential problem in interpreting 'Cells'
as the right menu item to select.
Group B
Excellent UAN. One minor point: where a mouse down causes one
feedback (highlighting) and mouse up removes the highlighting, it
is perhaps clearer to put the two mouse actions on separate rows.
See general comment 4 on Cognitive Walkthrough. In the
cognitive walkthrough, I would have liked to have seen at least a
summary of the questions you asked and your answers, not just a
problem summary. You did, however, give the reasons for the
problems you found - good.
Group C
In task B, 'drag mouse down column' would more precisely be
'move cursor to lower right-hand corner of column'.
See general comment 2 on UAN.
In task C you've used the order indendependent operator
('&') where the order of actions is important. You can't
change the width field until after selecting the menu item, for
example.
A better decomposition of tasks would be into 'make menu
selection' followed by 'specify column width'. Also, tasks D and E
are perhaps best kept together as a single task since they are
conceptually grouped together by the mouse button 'down - up'
sequence.
Your comments in task E indicate your interest in handing
possible error states. Good. Unfortunately, UAN is not
well-equipped to specify errors and error-handling - a weakness of
the notation (research is currently underway to deal with this!).
Good identification of problems. However, see general comment
4 on Cognitive Walkthrough.
Group D
Watch your spelling! 'Descriptions' not 'Discriptions'.
Thanks for catching the bug in the scenario in column
selection. Now fixed.
Interesting comments about physical sensation. Using muscle
tension and its release is a common interface design technique to
help users group actions together (this is the reason for the Mac
style of pull-down menu selection.
I would have liked to see you use the UAN action notation,
e.g., ~[top left column corner].
In "change column width in dialogue box" task you could have
used UAN's iteration operator to specify the repetition more
succinctly.
A minor point: it's perhaps easier to read if you put the
top-level task at the beginning of your specification rather than
at the end.
Excellent cognitive walkthrough. I was surprised that you
believe users stuck not knowing what to do would refer to a user
manual or on-line help. Anecdotal and experimental evidence
suggests strongly that users in such a situation are most likely
to try almost anything but looking at the manual. The most common
behaviour is experimentation, followed by asking a colleague or
friend.
A summary of problems would be useful. See general comment 4
on Cognitive Walkthrough.
Group E
Good use of parameterisation in your UAN.
The cells command being chosen in task "choose (command)"
isn't an application operation. In terms of UAN, the appearance of
the cells dialogue box on the display is all that one can state.
A minor point: it's perhaps easier to read if you put the
top-level task at the beginning of your specification rather than
at the end.
In task "reformat table" the last three rows are perhaps
better combined as one subtask to indicate that they occur within
the context of the dialogue box. This contextual grouping of
actions is missing in your description.
You caught most of the most likely problems. However, see
general comment 4 on Cognitive Walkthrough.
Group F
You've included much more in your task description than
appears in the scenario. For a cognitive walkthrough you need only
give the task as presented in the scenario.
Tasks "select column", "set correct width" are only performed
once in the scenario. Thus, don't use the repetition operator.
Task "enter column width" has an internal loop (it's performed
twice), but the loop doesn't include clicking the 'ok' button,
which occurs outside the loop.
The feedback column should be to the left of the interface
state column.
In task "set correct width" the first action should be to move
to the title of the menu.
I would have liked to see you use the UAN action notation,
e.g., ~[top left column corner].
Your cognitive walkthrough fails to identify the possible
problems. I think this is because you've asked the questions only
about the top-level task (and goal) and not about the subtasks
(and subgoals).
See general comments 1 and 4 on Cognitive Walkthrough.
Group G
What happened to the specification of the task to enter the
column width (i.e., the task associated with the dialogue box)?
You've got 'type in new value' but this must be followed by
pressing "apply" and, after the second attempt, pressing "ok".
You should have a top-level task which combines your three
subtasks to accomplish the main goal.
By leaving out the application operations column you can't
state that the column width has actually been changed at the end
of the task.
I couldn't find a report on your cognitive walkthrough.
Group H
A reasonably thoughful coognitive walkthrough. However, see
general comment 4 on Cognitive Walkthrough.
You've emphasised the fact that visual feedback makes
exploration of methods possible. Thus, if Kim doesn't know what
the Apply button does, she can experiment and watch the effect.
This means that lack of knowledge isn't serious. However, not
knowing what to do is an interaction problem, even if the fix
isn't costly. Therefore, your analysis should include such
potential problems. Also, not all users are equally happy
experimenting. Kim, being a garden centre worker relatively
inexperienced with the system, may well be reluctant to try things
out and watch the effects. See general comment 1 on Cognitive
Walkthrough.
Your UAN columns aren't labelled. I found this a problem when
trying to interpret 'Cells Form Appears'. Since this isn't in the
same column as other feedback, is this feedback (perceivable by
the user) or not? The same applies to the other entries in this
third column.
Your UAN doesn't reflect the decomposition of goals. It would
be better to try to identify action groupings which make up
subtasks, e.g., those actions related to entering the width (in
the context of the dialogue box) form a reasonable subtask.
Group J
An excellent UAN. A neater solution would have made menu item
selection into a separate subtask. Thus, your top-level task would
have three (complex) subtasks.
A minor point: the UI State column should be to the left of
the Application Operations column.
A reasonable cognitive walkthrough. However, a little more
reflection on the scenario might have caught the likely problems
with knowing the meaning of the "Cells" menu item and the "Apply"
button, as well as the problem of having to guess the correct cell
width.
See general comment 4 on Cognitive Walkthrough.
Group K
Your "set size of column" task doesn't reflect the
decomposition of goals. It would be better to separate out as two
subtasks the menu item selection and changing the column width in
the dialogue box.
The user action "check to see if more changes needed" is not a
user action in the UAN sense. Only actions which affect input
devices go in the user action column. Checking if more changes are
needed is a cognitive action (evaluation in terms of Norman's
model of action). Unfortunately, there is no place to put such
information in a UAN description. This is one of the weaknesses of
the notation!
Good use of parameterized subtasks.
If Kim doesn't know what the Apply button does, she can
experiment and watch the effect. However, not all users are
equally happy experimenting. Kim, being a garden centre worker
relatively inexperienced with the system, may well be reluctant to
try things out and watch the effects. See general comment 1 on
Cognitive Walkthrough.
Both UAN and Cognitive Walkthrough are excellent. Well done!
Group L
Excellent UAN.
In task D, "menu item selected" isn't an application
operation, it's part of of the user interface. Application
operations in this example refer to operations on the table. The
same comment applies ot "field selected" in task E.
A minor point: "editing completed" in task E isn't properly an
application operation. This is really a comment, if you want to
add it to the task description.
Watch spelling! 'Cognitive' not 'Cognative'.
In your cognitive walkthrough you should be hypothesizing
about what is the case, not reporting. Thus, use language like,
"the user is likely to perceive the highlighted column" not "the
user perceives the highlighted column". See general comment 1 on
Cognitive Walkthrough.
In C, question 2 it is surely unlikely that Kim will have
encountered the "Format: Cells" menu item before given the fact
she's never worked on a table before. Your answer suggests you
believe there is no potential problem, but then you list it as a
problem in your summary.
Good that you caught a potential problem of the dialogue box
obscuring the column.
Group M
Your UAN has the right subtasks and is functionally correct.
However, it's terribly "wordy" since you chose not to use UAN's
actual notation for actions and feedback. You should try using it
next time.
Lack of an application operations column means you fail to
capture the one application operation, viz., changing the table
column width.
Minor point: in the cognitive walkthrough who's Kate? It's Kim
in the scenario. (Maybe Kate is her middle name?)
A clear report that catches most of the potential problems.
You didn't identify the likely difficulty interpreting
"Format:Cells" as the right menu item. See general comments 1 and
4 on Cognitive Walkthrough.
Group N
Your last action in task "resize column" (namely, "close
menu") is properly part of the "modify column width" task. Better
to show the repetition inside that task and finish with the click
ok button subtask. Also, it's not closing a menu, but closing a
dialogue box, right?
Your UAN tables should have the columns labelled.
Column re-sizing is both a bit of feedback and an application
operation, since the actual document is changed. You need an
application operation column to show this.
Minor point: in task "select column" you've reversed the mouse
button actions.
Good cognitive walkthrough. You've caught all the likely
problems and given good justifications. However, see general
comment 4 on cognitive walkthrough.
Group P
Some of the entries in the application operations column
aren't really application operations. Application operations in
this case are those which change the document, not just user
interface operation. Therefore 'column is highlighted', 'dialogue
box appear' and 'new width entered...' shouldn't be here.
Your UAN doesn't capture the repetitive nature of entering the
new column width. In your description, Kim only enters the value
once, but in the scenario she enters it twice.
Comprehensive and complete cognitive walkthrough.
Group Q
Your UAN has the right subtasks and is mostly correct.
However, it's terribly "wordy" since you chose not to use UAN's
actual notation for actions and feedback. You should try using it
next time.
Lack of an application operations column means you fail to
capture the one application operation, viz., changing the table
column width.
Look back at the UAN table examples in the lecture notes. Each
table should have its columns labelled.
Your cognitive walkthrough is not anchored sufficiently in the
scenario. We know a bit about Kim (casual user with some previous
of the application but no experience of working with tables). You
can use this information to help decide the likelihood of
problems. See general comment 1 on Cognitive Walkthrough.
Glad you noticed that your second problem was addressed in a
later version. In Word 6, it's called "Cell Height and Width..."
and the Apply button has been removed (not necessarily an
improvement!).
Group R
Minor point: the usual term is 'cursor' not 'pointer'.
The user actions in subtask 3 are a bit muddled. It looks like
some of the actions from subtask 2 got entered by mistake.
Some of the entries in the application operations column
aren't really application operations. Application operations in
this case are those which change the document, not just user
interface operation. Therefore the only proper application
operation is the one you have in subtask 3.
Your task description is terribly "wordy" since you chose not
to use UAN's actual notation for actions and feedback. You should
try using it next time
By placing the repetition of entering the column width in the
top-level task, you've incorrectly included clicking the OK button
in the loop. The OK button is only clicked once, however. Put the
loop in subtask 3, with clicking the OK button outside the loop.
Very skimpy cognitive walkthrough report (could you have done
less?).
You claim that Kim will have the required knowledge to perform
all the subgoals because she "had been using the computer." But
she hasn't used the computer to format tables before. Surely this
is an important lack of experience and will mean that she won't
know certain important things, like the correct menu item for
changing column width.
Group S
Your task description is terribly "wordy" since you chose not
to use UAN's actual notation for actions and feedback. Also, by
combining several distinct user actions in the same row, it isn't
possible to determine which of these actions is actually
associated with the appearance of the feedback in the column to
the right. You should try using the UAN notation next time.
Lack of an application operations column means you fail to
capture the one application operation, viz., changing the table
column width.
Kim enters two values consecutively in the text field. This is
not captured in your description.
A very thoughtful and complete cognitive walkthrough.
Excellent. A summary of problems found would be helpful, though
(see general comment 4 on Cognitive Walkthrough).
Group T
Good use of subtasks.
See general UAN comment 2.
In task A, your first action ('~[topleftcolumn]) is redundant
since it is also the first action in subgoal B.
A minor point: it would be better to annotate the subtasks
with their identifier after the user action, thus: 'open cells
dialogue (D)'. This makes it easier to read the user action
columns.
Re: subtask B, see general UAN comment 3.
References to figures in tasks D, E and G should be
annotation, not entries in the user action column.
In subtask D, a comment similar to comment 3 applies, viz.,
after specifying a mouse down over the menu title, just use
'~[Cells item]'; you don't have to specify the intermediate
positions traversed by the cursor.
Good to see references to the affordances of screen objects,
like a change in cursor appearance.
See general Cognitive Walkthough comments 1, 2 and 4.