1: Naive Usability Assessment
(Weeks 1 & 2, Due January 28, 2003)
Assignment 2: Interviewing
(Weeks 4 & 5, Due February 25, 2003)
Assignment 3: Surveys
(Weeks 6 & 7, Due March 18, 2003)
Assignment 4: Heuristics
(Weeks 9 & 10, Due April 1, 2003)
1: Naive Usability Assessment
The purpose of this assignment is to get you thinking in terms
of usability, and noticing as a user usability issues.
During the next week, pick two
times that you use some sort of technology or information system:
a web site, an ATM machine, a car...the possibilities are many.
Choose one that you have used many times in the past, and one
that you are using for the first time. For the length a transaction,
or about 15 minutes, whichever is less, notice and record usability
issues or problems. That is, for the length
of this exercise, pay attention to such issues as the following:
- instances in which you had
to modify your own activity to suit the needs of the system
- moments of confusion or uncertainty,
such as questions about what actions you should take or what
a display or system response means;
- inconveniences, instances
requiring added effort from you -- physical (e.g., doing something,
or doing something in an awkward or effortful way) or mental
(e.g., having to remember something that takes some effort).
- annoyance, thoughts such as,
"why do I have to do this? why can't I do that?"
- added effort or attention
of any kind.
When you are using the system
or item with which you have experience, these may be harder
to notice. Pay attention to the ways in which you have been
trained by it to do things its way, the accommodations that
you are now used to making.
During the exercise, simply
keep track of these instances as they arise. Then summarize
your observations in a paper for me, spending a page or two
on each of the two instances. Then write a page or so reflecting
on what you learned about usability from this exercise. The
point is not to apply any of the readings, but to notice
your own experience as a user.
Stay with noticing the disjunctions
between how you act and how the system works; do not leap into
redesigning it to work better. This is hard to do; often our
tendency is to focus on how to make it better without first
noticing the problems. But premature re-design can lead to suboptimization:
we solve some problems while creating new ones.
Due Tues, January 28, 2003
up with another student in this class, preferably someone you
don't know very well, and take turns interviewing one another.
Aim for interviews of about ½ hour to one hour in length. TAPE
RECORD YOUR INTERVIEWS. I'll have one tape recorder in the computer
lab for you to check out; those of you who have machines, it
would be nice if you would share them with your fellow students.
Do NOT use the same topic for both interviews. The best topic
is one related to your project, so that this is in some ways
a dry run for interviews you will do later. If that's not feasible,
pick a topic (or two) of mutual interest about which you can
ask a number of questions. ('Why are you at SIMS?', for example.)
Before the interview, develop an interview schedule, i.e. a
set of questions and their ordering. During the interview, you
may need to re-order.
at least part of the interview. The rule of thumb is that it
takes three times as long to transcribe as the interview itself,
so you don't have to transcribe the whole thing, but you should
spend at least an hour transcribing. We'll have 1 or 2 transcription
machines available in the computer lab.
pair please turn in your papers together so that for #6, below,
I have both the interviewer's and interviewee's reports.
A short statement about the goals of your interview.
If applicable, a revised interview schedule: how you would
do it differently were you to interview someone else. (This
includes turning in a revision to show how you actually asked
the questions, as opposed to how you planned to: changes in
wording or question order.)
A report on the interview for which you were the interviewer.
Not the transcription, but a summary of key findings.
(2 parts) Two reflections: one on being the interviewee, one
as the interviewer.
As interviewee, a page or so of feedback to the interviewer:
what advice would you give him/her on questions, but especially
on behavior, attitude, and the like during the interview?
(Give them a copy of this, as well as one for me.)
In #6, reflect on such questions as:
OVERALL: in retrospect, what worked, what would you do differently,
CONTEXT: when and where and under what conditions did you
do the interviews, and what difference did it make?
THE RELATIONSHIP BETWEEN YOU: you had a prior relationship
as fellow students, and you also developed a relationship
during the interview. How did the relationship develop? How
did your prior relationship - and the fact that you have an
on-going relationship as fellow students - affect the interview?
QUESTION CONTENTS OR TOPICS: what topics proved fruitful or
difficult; which ones were helpful, which were not; which
topics you found easy to address, or difficult, or too personal,
or otherwise problematic.
QUESTION WORDING: which questions you found difficult to word
or understand. E.g., as interviewer, questions that were hard
to word well; that the interviewee misunderstood; as interviewee,
questions you found hard to understand, or to answer.
QUESTION ORDERING: what difference the question ordering made;
what re-ordering you had to do (or would have wished for)
on the fly, and why.
This is where the revised interview schedule comes in: how
would you re-word and/or re-order the questions?
TIME: did you have too much time, or too little? What different
might it have made to have had more or less time?
OVERALL: what did you learn?
Due Tues, February 25, 2003.
Please be concise while thorough; no part of this needs to be
Pair up with someone else from the
class. If you are doing a group project, work with another person
from your project. (If you have an odd number of people on your
team, 3 of you can work together.)
Working together, draft a short survey:
about 10 questions. If at all possible, it should be related
to your final project, regardless of whether you actually plan
to make this a part of your project.
Before you turn it in, pre-test it on at
least 3 people. You want them to be as much like your target
users as possible, although time and other constraints may mean
that you only use your fellow students. Then revise the survey,
if necessary, based on your pre-tests.
- A short rationale: what is your purpose,
what information do you want to get, and WHY? How will you
use this information once you get it?
- A description of your target group,
if you were to do this "for real," and what your
sampling method would be. Define the following, as applicable,
for your survey:
- The version that you pre-tested.
- A brief report on your pre-test.
- On whom did you test it and how?
How does this differ from how you would pre-test it if
this weren't a class assignment? E.g., on whom would you
- What did you learn about (a) the
questionnaire, and (b) the content area, your questions?
- What did you need to revise, and
- The revised questionnaire.
Due March 18.
you are doing your final project as part of a group, work with
that group. If you are not working in a group, you have the
option to do this individually or in a group. (Doing it in a
group is good experience.)
your project is to evaluate an existing website, use that site.
If it is not, find a site to evaluate. If possible, find one
that is related to your project, such as a competitor or a site
that does something related to the topic of your project.
will apply two or three sets of heuristics to your site: Nielsen's
and at least one of the following:
set of heuritics for the kind of site yours is, if you can
find one. (This requires doing some web searching.) . See,
for example, http://www.stcsig.org/usability/resources/toolkit/e-learning-checklist.doc
set of heuristics that you develop, specific to your site.
If you do this, you need at least five.
Using these heuristics, evaluate the site.
If you are working with your project group:
work separately. Each person will write a short report (for
me) summarizing your findings.
come together and consolidate your work. Write a 2-page report
summarizing your shared findings, INCLUDING any needed revisions
to the heuristics.
Findings must be tied to the heuristics violated; they can't
simply be 'good ideas' or 'obvious.' Include specifics of the
heuristic violation, the violation itself, severity rating,
and recommendations for improvement.
you are working in a group that is NOT your project group,
pick either one member's project or pick an application or website
that has nothing to do with your projects. Then follow the instructions
above: first apply the heuristics separately, then come together
and consolidate your findings.
If you are working individually, you will do the same
but without the group consolidation step.