ASSIGNMENT
4: METHODS AND MEASURES
Participants
Procedures
Task Scenarios
Test Measures
Participants
While
we did not seek out participants who explicitly fit our user profile
and project personas, we wanted users who shared a general set of
qualifications that would indicate they were both familiar with
the related technologies and capable enough to interact with our
system. We chose our participants based on the following general
criteria:
- Participants
should fit our user profile age range
- Participants
should have access to a digital camera
- Participants
should be cell phone users.
We
actively sought out people we knew, but who were not close friends
or SIMS students (using close friends or fellow students could have
led to a biased sampling of test cases). It was coincidence that
all participants were female. All participants were personally asked
by members of the Photocat team to take part in usability testing.
(From past experience, we've found this to be a better way of recruiting
participants compared to sending out impersonal mass e-mails.)
 Participant
#1
- Female
- Early
20s
- Single
- Student
- 5th
Year Undergraduate
- Environmental
Science Major
- Lives
on UC Berkeley campus in Graduate Student Housing
- Takes
and shares digital photos
|
 Participant
#2
- Female
- Early
20s
- Single
- Full-Time
Employed
- Lives
with in Berkeley with sister
- Doesn’t
own digital camera
- Uses
Poloroid and regular 35mm camera
|
 Participant
#3
- Female
- 33
years old
- Single
- Lives
in Alameda
-
Graphic Designer/Full-Time Student
- Pre-Law
and Creative Writing
- Owns
a digital camera
|
TOP
Procedures
Our
testing procdures were divided into three parts: Introduction,
Task Scenarios, and Debriefing. The text that follows
served as our script.
Introduction
Thank
you for agreeing to help us test PhotoCat, our class project for
IS 213--User Interface Design and Development. Introduce the
team members.
PhotoCat is a Web-based photo management system that allows you
to view, manage, and share your digital-camera photos. You access
PhotoCat using a Web browser.
Within
the PhotoCat interface, you can organize your photos by:
- The
time they were taken.
- The
location where they were taken.
- The
people who were present when the photos were taken.
During
this testing, you will be interacting with a large-scale, paper-based
version of PhotoCat. One of the team members will be acting as the
“computer,” and will make updates to the paper interface
as you interact with it.
The PhotoCat interface includes:
- A
novel "Bullseye View" that displays all or some of your
photos based on the time they were taken, the location where they
were taken, and people who were present when the photos were taken.
-
A set of navigational menus and buttons that allow you to filter
and organize the photos being displayed in the interface.
-
A "Photo Bin" in which you can view and share photos
of interest.
-
A popup "Details" display that allows you to view a
large version of your image and associated information.
We
will be asking you to use PhotoCat to perform several photo-management
tasks. These tasks will involve finding specific photos in your
collection and possibly saving them as groups or sharing them with
others.
Please
"think out loud" while you perform your tasks, as it will
help us understand your interaction with the interface. You can
use your finger as your cursor in this experiment.
You
are free to refuse to participate or quit the experiment at any
time.
Before
we start, we’d like you to read and sign this release that
explains more about what this testing is all about. Give the
participant a copy of the release for reading and signing.
Task
Scenarios
Our
participants completed four task scenarios. See below
for details.
Debriefing
Ask
the participant about the different aspects of the PhotoCat interface:
- Bullseye
-
Circle design
-
People Petal icons
-
Photo Bin
-
Time
-
Location
-
Album
-
Friends
-
Actions
-
Popups
Other
questions:
- What
was particularly easy about using the PhotoCat interface?
-
What was particularly difficult?
-
Have you used other photo management applications? Which ones?
-
How do they compare to the PhotoCat interface?
Ask
the participant if they have any questions or comments. Thank them
for taking the time to test the PhotoCat interface.
TOP
Task
Scenarios
We
designed our task scenarios so that they met the needs of our personas—especially
Darla Garcia who uses photos
flexibly to meet a broader variety of needs than some of the others.
Each task we set for our paper-prototype interviewees has specific
relevance. The relevance is broken down by task below:
1.
Today is Monday and earlier in the day you went to a botanical
garden. Your pictures automatically upload to the PhotoCat website
when you take them. One of the pictures you took was a view of
a pink rose. You want to see how this photo turned out so you
need to find and view the large-size photo to be sure it’s
not blurry. If it’s good you want to share it with a friend
who loves roses.
Discussion: This task is especially relevant because all
of our interviewees and personas mentioned viewing and sharing
photos soon after they were taken. The task is the simplest of
the four—to help our interviewees gain confidence using
the paper-prototype system. The ideal results are that the photo
needs to be found and shared very quickly with little effort.
The task approximates the base task for Darla—that of finding
good photos from earlier in the day and sharing them. In fact,
it is even easier than her base task since only one photo needs
to be viewed and shared.
2.
You took some camera-phone pictures at a party on Friday. The
party was held at your friend’s house in Oakland. Today
is Monday and you’re at work. You want to browse through
your party pictures and pick out the good ones to share with your
friends who were there. Your photos were automatically uploaded
to PhotoCat when they were taken on Friday.
Discussion:
The second tasks adds some complexity to our testing that
the first does not have. It asks the interviewee to find a photo
from an event within the past 3 or 4 days and it includes information
about the place for the event it mentions (Oakland). Since most
personas organize things primarily by time, this task can be completed
fairly easily since the time period is still recent. We desired
to see if our interviewees would access the photos still primarily
by time or also use the concept of place. The task scenario includes
a party event in the description since our personas often recording
some kind of event when taking photos. For instance, Darla was
recording a kind of event with Mario’s first day of school,
and Jake frequently documents parties. The task adds complexity
by asking the user to share more than one picture with more than
one person.
3.
Last summer (July 2004), you and your family took a vacation to
the Grand Canyon and other national parks. Now it is March, 2005,
and you want to look back at your photos. You want to save the
best ones (about 5) from the Grand Canyon.
Discussion:
Task three moves the desired photos back further in time (to
a family vacation last year). It requires the user to figure out
how to access different time scales in the interface. The task
was designed to see how people work when trying to deal with much
longer time frames. The longer time view may present much more
information and require the user to attempt other techniques to
find and share the specific desired photos. The task is based
on the persona of Steve Steep more than the other personas. It
evaluates the sharing of a group of photos.
4.
Your friend Steve visited you in Berkeley three weeks ago. He
posed as a student and you took pictures of him and some pictures
of the campanile. He asked you to share the pictures you took
of him because he wants to use them on his Web site.
Discussion:
The task deals with an intermediate time frame (within a month),
specifically mentions a person (Steve) and also mentions a place
(Berkeley). The task again requires the user to switch time scale
(or filter by time scale) and additionally may allow easier access
by filtering by people or place. Though most of our personas organized
by time, some interviewees mentioned captioning or renaming photos
based on the people in a photo. We wanted to see how interviewees
accessed this intermediate time frame and find out if they used
additional filtering in order to narrow the search. Again we asked
them to select and share a group of photos. The task allowed filtering
in many ways.
Test
Measures
As
shown in the paper prototype images below, the PhotoCat interface
has many elements for the user to interact with. On the highest
level, we were looking to see if our testers understood the relationship
between the "Bullseye View," the "Photo Bin,"
and the "Filter Menus" portions of the screen. The Bullseye
View was designed as the primary browsing interface allowing the
user to see thumbnails of all of his or her collection during a
specified time frame. The Photo Bin was designed as a holding area,
allowing users to view selections of pictures and perform actions
on them. The Filter Menus were designed as ways to decrease or increase
the number of photos shown on the bullseye view.
 |
Bullseye
View
- Does
the user understand the logic of each of the time layouts
(day view, week view, month view, year view, 5-year view)?
- Can
the user find a specific picture on the target?
- Does
the user understand what the bands on the target mean?
- Can
the user figure out how to move a picture from the target
to the Photo Bin?
These
questions were important to the team because although we understand
the time centered target interface, we were not sure if it
would be an understandable and logical information visualization
tool for our testers. |
 |
Photo
Bin
- Does
the user understand what the Photo Bin is for?
- Does
the user understand how to perform actions to the Photo
Bin?
- Does
the user understand the relationship of the Photo Bin to
the target?
Our
team members have used the bin concept in other photo tools,
and thought it was useful. We wanted to make sure that our
testers understood the relationship of the Bullseye View to
the bin, and what they could do with the pictures once they
got them into the bin. |
|
Filter
Menus
- Does
the user understand the basic functionality of the Filter
Menus?
- Do
they use more than the date function?
- Do
they understand that the actions grouping is supposed to
be tied to the Photo Bin and not the target?
The
panel that is the furthest right was used for filtering thumbnails
on the target and performing actions to the pictures in the
Photo Bin. It was important to see what the user did with
each of the widgets, what they understood, what they misunderstood
(suggesting we'd have to make changes), and what they didn’t
use at all. |
TOP
Next:
Results and Discussion |
