Method
..................................................................................................................................
Participants:
Working with limited time and resources, we tried to find three
participants who match our personae and target audience as closely
as possible. We tested two female and one male participants. Two
of the participants are between 31-45 years old, while the third
between 21-30. All three enjoy going out to bars and clubs, sometimes
for various events. Since our ultimate target customers are Interent/web
savvy, we also looked for relevant experience from our testers.
All three testers have been using the internet for more than five
years, and all have used entertainment sites such as Sfgate and
Citysearch. However, only one tester has created a personalized
page with websites such as MyYahoo.
Task
Scenarios: Based on task scenarios created during an earlier
phase of the project, we decided to test three tasks.
- Sending
some options of things to do to friends
- Signing
up for a newsletter, and creating
- Customizingand
exploring the MySFnight page.
The tasks
are in order of level of customization. The first task requires
no identifying information and uses a feature that is temporary
in nature. The second tasks requires only an email address. The
third task requires complete registration, perhaps some preference
selection, and provides users the option of adding specific events
and venues to their MySFnight page. Both Tasks 1 and 2 also provide
reasons for users to sign up with SFnight and access to the registration
page. As a result some users may complete parts of Task 3 before
getting to that step.
Task
1: We decided to first test how well a user can create a list
of options and email the choices to friends. This feature is available
to all visitors of SFnight, whether or not they have a customized
account with SFnight. This feature is also intended to encourage
users to start developing a meaningful relationship with SFnight.
Task
2: For users who may be reluctant to put too much effort into
customizing SFnight, but would still like to receive some relevant
information, we provided a short cut to
receiving a general newsletter. We wanted to see how easy it was
for users to locate the feature, and how well they liked the simple
process.
Task
3: This task involves the most of amount interaction. We were
especially concerned with the registration process. Most sites
loose customers because of the amount and nature of information
asked, indentification and other wise. On the other hand, we needed
to strike a balance with the information that is needed in order
for the service to be useful to a user. We have also been struggling
with how to combine functionalities in Task 1 with the MySFnight
page without confusing users which list is which. We wanted to
see if the users could help us better define the features.
Procedure:
We prepared a test kit for each user. The kit included a consent
form, a simple demographic and Internet/Web experience survey,
instructions to each task, surveys for each task and the overall
site, and reminder instructions for the team members. (See
Kit)
We scheduled
two testers on a late Friday evening and one more on a Tuesday
evening. When a tester arrived, a facilitator greeted the individual,
informed the individual the status of the system (there were a
few hangups when booting the prototype) , and asked the tester
to sign a consent form and complete the simple background survey.
When the system is ready, the facilitator brought the tester to
the testing location, introduced other team members and their
roles and explained how to operate the "computer".
The facilitator
went over the task instructions with the user, and explained any
parts of the instruction that the user may be confused about.
The user was encouraged to speak out loud as he/she works through
the task and their perceptions of the system. After the user completed
a task, he/she was asked to complete the task survey. When all
tasks were completed, and the user has completed the site survey,
team members explained the goals of the test and the overall site.
Then the team discussed in depth with the user various issues
that came up during the test and also asked for suggestions for
the design.
Part
of the conversations were recorded, a few photos were taken (see
photos), and team members who were observing took notes.
