User
#1, Task #1
(Find a class about networking)
User Comment:"I
would look for IS250."
The user browses
instead of searches for the class. The user scrolls the default course
list on the homepage.She clicks on IS250 link.
User Comment:"It
wouldn't be clear to me that I wouldn't get access to the comments [when
clicking on the IS213 link].It is nice that I came upon [the login page
or the comments??]."
User taken to login
page. The testers realize they forgot to put in a "Submit" button on
the login page.
User Question:"Am
I a new user?Would prospective students be able to register?"
User fills in login
screen. Comment:"I hit 'Submit'." Taken to page with comments regarding
IS250.
User notices that
the "Add Comment" button is no longer grayed out after she logged in.
User Comment:"I
would want to know what the course rating scale was. Nice visual."
User noticed that
the name of the person who submitted the comment is included by the
comment.
User Comment:"I'm
a little confused.There are no ratings by the individual comments.I
would want to see the information pertaining to each poster. I wouldn't
care about the comment until after I read it.The name of the poster
shouldn't be in bold."
User said she wanted
to see the course rating prominently.
It took a while
for the user to notice the course recommendations that are above the
comments. The user liked it but said she wished she had seen them earlier.
She wanted to see the actual name of the recommended class and not just
the course number as is now shown.
User
#1, Task #2 (Search for User Interface Class)
User comment:"I
keep looking at 'Search'. I don't want to mess with that. I don't know
what 'Course Description' means." Therefore, user does not perform a
search.
User looks at navigation
but wants to avoid it. She comments that she doesn't know what the "Shopping
Cart" button means.
User comment:"If
I was using the system for the first time, I would want to play with
it. I would want to post a comment to IS250."
User clicks on "Post
a Comment" button.
The user assumed
she could just submit the comment and not have to preview.
User question/comment:
"What does 'Preview' do? I would want to choose 'Enter' or 'Submit'."
User selects "cancel"
button.
User comment:"I
need to look at lefthand navigation.I don't know which one I would pick.I
guess I could look under 'Subject' and find 'User Interface'. I don't
know where I would go if I choose 'Subject'".
User chooses "Submit"
button??
User confused about
what semester she should be looking for. She is told to look for Spring
2001.
User clicks on "IS213"
link.
User comment:"If
I wasn't looking for courses for next semester, I would browse 'All
Courses'. From the IS213 page, I would want to find other UI classes
offered this semester."
User clicks on 'Subject'
button.
User comment:"I
don't know what UI would be under" while looking at subject list on
default course subject page.
User comment: "I
would think core courses would be under other categories [other than
the ones listed]".
User comment:"It
seems like IS213 is the only UI classes but that's not true."
User
#1, Task #3 (Post a comment)
User comment as
computer changes screens to homepage: "You took away the 'Add Comment'
button? I would like to add a comment to another class but can't."
Clicks on IS250
link. Clicks "Add Comment" button.
User comment:"I
wish 'Add Comment' would be by the comments and not on the [left hand]
side [navigation bar]."
When filling out
course rating information, user states she finds it hard to find out
which graphic corresponds to which radio button.
The user clicks
the "Preview" button.
User comment:"Is
this the most 'Thumbs up' possible? I would want to check this."
User selects the
back button on the browser in order to verify the 'Thumbs up' issue.
Hits the "Preview" button again.
User comment:"I
might be torn between the 'Forward' button and 'Preview Button'"
User hits "Submit"
button.
User
#1, Debriefing
Question:"Is it
better to not have a search button?"
User Answer:"I would
rather work with what's right in front of my face."
User says she was
confused by default homepage. She says she didn't know what would come
up when she clicked on the IS250 link the first time.
User asks about
the way the courses were classified in the default course subject page.She
says that course dealing with user interface design shouldn't be under
the "Users and Society" course heading.
User question:"What
the difference if I hit the 'All Courses' button?"
User question:"Does
the site contain the whole UC Catalog?"
User comment:"I
would have no idea what outside classes are doing [on a SIMS course
site]."
User says she would
prioritize the way the courses are grouped (on the 'All Courses' page).
She would put SIMS courses first.
User commented on
how toolbar changed while she used the site.
User said that this
site would not be relevant to new students since first year first semester
courses are fixed.
User comment:"Maybe
comments on core courses would be good."
User comment:"I
don't get 'Search' part."
User comment:"I
am only concerned with course categories [the bottom toolbar on the
left hand side of the page] to do my tasks. I am confused by the first
toolbar. The second toolbar is devalued by being in bottom left hand
corner."
User says that when
she views a site, she starts at bottom right, then looks up, and then
left and then down. Therefore, the main navigation toolbars are the
last thing she saw.
User
#2, Task #1 (Find a class about networking)
User Question: Is
this the first time I access the site?
Answer: Yes. But
we won't have to register you…
User chooses to
browse the system.
Clicks on "All Course"
button. Clicks on IS250 link.
User Question:"Do
I have to login first when I click on course name?"
Answer:"No."
User logs in. Sees
course page for IS250.
User wonders what
do the thumbs means? What is the scale 0-5, etc.? He knew that thumbs
up was good but didn't understand the grouping of thumbs.
The user was later
asked- "If you knew what the rating scale was, would you want to compare
the rating of this course to other courses?"
User Answer:"Yes."
User wants to look
at other course to compare to IS250. He wants to see how people review
IS202/204.
User navigates to
"Required Course" button and selects it.Selects IS202 link from the
page that comes up.
User
#2, Task #2 (Search for User Interface class)
I noticed the user
tried to click on the word search instead of first typing search terms
into text box.
The user wrote "User
interface, usability" into "Search" text box.User did not pay any attention
to "Course Description" attribute next to the text box.
User clicked on
IS213 link on search results page.
User browses IS213
page.
User Question: "What
is the link at the bottom?" User is referring to the IS213, Fall
2000 link. User said he would click on it. User clicks on it.
User clicks the
"Add Comment" button. Taken to add comment page.
User clicks "Preview"
button. Clicks "Submit" button after reviewing the comment preview page.
User
#2, Task #3 (Post a Comment)
The user confidently
proceeded through the task without noticing that the displayed class
(IS213) was not the same as the class mentioned in the task statement
(IS250). That is, he easily and successfully added a course comment,
but to the wrong course.
User
#2, Debriefing
User comments about
problems he had had figuring out course rating scale.
User Question: "Who
would login to the site- only SIMS students?"
User Comment: "I
would like to see a lot of postings. One or two would be a good indicator
[of quality of class, etc.]."
User mentions privacy.
User liked recommended
courses features. It provides a correlation. It gives an impression
about other classes.Would like to see what each person recommends.
User says he would
like to view all comments posted by a particular person.
User
#3,
Task #1 (Find a class about networking)
User question:"Can
I be anonymous?"
User comment: "I
would type 'networking' in your search." Types in info, press "Submit"
although says she would like to just press the "Return" key.
Selects IS250 link
that appears on search results page. Logs onto system and submits. User
comment: "I assume I'm not a new user."
Looks at description
and comments for IS250 a long while.
User suggests that
the e-mail address of the poster should be added. However, e-mail addresses
might change (when people graduate, etc.).
User question/comment:
"How do I get out of the screen? Would I want to close the entire thing?
Would I need to login again? I'm looking for a way to exit or return
to the course listings."
User question: "Has
[the site] changed? You changed something." We pointed out the various
changes that go on when people login, etc.
User comment: "Always
give people exits. I wouldn't want to go back to listing of courses."
User
#3, Task #2 (Browse to find a User Interface class)
The user scrolls
and picks the "Users and Society" subject heading link from the homepage.
The user said that she was looking for a heading of "HCI".
User says she found
numbers by course titles confusing.
User clicks on IS213
link.
User question after
being taken to IS213 page with comments: "I got this page because I
already logged on?"
User says she liked
the related courses feature. She says she would click on one of the
recommended classes.
User
#3, Task #3 (Post a comment)
User comment: "I
still need an exit to get out." She presses an imaginary exit button
and we show her the default course subject page.
User scrolls and
clicks on IS250 link.
User comment: "I
always want my actions to be on the right hand side. You do things left
to right."
User presses "Add
Comment" button. User has no problem filling out the comment. However,
she hesitates when it comes time to preview the comment.
User comment: "I
want to just say I'm finished."
User hits "Preview
button. User submits comment by clicking on "Submit" button.
User
#3, Debriefing
User says she thinks
that the comments should be made anonymously. She doesn't want retribution.
Discussed posting policies for bulletin board which she managed- no
hate speech allowed, etc.
She discussed the
buttons on the post a comment page. She says she just wanted to finish
and submit the comment without previewing it. She also didn't like the
use of the word "preview". She agrees, however, that it might be important
for someone to see what she's done. Perhaps the comment could be placed
next to the comment preview on the same screen. Allow user to edit comment.
User says she didn't
have a problem finding the "Post Comment" button.
User says she wanted
to exit the screens.
User says there
should be a space that says "Protect my privacy" somewhere on the site.
User says she would
have problem making a negative comment if comments were not anonymous.
When asked, the
user says she didn't notice that the individual poster's ratings were
not posted, only the aggregate ratings. She said that she would simply
like to glance at the comments and tell if the course is good or not.