| [an error occurred while processing this directive] |
Tester
1
Tester 2
Tester 3
Tester 4
General comments about all testers
Tester
1
Task 1
Actions:
- Clicked on
"login" link on left of page.
- Fills in fields,
uses mouse to click "submit" button.
- Types is250
in search field; hits return; search unsuccessful.
- Types IS250
in search field; hits return; search unsuccessful.
- Gives up searching;
chooses "browse by course number" instead.
- Finds IS 250
and clicks on link.
- Scans ratings/comments
Comments:
- Wanted to hit
return when submitting username and password, but submit button
did not have focus.
- Liked that
"submit" button for searching did have focus.
- Didn't understand
difficulty ratings. Is a high number more or less difficult?
- Liked the details
of the comments, how they expressed students' feelings.
Task 2
Actions:
- Enters "user
interface" into search field and submits.
- Gets three results;
clicks on IS 213.
- Sees no ratings/comments
posted so goes to IS 213 homepage on SIMS website.
- After looking
at current year's syllabus, hits back button to return to CoCoFo.
- Sees IS 245 is
recommended by students who recommended IS 213 so clicks there to
see comments.
- Since IS 245
is not a similar topic, user returns to search results page by hitting
back button.
- Clicks on each
of the two other courses returned in search results, looking at ratings
and course homepages.
Comments:
Task 3
Actions:
- Finds and clicks
on "rate a course…" button.
- Chooses IS 206
from the pull down menu.
- Inputs ratings
and comments. One of the radio buttons does not take the ratings.
- Clicks "preview".
- Decides to edit
comment. Clicks "edit".
- Edits comment
and clicks "submit".
Comments:
- Wanted to be
able to type in the name of a course when choosing a course to rate,
rather than being restricted to pull down menu.
- Liked the large
text box for adding a comment.
- Noticed that
text doesn't wrap when entering a comment.
- Noticed that,
nevertheless, comment text is wrapped when comment is submitted.
- Wondered if she
needed to log out of system.
Post test interview
General comments:
- Likes color of
the interface. It's comfortable and doesn't shout out.
- Would like a
way to email students who have posted comments.
- Would like to
see graphical distribution of ratings, not just average of ratings.
- Would not generally
respond to individual comments unless had extremely strong feelings
about the comment.
- Would like a
one line explanation of the difficulty ratings, but that the other
ratings were self-explanatory.
- Did not understand
what was wrong with the search results, whether search was case-sensitive
or not.
- Would find the
system useful if implemented.
General observations
of tester:
- User had problems
searching.
- User didn't notice
a missed click on instructor ratings.
- User used a lot
of the links to course home pages.
- User did not
seem to notice "add a comment" link on course description pages.
Tester
2
Task 1
Actions:
- Looks at search
function but does not use it.
- Clicks on "browse
by course number" link and finds IS 250; clicks on IS 250.
- Logs in to read
comments.
- Reads comments
for IS 250.
Comments:
- Assumed logging
in would show comments.
- Noticed and approved
of recommended course feature.
Task 2
Actions:
- Chooses not to
search for course.
- Clicks on "browse
by subject" link.
- Doesn't see "user
interface" subject so scans all courses and finds IS 213 listed.
- Clicks on IS
213 to read comments.
- Decides to conduct
a search; enters "user interface design" and submits search.
- Clicks on CS
UI course.
Comments:
- Only found course
by scanning, not by searching.
Task 3
Actions:
- Finds and clicks
on "rate a course…" link.
- Selects IS 206
from drop down menu.
- Scans the ratings
form and fills in the fields.
- Previews rating
and submits.
Comments:
- Noticed that
the text box doesn't wrap and wondered if carriage returns were needed.
Post test interview
General comments:
- Liked the CoCoFo
system.
- Would like to
edit, append, or delete own comments. Users might regret what they
have written at the spur of the moment.
- Wondered about
registration and how users would be assigned user names or whether
they could be picked out by user.
- Wondered how
site would be advertised/marketed.
- Discussed the
pros and cons of being able to post anonymously. Anonymity might engender
stupid comments. Non-anonymity might limit what is said, but in a
good way, limiting to constructive criticism.
- Would like to
have the ability to respond to individual comments so that conversations
could develop. If conversations became irrelevant, they could be taken
outside of system.
- There would be
times when it would be suitable to reply to a comment, and other times
when it would be more suitable to reply in a personal email.
- Vaguely understood
the ratings system. Would rely more on comments than on the ratings,
and therefore wasn't too concerned with the ratings system. Another
reason why non-anonymity would be useful - so that you would know
what type of personality was posting what.
- Thought the ratings
system might have gone up to 10, not just 5.
General observations
of tester:
- User first browsed
for a course and then searched for a wider selection.
Tester
3
Task 1
Actions:
- Attempts to log
in but is unsuccessful because of invalid username (our fault).
- When valid username
used, logs in and reads information on entry page.
- Clicks on "browse
by course number".
- Clicks on IS
250 and reads comments.
Comments:
- Thinks system
is very nice.
Task 2
Actions:
- Does not search.
- Clicks on "browse
by subject" link.
- Scrolls to bottom
of page and back up again before seeing IS 213 link.
- Clicks on link.
- Reads comments.
Comments:
Task 3
Actions:
- Clicks on "browse
by course number" and finds IS 206; clicks on link.
- Clicks on "add
a comment to this course" link on the IS 206 page.
- Fills in ratings
and comment.
- Clicks on "submit"
without previewing comment first.
- Scrolls down
to see all of the IS 206 comments.
Comments:
Post test interview
General comments:
-
Was uncertain
about when login was required.
- There was no
indication that login was necessary on the entry page.
- Didn't find course
subject categories very useful. Didn't know where IS 213 would fall,
so scanned courses rather than categories.
- Thought there
should have been a User Interface category.
- Would like to
see more information about each user who had posted a comment, such
as their background, education, and degree status.
- Would not be
interested in responding directly to a comment.
- Would like to
have seen email address of commenters and possibly whether they would
be interested in accepting emails from other students.
- Would like to
see a rating of the course load, e.g. how much time per week was spent
on the course.
- Found the rating
system intuitive. Liked the 1-5 scale, and 1-10 scale would be too
large.
- Liked aesthetic
of CoCoFo; nice and minimal, not much clutter.
Tester
4
Task 1
Actions:
- Clicks on "login"
link on left, inputs information and submits.
- Clicks on "browse
by course number".
- Finds IS 250
and clicks on it.
- Notices links
to course home page and professor home page.
- Scans comments
and ratings.
Comments:
- Thought login
was a way to browse (because of location under "browse by") until
gap between "browse by" links and "login" link was noticed.
- Wanted to be
able to hit "enter" to submit login information.
- Likes the protection
of the system that needs a username and password to enter, but would
like to know who the user base is, i.e. who will see comments and
ratings.
- Found the "browse
by course number" listings well organized.
- Understood ratings
system.
Task 2
Actions:
- Clicks on "browse
by subject" rather than search.
- Finds Human Computer
Interaction courses after scrolling to bottom of page and back up.
- Looks further
for individual User Interface group of courses, but returns to HCI
courses when none is found.
- Does browser
search for "interface" and finds a CS UI course.
- Clicks on IS
213 and reads comments.
Comments:
- Wondered why
there wasn't a specific User Interface subject heading.
Task 3
Actions:
- Clicks "rate
a course…" link.
- Clicks on drop
down menu and chooses IS 206.
- Fills in ratings
and comment fields.
- Notices that
the text does not wrap and adds carriage returns to force wrapping.
- Clicks "submit"
without first previewing.
- Scans through
comments to find just-submitted one; goes through to bottom and back
up to top before seeing it.
Comments:
- Wondered if non-wrapping
in text box was a problem.
- Found the drop
down menu well organized.
Post test interview
General comments:
- Finds the system
offers many useful features, it meets a real need, and is very "robust".
- Finds that it
is somewhat controversial, a good thing. Anything that requires a
login should be somewhat contrtoversial. But because login requires
a compelling reason, should know what the material beyond the login
is.
- Would like to
know more about the system - e.g. who has access, for whom is it meant
- before seeing it.
- Would like to
see a "rate the rater" or "rate the comment" function.
- Likes that it
is not anonymous; would have less trust in comments by an anonymous
user.
- If anonymous,
or if threaded, would be afraid of flaming wars.
- Would possibly
like email links to commenters, but would make it an "opt in" system.
- Would like to
edit or delete own comments, but if edited or deleted, would like
to see a flag indicating such changes.
- Thought workload
rating would be useful.
General observations
of the tester:
- User spent a
lot more time than the other testers looking around and browsing before
focusing on given tasks.
General
observations of all the testers
- No one tried
a different search term unless the search returned zero items. This
might be because the users were satisfied with the results, or because
all of the users have a general knowledge of the courses that could
be returned. Therefore, they know that when user interface is searched
for, IS 213 should come up. They may be satisfied when it does come
up and not be compelled to search any further.
- Three of the
four users used the verb-noun interaction style for adding a comment.
Only one of the users even seemed to notice, let alone use, the noun-verb
interaction style.
- Users had a strong
preference for browsing rather than searching, even when given tasks
that directed them to search rather than browse.
|