IS213 Projects Home

Assignment 1

• Problem Statement
• Project Members

Assignment 2

• Personas
• Task Analysis
• Appendices
• Work Distribution

Assignment 3

• Scenarios
• Comparative Analyses
• Initial Design 1
• Initial Design 2
• Initial Design 3
• Work Distribution

Assignment 4

• Introduction
• Low-Fi Prototype
• Methods
• Test Measures
• Results
• Discussion
• Appendices
• Work Distribution

Assignment 5

• Design Revisions
• Revised Scenarios
• 1st Hi-Fi Prototype
• Storyboards
• Instructions for Use
• Slide Presentation
• Work Distribution

Assignment 6

• Heuristic Evaluation
• Work Distribution

Assignment 7

• 2nd Hi-Fi Prototype
• Responses to HEs
• Work Distribution
• Instructions for Use

Assignment 8

• Introduction
• Method
• Test Measures
• Results
• Discussion
• Appendices
• Work Distribution
• Slide Presentation

Final Write-up

• Problem Statement
• Solution Overview
• Final Personas
• Final Scenarios
• Revised Storyboards
• Final Interface Design
• Design Evolution
• Interactive Prototype
• Instructions for Use
• Work Distribution
• Slide Presentation

Results

Over all, we were pleased with the main characteristics of the prototype. All users expressed their approval of the basic functionality and flow. The fellow/member system was also received with enthusiasm. (Please see the full transcript of the user sessions).

However, several usability problems were identified:

1. The "Prefill drop-down menu" caused confusion for all three users.
   Severity: 4
   1. All three users did not find it intuitive and could not see what its functionality was;
   2. User A used it erroneously and changed all of the COG, KEGG & EC numbers when he only wanted to change COG;
   3. User B opened the drop-down menu but left it at "previous annotation" setting and filled in the data manually. He felt that the drop-down menu didn't give him enough information about homologues;
   4. User B also noted in de-briefing that it might be a "potentially dangerous" in annotating batches and that there needs to be a mechanism built in for ensuring proper checks and balances for propagating corrections if batch annotation is supported.
2. Users expressed disagreement with an annotation via both discussion and voting (clicking on "I Disagree" button).
   Severity: 2
   1. User A & User B clicked on "I Disagree."
   2. User B also left a comment in Discussion supporting his disagreement. He also felt that he would not click on "I Disagree" if the majority of the annotation were correct. In that case, he would rather use discussion tools or, most likely, leave it untouched if he didn't have a vested interest in the gene.
   3. User C clicked on the "Discuss" button from the Gene Cart page.
3. Discussion for sets of genes with batch comments is not facilitated from a single gene's annotation history page.
   Severity: 2
   1. User B noted that from a single gene's annotation history page he would want to have a way of making a batch comment to all of the genes that were part of a batch discussion and would want to see if the comment on the discussion thread was left for a batch of genes and what they were.
4. "Save Draft" page caused confusion for all three users by not having "OK" or "Accept Annotation" button.
   Severity: 2
   1. User C noted that he would want to click "OK" button and have the page disappear;
   2. Users A & B also looked for an "OK" button.
5. The logistics of allowing users to save drafts need to be addressed to avoid locking out other users or overwriting other users' annotations.
   Severity: 3
   1. User C noted that saving a draft could create problems if another user wants to annotate the same gene.

When asked about the usefulness of the annotation metadata shown on the Gene Details page, all users expressed their approval at seeing the name of the last annotator. User A noted that he would give more credibility to the annotator's name than to the "batting average." Unfortunately, we unintentionally left out the "x out of y voters agreed with this annotation" statement on the Gene Details page and only realized it after the testing, too late for comment. We did ask users whether they thought they would be equally likely to vote "I Agree" or "I Disagree". User A thought he would be more likely to click "I Disagree," User B thought he would be equally likely to use either button, and User C thought he might be slightly more likely to vote "I Agree." User B said that he would take agree/disagree votes with a grain of salt.

When asked what additional functionality the users would want to see, the users suggested the following:

1. The Preview page could be served automatically to all new users submitting annotations, but users could be given a control to turn off previews as they become more familiar with the system and feel more confident when submitting annotations.
2. A way to view the values in the annotation being modified next to the values for another gene from which they wish to copy (2 users).
3. More substantial evidence in the annotations and the page to update them. (User B suggested PFAM and UNIPROT data. He also wanted support for hyperlinks in the discussion section.)
4. Flagging genes whose annotations are under active discussion, perhaps in gene search results.
5. Batch annotations. If batch annotation is supported, additional checks will need to be implemented. Two users believed that they would rarely use it.