IS213 Projects Home

Assignment 1

• Problem Statement
• Project Members

Assignment 2

• Personas
• Task Analysis
• Appendices
• Work Distribution

Assignment 3

• Scenarios
• Comparative Analyses
• Initial Design 1
• Initial Design 2
• Initial Design 3
• Work Distribution

Assignment 4

• Introduction
• Low-Fi Prototype
• Methods
• Test Measures
• Results
• Discussion
• Appendices
• Work Distribution

Assignment 5

• Design Revisions
• Revised Scenarios
• 1st Hi-Fi Prototype
• Storyboards
• Instructions for Use
• Slide Presentation
• Work Distribution

Assignment 6

• Heuristic Evaluation
• Work Distribution

Assignment 7

• 2nd Hi-Fi Prototype
• Responses to HEs
• Work Distribution
• Instructions for Use

Assignment 8

• Introduction
• Method
• Test Measures
• Results
• Discussion
• Appendices
• Work Distribution
• Slide Presentation

Final Write-up

• Problem Statement
• Solution Overview
• Final Personas
• Final Scenarios
• Revised Storyboards
• Final Interface Design
• Design Evolution
• Interactive Prototype
• Instructions for Use
• Work Distribution
• Slide Presentation

Discussion

What We Learned

We found that users like the idea of a collaborative annotation system very much. They also like the idea of having a two-tiered membership structure (members and fellows). They felt the proposed criteria for becoming a fellow were reasonable.

Users differed in their opinion about allowing transfer of annotation information from one gene to another. While intuitively it seems like a good idea to save users the hassle of copying data from one gene into another, in practice the prefill drop-down menu confused all of our users. They also felt that careful consideration should be given to each field in a new annotation, instead of making the change process too easy. As per user suggestions, we currently plan to remove the drop-down prefill menu. Other options for making it easy to copy data from other genes' annotations are under discussion. One possibility is the use of a menu that allows the user to select a similar gene and then reveals annotation information for the selected gene beside that for the one being annotated, so that users can refer to and copy data more easily.

Users took a variety of approaches in expressing opposition to a given annotation. They typically chose the route we identified as "the easiest" (click on "I Disagree"), but it was encouraging to see them make use of all the modes of feedback.

Users expressed a desire for replies to the discussion forums to extend to batches of genes, not just individual genes. This is a particularly wise idea since we already have a mode for commenting on batches of genes. In order to enable batch discussions, the Annotation History page will need to include information about genes with comments applied in batches, as well as an option to reply to discussions for the entire batch of genes. Users should be able to enter a reply for all the genes in the batch from anywhere they can currently reply for the single gene.

Users wanted an "Okay" button on the Saved Draft page. We will add this item, which will close the annotation popup, returning the user to the IMG page they were last viewing. The buttons linking to Gene Details and MyAnnotations will remain.

The additional functionality that users proposed will be incorporated where feasible. We plan to add fields for PFam and UniProt references with links to these databases.

Batch annotation is probably the most significant feature that was requested, to the extent that one user envisioned annotating entire branches of phylogenetic trees simultaneously. If we decide to support such a change, we will need to undertake significant design work to determine how to implement this feature without introducing the potential for multiple hard-to-correct errors. One solution would be to allow users to update annotations in series, simply opening the annotation update form for each one in a selected set as soon as the previous one is submitted. Our decision and further development will be discussed in the next phase of this project.

What We Were Not Able To Learn

A future test will need to address whether the different modes of providing feedback (Agree/Disagree, Discussion, and Modify Annotation) will be useful or confusing; this will require generating relevant content, for user behavior is dependent on the content in this case. We realized that scenario-based testing cannot answer the question of how readily people will use the "I Agree" and "I Disagree" buttons, since including them in a scenario skews usage toward using them and leaving them out skews it the other way. On a related note, users expressed subjective approval for the "batting average" rating of annotations, but we will need to witness more involved tasks in order to see whether such ranking information is actually useful.

Another aspect of this system that remains untested is the nature of the collaborative work that it enables. We suspect that there are unpredictable cases that will arise when users are in earnest dialogue with other users about real data. For instance, users may start engaging in furious discussion that will require something close to instant-messaging capability. Or users might save drafts and forget to submit them, thereby locking anyone else out from editing those annotations. We know we will need an edit lock for annotations being worked on, but we're not yet sure how long it should last before timing out.

Also, we decided not to directly test the Member/Fellow distinction in this iteration, since the "Fellow" aspect of this interface would only become available to users who already had a good familiarity with this system. Thus, something that might be confusing at first glance to a member would be intuitive to a potential fellow. We anticipate testing the "Fellow" initiation and functionality in the next design iteration, using members we tested in this iteration.