Introduction | Group Heuristic Evaluation | Conclusion
Individual Evaluations | Geneboree's Evaluation of LightsOn


Our group had the opportunity to perform a heuristic evaluation on Geneboree's first interactive prototype. The following is a collaborated list of several potential user interface problems we found. We would like to thank the members of Geneboree for taking the time to provide us with a short introduction to genomics. Geneboree's project promises to be extremely useful for future genetic research collaboration. We hope this heuristic evaluation will help Geneboree create improved future versions of their tool.


Group Heuristic Evaluation

1. [H1 Visibility of system status] (Severity 1, Found By: 2)
On the Gene Details page the "Loading" text in the upper right-hand corner of the page seems slightly redundant with the browser's own loading feedback. Not really a problem, but it might be nice to have the word "Loaded" removed after the page is loaded. Users who missed the beginning loading, might wonder what has actually been loaded. "Loaded" doesn't convey much information. Too informal?

2. [H2 Match between system and the real world] (Severity 2, Found By: 2)
On the Gene Details page, the text "V.1.1 and V1.0" exists. It is unclear to me what the linked V1.0 does, or why the V1.1 is not linked. (the extra white space before V1.0 is also linked) Maybe to a targeted user, this link is clearer. It is not clear which one is active.

3. [H4 Consistency & Standards] (Severity 2, Found By: 2)
Under "Discussion" the user name is linked. I expected it to bring me to a page that had details about the user, much as clicking on a gene brought me more details about the gene. I was not expecting it to be a mailto link. Maybe if there is no info about that user, a small "envelope icon" might make the functionality more clear. For many users, mailto links cause work flow disruptions when a new application is launched.

4. [H2 Match between system and the real world] (Severity 2, Found By: 3)
On the Gene Details page, under "discussion," there is a "More" link. I expected this link to show me more of the text of the message shown. It instead drops me to a full Annotation History page and I have to scan the page to find the quote that is begun on the Details page. Oddly, I can't even find the quote shown. Perhaps it is just hard-coded for this prototype. There should be a value to indicate how many "more" comments to view. Users will know if the annotation is "hot."

5. [H4 Consistency & Standards] (Severity 2, Found By: 2)
The Gene Details page has two buttons for submitting opinion: "I Agree", and "I Disagree". They are both red, which is troubling since, to me, that seems to indicate they are both negative feedback. Maybe "I Agree" should be green.

6. [H1 Visibility of system status] (Severity 3, Found By: 4)
When clicked, both the "I Agree" and "I Disagree" buttons stay blue (highlighted) even after the results have been "saved." They should revert back to normal state to avoid giving the impression the system has failed to complete the save. I would suggest instead you "gray out" the button that was clicked, to indicate you can't agree (or disagree) twice. Maybe a set of 3 radio buttons, with one being "No Opinion." Votes get submitted as soon as the user changes the vote off of the default "No Opinion."

7. [H4 Consistency & Standards] (Severity 2, Found By: 2)
"Update Annotation" and "Discuss" buttons have slightly different windowing behavior. It looks like "Update" brings up a new window, while "Discuss" redirects the current one. I would suggest making them consistent, or at the very least, change the frame and look of the new window. Make it clear that it is a "dialog" rather than the new window of interaction. I tend to find myself using the new window to finish my work flow and forget about the parent window from which spawned. Generally windowing is behavior inconsistent.

8. [H2 Match between system and the real world] (Severity 1, Found By: 1)
I find the phrase "0 of 1 voters agreed with this annotation" cumbersome. Maybe it would better be served with "Agreed: 10, Disagreed: 30, Total: 40" or something like that.

9. [H4 Consistency & Standards] (Severity 1, Found By: 1)
On the "Annotation" page, the use of colons is inconsistent. The first set of questions has none, but the "Evidence" set mostly uses them. I suggest you use them for all fields, even checkboxes, as ungrammatical as that seems.

10. [H4 Consistency & Standards] (Severity 0, Found By: 1)
On the Annotation page, change "Comment (will be..." to "Comments (will be..." as most users think of comments in the plural.

11. [H4 Consistency & Standards] (Severity 1, Found By: 1)
At the end of the Annotation page, two checkboxes exist. They break the pattern of label first and then input field, as established earlier on the page. On the second field, "Notify me via Email of discussion and changes," capitalization should be kept consistent with the label above it, either title-case it, or convert both to normal case style. Example: "Notify Me Via Email of Discussions and Changes."

12. [H3 User control and freedom] (Severity 3, Found By: 1)
From the Gene Cart, if you choose to "Discuss Selected" one is brought to a discussion page. On this page, there is a cancel button which does nothing. There should be a clear way to get back to the Gene Cart if the user decides not to discuss those genes.

13. [H2 Match between system and the real world] (Severity 3, Found By: 2)
Discuss Selected Genes and Gene Cart page lists a set of "Gene Object IDs." This might make more sense to a target user, but I am guessing this is an example of the code peeking through into the UI. If the ID is needed, maybe the Discuss page should list the descriptions as well. Otherwise the user must remember the IDs and be confident they are the ones they wish to comment on.

14. [H3 User control and freedom] (Severity 4, Found By: 3)
"Discuss Selected Genes" allows you to submit your comment, but after you do so, you are brought to a page that only lets you "close the window." This button didn't actually work, but nor did I want it to as this was the only window I had open. It does work if you get to that page through the Update Discussion page, at which point it really does close your browser window! The user's only escape is the browser back button.

15. [H4 Consistency & Standards] (Severity 1, Found By: 2)
The Annotation page refers to the free form text areas as "Comments," where as elsewhere they are noted as "Discussions." If these are the same thing, they probably should be consistently referred to as one or the other.

16. [H8 Aesthetic and minimalist design] (Severity 2, Found By: 3)
The "Discuss This Annotation" page, and elsewhere, includes a full time stamp of when the annotation was made. It is probably gratuitous to include the time full information. At least the "seconds" are unlikely to be useful and it clutters the display. I would suggest visually separating the timestamp, author, and comments a little more. A slight use of color or a gestalt grouping might be enough to make this page easier to read. Long comments could quickly become a long jumble of text otherwise. I am not sure what the "-0800" code is for, but it is unlikely to mean much to the user.

17. [H8 Aesthetic and minimalist design] (Severity 1, Found By: 1)
On the list of current annotations, the green flag denoting that an annotation has changed in the last seven days should be placed next to the checkbox instead of at the end of the line. This will yield a cleaner, less cluttered look as well as reducing the amount of time a user spends on scanning the line for information.

18. [H2 Match between system and the real world] (Severity 4, Found By: 2)
It is not clear that the Homolog Selection dropbox is functionally related to everything below it (including the 'Add Selections to Gene Cart', etc. buttons). Make it more obvious that everything including and below the Homolog Selection is in one group.

19. [H1 Visibility of system status] (Severity 1, Found By: 1)
Buttons under the 'Functional Annotation' column are too close together and cluttered. It may be easy for the user to accidentally click on the the wrong button (e.g. Clicking on Agree when the intended action is to click on Update). Additionally, the use of a red highlight on the Agree button may be a source of ambiguity.

20. [H5 Error prevention] (Severity 1, Found By: 2)
The separate sections (Export Genes, Sequence Alignments, etc.) should be "Gestalted" together better. Especially since the buttons resemble tabs in many ways, users may erroneously interpret the buttons as the headers of each section.

21. [H1 Visibility of system status] (Severity 3, Found By: 2)
The layout of each page makes it very hard to understand. It seems like there's no order, or, if there is an order, no distinction between sections. The buttons and the headings seem to act as section delimiters, but it's really hard to perceive the different groups. It seems like this would be easy to fix, adding horizontal lines or distinguishing the section headings.

22. [H2 Match between system and the real world], [H4 Consistency and standards ] (Severity 2, Found By: 2)
Batting Average is confusing, seems unrelated to gene annotation and is plausibly unprofessional. Additionally (agrees/votes) is in ratio while 0.788 is in decimal format. Most websites today use visual scales or rating systems to indicate approval.

23. [H8 Aesthetic and minimalist design] (Severity 0, found By: 2)
"Evidence" table row is highlighted at the bottom of the first table on the functional decomposition table.

24. [H1 Visibility of system status] (Severity 3, Found By: 3)
In general, it was difficult to perceive location within the greater website; we were often lost and did not know how to return to previous pages without using the BACK button.

25. [H1 Visibility of system status] (Severity 1, Found By: 1)
There was an error message in the "Ortholog Clusters" section. Since this section is below the initial view of the screen and possibly not in the control of the current designers they should at least check for errors and make a notation on the top of the page in plain view of the user.

26. [H8 Aesthetic and minimalist design] (Severity 1, Found By: 1)
The "your log in information is not valid" message is poorly aligned under the menu options; it is only in partial view.

27. [H4 Consistency & Standards] (Severity 3, Found By: 1)
The buttons in the interface do not appear clickable. There is no mouse over event or change in mouse cursor to indicate clickability. This is especially true on the Gene Cart page where the blue or red outline does not suffice to show button or link behavior exists.

28. [H5 Error prevention] (Severity 1, Found By: 1)
You can select all the genes and still click Select All. Instead, the button should become disabled if the action has already been completed and become activated again when it makes sense.

29. [H4 Consistency & Standards] (Severity 1, Found By: 1)
The wording "Add to My Watched Annotations" is not consistent with "MyAnnotations."

30. [H2 Match between system and the real world] (Severity 3, Found By: 1)
It seems that the annotation rating is very important. Therefore, I think that the score (batting average or whatever scale is used) should be shown prominently at the top of the page. Currently, it is very small and hard to pick out. Also, it might be helpful to show it when the genes are listed in a table.

31. [H4 Consistency & Standards] (Severity 1, Found By: 1)
There is a combination of serif and sans-serif fonts on the page. These should be made consistent as the difference does not seem to serve any real purpose.


Severity Rating Totals

Severity Rating Number Found
0. Not a Usability Problem 2
1. Cosmetic Problem 13
2. Minor Usability Problem 7
3. Major Usability Problem 7
4. Usability Catastrophe 3


Heuristic Totals

Heuristic Considered Number Found
H1 Visibility of system status 1
H2 Match between system and the real world 8
H3 User control and freedom 3
H4 Consistency & Standards 12
H5 Error prevention 3
H6 Recognition rather than recall 0
H7 Flexibility and efficiency of use 0
H8 Aesthetic and minimalist design 4
H9 Help users recognize, diagnose, and recover from errors 1
H10 Help and documentation 0



The majority of problems we found while exploring the interface were determined to have a severity of 2 or less. For the most part, we all agreed that the changes needed were small and there was very little that would actually prevent a user from completing her task.

Of the more severe problems, the majority were related to issues of window behavior and navigation. We felt in several instances that the use of new windows and the lack of clear exits made for a slightly frustrating user experience. We believe that many of these problems would be solved with consistent link behavior and a clearer representation of navigation. It was suggested that even a simple breadcrumb trail would help a user find her way back after many of the annotation tasks.

As a whole, we felt the button look and behavior was not adequate. While it is important to maintain a consistency within the site outside of Geneboree's control (IMG), we believe it is worth breaking this consistency to provide more intuitive, standard buttons. We felt the buttons should look more "clickable," change as you mouse over and be more visually distinct from tabs or other textual elements of the page. In addition, there were circumstances where buttons would be better replaced by other form fields such as radio buttons (e.g. the Agree and Disagree buttons).

Most of the problems we encountered had to do with the use or misuse of consistency and standards. Many of these problems were minor, but we felt adhering to web standards and naming standards established within the prototype -- such as "discussions" vs. "comments" -- would greatly help the overall user experience.

We also felt there was an occasional disconnect between the system and the real world. It is possible that this gulf was made wider by our lack of domain explicit knowledge. In addition, the voting system looked a little clunky. We felt a few slight changes to the wording and the way results were displayed would greatly enhance that functionality.

Many of the minor problems we found fell under the aesthetic and design heuristic. These issues were mostly included for completeness sake. The misuse of colons or mismatching font faces is unlikely to sink an interface. We also realized that some of these problems might lie beyond the scope of this project. Often we were not sure if a design decision was made by Geneboree or by the original creators of the IMG website.

We hope these suggestions prove helpful.


Individual Evaluations

John Mark

Geneboree's Evaluation of LightsOn

HE of LightsOn