Heuristic Evaluation of Web UI Design Patterns on First Interactive Prototype

Evaluators:

Tofer Merritt, Minakshi Mukherjee, Lois Wei

Heuristic Table:

Heuristic number Short description
H1 Visibility of system status
H2 Match between system and the real world
H3 User control and freedom
H4 Consistency and Standards
H5 Error Prevention
H6 Recognition rather than recall
H7 Flexibility and efficiency of use
H8 Aesthetics and minimalist design
H9 Help users recognize, diagnose and recover from errors
H10 Help and documentation

Total number of violations:

21 violations (but 3 of them violated two heuristics)

Broken down by heuristic violated:

H1 Visibility of system status 1
H2 Match between the system and the real world 3
H3 User control and freedom 1
H4 Consistency and standards 5
H5 Error prevention 1
H6 Recognition rather than recall 2
H7 Flexibility and efficiency of use 1
H8 Aesthetic and minimalist design 5
H9 Help users recognized, diagnose, and recover from errors 0
H10 Help and documentation 2


Broken down by severity of violation:

0 Don't think this is a usability problem 1
1 Cosmetic problem 1
2 Minor usability problem 8
3 Major usability problem 11
4 Usability catastrophe 0


List of Heuristic Violations:

Note: The number of evaluators that identified each violation is shown in the parenthesis at the end of each heuristic violation.
  1. [H2 Match Between System and the Real World] (Severity 3)
  2. While adding a comment with the HTML source code for the Google navigation tab for the Navigation Tabs pattern, the Comments and Code section shows the end result of how the content would look like if the source code is interpreted by the browser. In this case, it shows the actual Google navigation tab. This does not seem to be very intuitive. However, it seems to us that it is your intention to display the pattern source code to the user, but it is the current technical implementation that is hindering the display of the actual source code when the user enters it into the form. (1)

  3. [H2 Match between System and the Real World] (Severity 3)
  4. It seems there should be some kind of separator between the different comments. It was confusing to us while scrolling through the comments to remember that the contributor and the date published corresponds to the comment below it. (1)

  5. [H3 User Control and Freedom] (Severity 3)
  6. Under the Add a Comment section, it would be better to have a reset or undo button if the user wants to cancel a half completed form. (1)

  7. [H7 Flexibility and efficiency of Use] (Severity 2)
  8. For the most recent comment section under the About This Pattern section of the home page, it would be better if the user can click on the preview of the recent comment to view the complete comment. (1)

  9. [H4 Consistency and Standards] (Severity 3)
  10. The headers for user comment and pattern code needs to be consistent. Currently, it is written in two different ways: User Comments & Code on the right comment overview section, but on the main page, the header is Comments and Code. (2)

  11. [H4 Consistency and Standards] (Severity 3)
  12. It would be better if the link names and their ordering on the About This Pattern section can match up with the pattern subheading titles on the main pattern info page. For now, it seems solution and rationale corresponds to the subheadings on Solution and Used When. (1)

  13. [H4 Consistency and Standards] (Severity 0)
  14. Under the Add a Comment section, since all the input fields label ends with colon ":" except Your Comment, it seems to make more sense for this label to end with colon as well. (1)

  15. [H5 Feedback] (Severity 3)
  16. When the user clicks on a design pattern on the navigation bar, the link itself does not provide any visual cue that the page associated to it is been displayed in the main content section. (3)

  17. [H8 Good Error Messages] (Severity 3)
  18. After adding a comment that contains an apostrophe in the content, and the message returned is You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near I'm Feeling Lucky" name=btnI>&n' at line 1. This error message is too technical for the users who do not have programming background and know MYSQL. If improving the error message is something you are in the process of doing, then please dismiss this comment.(1)

  19. [H8 Aesthetic and minimalist design] (Severity 1)
  20. Perhaps it would be clearer if under the Add Comments section, the checkbox and the text Contains Code? are on the same line.(1)

  21. [H2 Match between system and the real world] (Severity 2)
  22. The Contains Code? checkbox in the Comments section indicates that there is something more to the comment if there is code involved. This feature may be confusing to the user, since he/she may expect something to happen to the comment box when he/she checks the contains code checkbox. Maybe an alternative solution would be to have a separate input box for inserting the code. (1)

  23. [H1 Visibility of system status] (Severity 3)
  24. Having the next and previous button in addition to the breadcrumb navigation may be useful. Although the breadcrumbs may provide an easy replacement of the previous or back button, a user may still need to depend on their memory in order to traverse forward and back without explicit buttons to do so. (1)

  25. [H4 Consistency and Standards] (Severity 2)
  26. The wording of definitions for Breadcrumbs and Navigation pattern on the home page is different from the wording of the definitions shown on the mouse-over pop-ups. Consistent definition in both the places would be useful. (1)

  27. [H8 Aesthetic and Minimalist Design] (Severity 2)
  28. The location and wording of the Common Problems that Patterns Can Solve section seems to be a little misleading. The title indicates that it is general help, but the wording (e.g. Your navigation is complicated ….) seems to indicate that it is specific help with the current UI problem. Also, the font color is red and the location of the information is in the center of the page, making it stand out as further indication that the user is faced with the current problems of the UI. (1)

  29. [H8 Aesthetic and Minimalist Design] (Severity 2)
  30. The observation for #14 seems to apply to the Example Patterns as well. Is the user getting general help, or personalized help for the current problem? Making these examples front-and-center seems to indicate that the user is getting personalized help. (1)

  31. [H6 Recognition Rather than Recall] (Severity 2)
  32. The headers on Find a Pattern, Example Patterns, and Recent Comments needs to stand out more. With the current font color, the users may have trouble immediately spotting them. For example, in the Find a Pattern section, Forms, Info Organization, etc. are all in a red font and stand out more than the title of the section. (1)

  33. [H10 Help and documentation] (Severity 2)
  34. There is nothing that verbally says Help, but users may have to make assumptions that Developer Resources and About Patterns means Help. (1)

  35. [H4 Consistency and Standards] (severity 3)
  36. Different snapshots for the examples of the same pattern can be confusing. For example, the snapshot of navigation tab pattern in the home page is different from the one in the navigation patterns page. (1)

  37. [H6 Recognition rather than recall] (Severity 2)
  38. On the About This Pattern section, it might be better to replace the header with, About the Breadcrumbs Pattern. (1)

  39. [H8 Aesthetics and minimalist design] (Severity 3)
  40. The significance of the arrows E,I and A,D,H pointing to the solutions section in the breadcrumbs page is not clear.

  41. [H10 Help and documentation] (Severity 3)
  42. It would be better to provide a hyperlink to a help section to explain terms like double tabular and nested tabular. On its own, the meanings of the terms can be difficult to understand for novice users. Thus a help section for the terms may be required.


Overall Recommendations:

Our project group finds UI Design Patterns to be a great and useful application. We feel that the concept is very unique, and we envision the tool to be very helpful for short-staffed and small-budgeted development companies and organizations.

Overall, we really liked the layout and the structure of UI Design Patterns. We think you did a good job in organizing and creating a browsing interface for the web design patterns. We liked your idea of showing the sample design pattern snapshots on the home page as well as through the mouse over pop-ups. We feel that the key areas for improvement are the site navigation, consistency of terminology, good error feedback, and help documentation. It is also important to ensure that the contributed source code for web design patterns can be added and displayed properly. We have described the details of the other suggestions in the heuristic evaluation.

Individual Heuristic Evaluations: