Assignment #5: Heuristic Evaluation
Assigned: March 17th, 2009
Due: March 30th, 2009
Objective: In this assignment, you will perform a heuristic evaluation, and integrate your results with other evaluators to generate an evaluation report. Moreover, you will act as a design consultant for another group's final project, by informing their design based on your evaluation.
What to do:
1) Each person in the class should choose one other group's project to evaluate. Additionally, each group should find at least three other people in the class to evaluate their project. Use the first 10 minutes of class to decide these pairings. The recommended approach is to form evaluation groups of three members each.
2) Spend approximately 30 minutes conducting a heuristic evaluation of another group's interactive prototype. You are encouraged to use Nielsen's heuristics (see lecture 7), or any of the other design heuristics we have learned in class (see lectures 2, 5, and 9 for details). These evaluations should be done independently of one another, making sure to take detailed notes of your observations. You will have the rest of class on March 17th to perform these evaluations. Any evaluations that cannot be completed during class time must be done after class.
3) At all times, at least one (and preferably two or more) members of each group should be demonstrating their interface, while the rest are evaluating another group's prototype. The demonstrators should begin by outlining a task or scenario demonstrating the use of their system. After that, the evaluators are free to ask questions. When the evaluator is finished assessing one scenario, the demonstrators should restart the conversation by demonstrating the next task or scenario.
4) Use the following format to keep track of your observations: HE.xls. Each observation requires a numeric index, a heuristic that was violated, a location on the user interface, a description of the problem, a severity rating, a fixability rating and a possible fix (use the 0-4 scale presented in lecture for severity and fixability ratings). Each evaluator should aim to document a minimum of 10 usability problems, covering 5 distinct heuristics. Some usability problems may not violate an established heuristic - in that case you can label them "misc" for miscellaneous. However, please make sure that an existing heuristic does not cover what you are describing. "misc" should only be used rarely.
5) For each problem, you should suggest a possible fix. This is not a technical description, but a simple recommendation of how to fix the problem. Try to be concise - for example, for a button that needs to be changed, simply mentioning that "Button 'X' should be renamed 'Y'" is sufficient.
6) The next step is to combine your report with the other students who evaluated the same prototype. The three of you should plan a time to meet to consolidate your individual evaluation reports. The group whose project you evaluated need not be present, but you will probably find it helpful to have a version of their prototype available for reference. You will create one master Excel spreadsheet that contains each unique problem found. Remember - a duplicate is a violation of the same heuristic, in the same location. A violation of a different heuristic in the same location is considered distinct. For each unique problem found, you will need to discuss amongst yourselves to decide on a final severity rating, fixability rating and possible fix. Before turning in your final list, please sort by the sum of the severity and the fixability, to prioritize the most severe and most fixable problems.
7) In collaboration with the other student evaluators, write a short (less then one page) executive summary that outlines the major problems that you found, and possible solutions for the same, prioritizing those with the highest severity + fixability ratings.
What to turn in:
The preferred method of turn-in is to create a page for your evaluation on the course Wiki. Create a dedicated page for your evaluation, including each of the following components:
1) Cover sheet including yourself and the other evaluators' names, and the project you evaluated. Note the time, duration and attendance of each evaluation session, including the final consolidation session [1/2 page].
2) The full set of heuristics you used (including any of Nielsen, Norman or general graphic design heuristics).
3) Each of your individual evaluation reports, preferably in the following Excel format: HE.xls.
4) The final consolidated evaluation report, once again using the same format.
5) A short executive summary that outlines the major problems you found, and possible solutions for the same, prioritizing those with the highest severity + fixability ratings [1 page].
6) Reflect on the process of conducting a heuristic evaluation. What was easy? What was hard? What did you learn? Did you feel that the evaluation you conducted was useful for the other group's project? Did you feel that the evaluation you received was useful for your own group's project? [1/2 page]
E-mail a link to your group evaluation's Wiki page to the professor and TA, cc'ing the team members of the project you evaluated, before 11:59 PM on Monday, March 30th. Please note that this is a *hard deadline*, as the other group will have to make decisions based on your assessment. For this reason, late submissions will not receive any credit for this assignment.
Please contact the professor or the class TA using the course mailing list if you have any questions with this assignment.