Assignment 6: Heuristic Evalution
Introduction | Group Heuristic Evaluation | Conclusion | Individual Heuristic Evaluations
Work Breakdown | FAST's Evaluation of SkillShop
The SkillShop team performed a heuristic evaluation on FAST's (Faculty Advancement Support Technology) first interactive prototype. Each team member first did an individual evalution on the system appling Nielsen's heuristics. Then the group compiled all the usabilty violations to eliminate duplicates. FAST's system to replace the paper Annual Supplement to the Bio-Bibliography report with an electronic form will definitely reduce the effort and cost of producing this report and would be a major service to the faculty.
1. [H1 Visibility of system status] ( Severity 3, Found By: 3 )
"Create New Bib" Page: Tabs do not give an indication of what has been completed on the bib. Some device is needed to display overview to user.
2. [H2 Match between system and real world] ( Severity 2, Found By: 2 )
"Login" Page: "Sponsors" link is ambiguous.
3. [H2 Match between system and real world] ( Severity 2, Found By: 2 )
"Create New Bib" Page: Tab names are not sufficiently descriptive.
4. [H2 Match between system and real world] ( Severity 2, Found By: 2 )
On the login page, there is a note under the login box. The statement is confusing as to which faculty members need to submit a Bio-Bibliography. A link to the “Section 32-0 Academic Personnel Manual” indicated in the note should be provided.
5. [H3 User Control and Freedom] ( Severity 3, Found By: 3 )
After submitting an entry, it is not obvious how one would return to the starting page to submit another entry. Instead of having users click on the publication year (which seems unintuitive), perhaps it would be helpful to add a "back to start" or "add another entry" button, or maybe even automatically go to that page once a user is finished adding an entry.
6. [H3 User control and freedom] ( Severity 3, Found By: 2 )
In the publications tab, there are three types but publications that are student-edited are also Non peer-reviewed. Groups are not mutually exclusive.
7. [H3 User control and freedom] ( Severity 3, Found By: 1 )
There is no exit or logout option while submitting an entry.
8. [H3 User control and freedom] ( Severity 3, Found By: 3 )
"Create New Bib" Page: There should be a clearly visible option available from all tabs to save the bib in progress and return to continue editing it later.
9. [H3 User control and freedom] ( Severity 2, Found By: 2 )
The journal article entry form needs to have an “other” option for citation style because the list is not comprehensive.
10. [H4 Consistency and standards] ( Severity 3, Found By: 1 )
On the login page instead of “Enter name” it should be “Employee ID,” The message above the login box is not consistent with the login text box.
11. [H4 Consistency and standards] ( Severity 2, Found By: 1 )
The initial page that a user sees after login doesn't include the navigational bar or tabs that one is exposed to on subsequent pages It would be more consistent to have that page presented in the same format as the other pages.
12. [H4 Consistency and standards] ( Severity 2, Found By: 2 )
"Welcome" Page: Heading says "Instructions" but contains options that the user can pursue, as opposed to actual directions or help.
13. [H4 Consistency and standards] ( Severity 1, Found By: 2 )
The “Authenticate” button should be under the two login text boxes.
14. [H5 Error prevention] ( Severity 2, Found By: 1 )
Having an open citation field will lead to users entering a variety of different formats and the citation style the user selects may not be consistent with what the user cuts and pastes into the field.
15. [H5 Error prevention] ( Severity 2, Found By: 1 )
There should be a message that indicates that the passphrase is case sensitive so the user can make sure to use correct capitalization.
16. [H5 Error prevention] ( Severity 2, Found By: 1 )
When the user chooses to delete an entry, there should be a confirmation message asking the user if they wish to continue with the operation before actually deleting the entry.
17. [H5 Error prevention] ( Severity 2, Found By: 2 )
On the main page after login, it may be a good idea to include automation error prevention techniques such as making sure start dates and end dates for academic years are valid (prevent entries like 2004-2004 or 2005-2004).
18. [H6 Recognition rather than recall] ( Severity 2, Found By: 2 )
On the pages that include the navigation bar and tabs, the "Drafts" section in the top left was confusing. It is not clear what "Drafts" really means or what clicking on the academic year range that appears beneath it will do. This section does not appear to be fully implemented yet, but it may be helpful to examine word choice here.
19. [H6 Recognition rather than recall] ( Severity 2, Found By: 2 )
"Create New Bib" Page: Under "Resources", the two links should be identified by more than a section number, but also include some text indication of what each refers to, perhaps the section heading.
20. [H8 Aesthetic and Minimalist Design] ( Severity 2, Found By: 2 )
The publication selection dropdown seems excessively long and has two levels of information for users to process. A good idea to minimize that load would be to have two dropdowns, one for the higher level items (Article, Book, etc.) and another drop down that would populate with the lower level choices once a selection is made in the previous dropdown.
21. [H8 Aesthetic and Minimalist Design] ( Severity 1, Found By: 1 )
"Login" Page: "Please identify yourself..." should be better aligned over the rest of the content on the page.
22. [H9 Help users recognize, diagnose and recover from errors] ( Severity 2, Found By: 2 )
Although the preview feature is a good way for users to double check for errors before submission, it may also be helpful to indicate to users which fields are required and which are optional and to enforce those requirements. This could help users who want to go through the process quickly to focus on the minimum necessities for a valid submission.
23. [H10 Help and documentation] ( Severity 3, Found By: 2 )
"Login" Page: An option is needed for those users who have forgotten their user ID or passphrase.
24. [H10 Help and documentation] ( Severity 2, Found By: 1 )
The homepage doesn't have any kind of orientation for a first time user.
Severity Rating Totals
Severity Ratings |
Violations |
0 Not a usability problem |
0 |
1 Cosmetic problem only |
2 |
2 Minor usability problem |
15 |
3 Major usability problem |
7 |
4 Usability catastrophe |
0 |
Heuristic Totals
Criteria |
Violations |
H1 Visibility of System Status |
2 |
H2 Match between System and the Real World |
3 |
H3 User Control and Freedom |
5 |
H4 Consistency and Standards |
4 |
H5 Error Prevention |
4 |
H6 Recognition Rather than Recall |
2 |
H7 Flexibility and Efficiency of Use |
0 |
H8 Aesthetic and Minimalist Design |
2 |
H9 Help Users Recognize, Diagnose, and Recover from Errors |
1 |
H10 Help and Documentation |
1 |
TOTAL |
24 |
The majority of problems our group found while exploring the Bio-Bibliography interface were of severity of 2. This implies that most of the problems which needed to be addressed were minor usability problems.
Of the more severe problems, the majority were related to issues of user control and freedom. Little things like improved navigation between the parts of the form and the added ability to save a form and return to it at a later time before submission would help to enhance the user experience and avoid many potential cases of frustration with the system.
Another area that we felt that needs to be addressed is the overall lack of system feedback. The importance of keeping the user informed about system status should not be underestimated. For example, in the current implementation, the user has no way of finding out which parts of the form has been already completed. We think that keeping the user informed about these details will avoid having to unnecessarily hop back between parts of the form, saving the users much time and energy, and leading to a more efficient user experience.
We also thought that it would be helpful to have features that cater to different kinds of users using the system. By reviewing the personas you have created, it appeared that some people may use the system as a formality and would like to have quick shortcuts, whereas others, still climbing the professorship ranks, will spend much more time an effort using the system. It would be useful to have a few more features to accommodate the former set of users so that they are able to complete the process in the minimal amount of time. We also found that some of the options provided (in a dropdown for example) may not be comprehensive, and the user should be able to choose an “other”option. We think these small but important changes will add much flexibility to the system, enhancing its usability.
As a final note, we found that the area of “help and documentation” is another that needs to be addressed. Users will most likely need instructions or guidelines when filling out the forms. Also an ‘about us’ section may be a good way to explain who you are and what you are trying to accomplish with the new system. While this last area may be one that you were planning to implement in the future, we thought it important to emphasize the need for “help and documentation” sections in order to prevent many potential errors and avoid user confusion and frustration.
All in all we believe that you have done a good job with the design and we hope that you will find these comments constructive and useful.
|
Saud Al Shamsi |
Debbie Cheng |
Alana Pechon |
Bindiya Jadhwani |
| Individual Evaluations | 25% |
25% |
25% |
25% |
| Group Evaluation | 33% |
1% |
33% |
33% |
| Conclusion | 50% |
0% |
0% |
50% |
| Website | 0% |
25% |
75% |
0% |
FAST's Heuristic Evaluation on SkillShop