Assignment 6: Heuristic Evaluation of Play it by Ear interface
User Interface Design
- Assignment 1: Project Proposal
- Assignment 2: Project Personas, Goals, and Task Analysis
- Assignment 2 Appendix A: Interviews
- Assignment 3: Project Scenarios, Comparative Analysis, and Preliminary …
- Assignment 4: Low-fi Prototying & Usability Testing
- Assignment 5: First Interactive Prototype
- Assignment 6: Heuristic Evaluation of …
- Assignment 6: Play it by Ear's Heuristic Evaluation of iNaturalist
- Assignment 7: Second Interactive Prototype
- Assignment 8: Pilot Usability Study
- Assignment 9: Third Interactive Prototype
System Architecture / Document Engineering
I213 User Interface Design and Development
Prof. Marti Hearst
Nate Agrin
Jessica Kline
Andrew McDiarmid
Ken-ichi Ueda
Contents
Group Evaluation
(References in parentheses are to source positions in individual evaluations.)
Introduction
The following is the group heuristic evaluation for the Play it by Ear First Interactive Prototype. It covers the interfaces represented by both the interactive prototype and the representative storyboards found on the Play it by Ear website. The following usability violations described in this evaluation refer to the Ten Usability Heuristics outlined by Jakob Nielson. These violations are consolidated from our individual evaluations (they reference their original sources) and include general violations and violations specific to the different interfaces featured in the prototype. In this evaluation we also include our recommendations for future interface design and development and the individual evaluations compiled by each group member.
General Violations
1. [H4 Consistency and Standards/H2 Match between System and Real World] (Severity 4)
(Andrew 5; Ken-ichi 2, 5; Nate 1)
The center button should be "Select." (This may conflict with some phone conventions, but not the ones we know.)
2. [H8 Aesthetic and Minimalist Design] (Severity 4)
(Ken-ichi 7; Andrew 4; Jess 2)
Will text be readable on a phone? The mock interface is relatively huge and text is already small. The Comedy icon also won't scale well, and some color choices are hard to read.
3. [H4 Consistency and Standards] (Severity 3)
(Ken-ichi 3, 4; Andrew 9; Jess 11; Nate 2)
"Back" is not consistently mapped to the right soft key. "Map" appears on both the left and right soft keys in different interfaces.
4. [H10 Help and Documentation] (Severity 2)
(Ken-ichi 7; Nate 11; Jess 8)
Application as a whole could use more contextual and general help/instructions. The transparent "Press 0-9" in the storyboards is a good example of how this could work.
5. [H2 Match between the System and the Real World] (Severity 2)
(Jess 3, 5)
The use of the 24-hour clock could be confusing. Also, seconds may not be a useful level of detail in event times.
List View Violations
6. [H1 Visibility of System Status] (Severity 2)
(Nate 8; Ken-ichi 1; Jess 10)
List view should provide an indication as to how the list is currently sorted.
7. [H2 Match between System and Real World] (Severity 2)
(Jess 1)
The wording "Etc" has the potential to be confusing to users. The word "Etc" is generally used to describe the continuation of some concept. According to this interpretation, the "Etc" representation would imply that the respective category is a continuation of the previously listed categories. A more appropriate word choice might be "Misc."
8. [H7 Flexibility and Efficiency of Use] (Severity 2)
(Nate 6)
On the landing page, scrolling right or left is halted at the end of the list of categories. It should cycle back through to the beginning of the menu.
Options Menu Violations
9. [H2 Match between System and Real World/H1 Visibility of System Status] (Severity 2)
(Nate 3; Andrew 6a, 6b)
On the List Options view, "Select" and "Back" are confusing. Language does not communicate whether changes to options are being saved.
10. [H1 Visibility of System Status/H3 User Control] (Severity 2)
(Jess 9)
Changes to list settings only apply to the event list specified at the top of the screen. However, users may want to apply setting changes to all event lists. The interface could provide this option by including a check box that applies the current change to all event lists.
11. [H4 Consistency and Standards] (Severity 3)
(Nate 5; Andrew 2)
Options screen: up/down arrows on the distance and time selectors imply mouse-based interaction, but I doubt this is how it would work. Graphics should suggest arrow-key interaction. (If this is an artifact of a Flash widget, please ignore.)
12. [H4 Consistency and Standards] (Severity 2)
(Andrew 10)
Wording: “Call Venue” (verb noun) and “Venue Website” (noun noun) is inconsistent. Change latter to “View Website”?
Map View Violations
13. [H6 Recognition over Recall] (Severity 3)
(Nate 7; Jess 6)
The numbers on the map markers do not correlate with anything in the list view. Such a correlation of event numbers would provide the user with an easier way to remember a particular event found on the general events interface.
14. [H5 Error Prevention] (Severity 3)
(Jess 7)
The writing used on the map marker does not fit on the screen provided. This could lead to errors and confusion because the user cannot view the complete message conveyed in the marker. (Possibly this is just a bug in the Flash implementation.)
15. [H4 Consistency] (Severity 2)
(Nate 9)
The map interface has potentially many more functions than simply ‘Zoom’ available to it. Perhaps the ‘Zoom’ function could be changed to the ‘Options’ function (for consistency), allowing for the addition of functions like ‘Find my Location’, etc.
16. [H1 Aesthetic and Minimalist Design] (Severity 2)
(Andrew 8)
The function of “Select” before a flag has been highlighted is ambiguous. Don’t I select using the number keys first? Select should be hidden until a marker is chosen. Wording is also somewhat confusing.
17. [H6 Recognition Over Recall] (Severity 2)
(Ken-ichi 3)
Map view does not indicate time until event except when event is selected. Might be helpful to indicate events that are about to start with a different color or shape.
Event Detail View Violations
18. [H8 Aesthetic and Minimalist Design] (Severity 2)
(Nate 10)
If an event detail is navigated to from the map screen, "Map" and "Back" duplicate the same action.
Recommendations
Our overall recommendation is to fix the usability violations that include severity level ratings of three and four. The following describes specific recommendations for each of these more severe violations. General recommendations include changing the select option to the center button, increasing the font size as well as modifying color choice and the comedy icon in order to improve readability, and maintaining consistency with the right and left soft keys. Recommendations specific to the map view include aligning map marker numbers with their respective list view entries and adjusting the size of the map marker to fit on the screen provided. And lastly, recommendations specific to the event detail view include providing arrow and key instructions.
Andrew's Evaluation
Task 1
1. H4–Consistency and standards, Severity: 2
Up-down buttons and left-right buttons control different cursors. If this can’t be avoided (it certainly is efficient), it should be clearly indicated.
2. H4–Consistency and standards, Severity: 3
Options screen: up-down arrows on the distance and time selectors imply mouse-based interaction, but I doubt this is how it would work. Graphics should suggest arrow-key interaction.
3. H4–Consistency and standards, Severity: 3
Back button turns into map button on main screen. Back is a function I expect to be persistent across pages.
4. H8–Aesthetic and minimalist design, Severity: 3
Will text be readable on a phone? The mock phone is relatively huge, and the text is already very small.
5. H4–Consistency and standards, Severity: 2
To me, the middle button should be the “select” key so I can move among options and choose one without moving my thumb (this could depend on the phone).
6a. H2–Speak the user’s language, Severity: 1
The meaning of “back” on the options screen is unclear.
6b. H5–Provide feedback, Severity: 2
Once I’ve made changes to the options, “back” should change to “done” so I know changes aren’t being lost.
Task 2
7. H4–Consistency and standards, Severity: 3
That the arrow keys have switched from selection to panning is not intuitive or clearly marked.
8. H8–Aesthetic and minimalist design, Severity: 2
What does “select” mean before a flag is highlighted? Don’t I select using the number keys? Select should be hidden until a balloon is chosen.
9. H4–Consistency and standards, Severity: 2
On detail screen, “map” switches sides of the screen.
Task 3
Similar to Task 2
Task 4
10. H4–Consistency and standards, Severity: 2
Wording: “Call Venue” (verb noun) and “Venue Website” (noun noun) is inconsistent. Change latter to “View Website”?
11(9). H4–Consistency and standards, Severity: 2
"Map" switches sides again.
Jess's Evaluation
Introduction
The following is the individual heuristic evaluation for the Play it by Ear First Interactive Prototype. It covers the interfaces represented on both the interactive prototype and the representative storyboards found on the Play it by Ear website. The evaluation includes separate evaluations of each distinguishable interface included in the prototype. These separate interfaces consist of the general event interface, the map interface, the individual event interface, and the settings interface.
In general the prototype was fun to use and does a great job of balancing the tough constraints of a mobile phone interface: the limited screen space and the limited buttons available to select multiple options. The prototype does a good job maintaining consistency throughout the different interfaces, such as using a uniform selection box (the red outline found on the general event interface and the settings interface), displaying the event category icons (found at the top of the screen on both the map and general event interfaces), and maintaining the consistent location of the back button (the selection button located on the right). Heuristic Evaluation:
General Event Interface
1. [H2 System and Real World] (Severity 2) The wording "Etc" has the potential to be confusing to users. The word "Etc" is generally used to describe the continuation of some concept. According to this interpretation, the "Etc" representation would imply that the respective category is a continuation of the previously listed categories. However, the contents of this category would be better described as events that don't fit under the other categories. Therefore, a more appropriate word choice might be "Miscellaneous", with the shortcut represented by the abbreviation "Misc".
2. [H7 Flexibility and Efficiency] (Severity 4) The font size and choice of font and background colors make this interface difficult to read. This heuristic violation applies to both to this interface and the individual event interface. On the other hand, the size and color choices of the settings interface are very easy to read; it would be great if the the general event and individual event interfaces could adopt the same readability.
3. [H2 System and Real World] (Severity 1) The 24-hour clock is not intuitive to many domestic users. In general, domestic users find a 12-hour clock more intuitive. However, international users will find the current clock metric more intuitive. Therefore, it is important to adopt the clock metric most appropriate to the potential users.
4. [H6 Recognition not Recall / H7 Flexibility and Efficiency] (Severity 2) There are currently no category types within an event grouping. Category types could provide both efficiency and other required background information about an event. An example of a category type is specifying that the Giant and Dodger game is a "Baseball" sporting event. This could be useful for users that aren't familiar with the area or with the types of events that occur at specific venues. While a local will likely know that baseball games are played at AT&T park, a visitor (unless a baseball fan) will not know that the park is a baseball stadium or that the Giants are a baseball team.
Map Interface
5. [H2 System and Real World] (Severity 2) When the user scrolls over a map marker, the message displayed on the marker ("undefined 20:00:00") includes confusing wording and unnecessary detail. This is because the interface does not clarify what exactly is undefined and the third set of decimals (that indicate the seconds component of the time) are not useful in this scenario.
6. [H6 Recognition not Recall/H7 Flexibility and Efficiency] (Severity 2) The numbers used to represent the map markings do not correlate with the events found on the general events interface. The correlation of event numbers would provide the user with an easier way to remember a particular event found on the general events interface.
7. [H5 Error Prevention] (Severity 3) The writing used on the map marker does not fit on the screen provided. This could lead to errors and confusion because the user cannot view the complete message conveyed in the marker.
Settings Interface
8. [H10 Help and Documentation] (Severity 3) This interface does not include documentation that instructs the user to touch the screen in order to make setting changes. The interface could provide this documentation through the use of a light box, such as the "Press 0-9" documentation provided on the map interface.
9. [H3 User Control] (Severity 1) Changes to list settings only apply to the event list specified at the top of the screen. However, users may want to apply setting changes to all event lists. The interface could provide this option by including a check box that applies the current change to all event lists.
10. [H1 Visibility] (Severity 2) Once the user changes the list settings, the interface provides no indication whether his or her changes were successful. Therefore, a message indicating that the settings were successfully updated would provide the user with helpful feedback.
Individual Event Interface
11. [H4 Consistency and Standards] (Severity 3) The general event interface coordinates the map with the right-most selection button. However, on this interface, the map is designated as the left-most selection button. This change in location is confusing, considering that all other selection buttons are consistent throughout the interfaces.
Recommendations
In general, the event details provided by the prototype thoroughly cover the information components required by a potential user. However, two potentially helpful details that are absent from this prototype include the business hours of the venue and estimated duration of the event. Furthermore, for the map interface, other potential considerations include the need to differentiate between events located in the same venue and maintaining a map display without too many map markings.
Ken-ichi's Evaluation
Violations
1. [H1 Visibility of System Status] (Severity 2) Sort order of initial List View is not immediately obvious. Time (or perhaps time from present) could be emphasized to indicate chronological sort, or sort order could be stated explicitly.
2. [H5 Error Prevention] (Severity 2) If the list does not update immediately after navigating to the category in the List View, they may be tempted to hit the center button to select the category, and accidentally open the sort options.
3. [H6 Recognition Over Recall] (Severity 2) Map view does not indicate time until event except when event is selected. Might be helpful to indicate events that are about to start with a different color or shape.
4. [H4 Consistency and Standards] (Severity 3) Right soft key is "Back" in almost all screens except the List View, where it is "Map." This causes problems when the user repeatedly hits "Back" to get back to the List View, without thinking how many times he hits it, and then starts to oscillate between the List and Map Views. Additionally, the "Map" action switches to the left soft key in the Detail View, which is inconsistent.
5. [H2 Match Between System and Real World] (Severity 3) The center key seems the natural choice for "Select" in the List View, but instead of selecting an event, it opens the sort options. Mapping the "Select" action to the left soft key may adhere to some convention for mobile phones (it isn't a convention on my Samsung), but it is unnatural because selecting an event will be a much more frequent action than changing sort options, and since the center button is centrally located right under the thumb, it feels like the natural selecting key. There is no natural mapping for "Sort," suggesting it might be better mapped to a soft key, where it can have textual annotation.
6. [H6 Recognition Over Recall] (Severity 3) It is not obvious how to actually call a venue with a listed phone number in the List View.
7. [H8 Aesthetic and Minimalist Design] (Severity 1) Comedy icon does not scale well at the resolution of this prototype, and would presumably be worse on a phone. Perhaps an image that was more iconic and less representational would scale better, and might have more semantic clarity.
Things I Like
- Text in addn to categorical icon
- comedy icon = circus icon
Technical Problems
- "Back" from an event viewed from the map view doesn't refresh the screen (first "Back" shows the same Detail View, "Next" shows the List View).
- "Map" in Detail View is broken.
Notes
Task 1
- I went immediately to the map view after selecting the music category
- Event titles are "Undefined" in map view
Task 3
- Task 3 is impossible w/o a functional map
Nate's Evaluation
General
1. [H3 - Consistency and Standards - S3] - I consistently used the center button inside the navigation controls as the selector button. This is mapped to the options action, which seems awkward.
2. [H3 - Minimize Memory Load - S3] - The ‘Map’ button on the right is awkward because the right button is ‘Back’ in all other parts of the application. This should be changed to the least used control, which I suggest would be ‘Options’. ‘Map’ could then become the left-most button.
3. [H2 - Speak the User’s Language - S3] - The affirmative and cancel commands under ‘Options’ are ‘Select’ and ‘Back’. The vocabulary might be something along the lines of ‘Accept’ and ‘Cancel’.
4. [H4 - Consistency - S2] - Under the ‘Options’ menu, it appears that the control paradigm shifts towards using the center button as the primary action button as I suggested in #1, while the left ‘Select’ button is the final acceptor button. This is inconsistent from the current design of the rest of the application.
5. [H9 - Prevent Errors - S3] - The ‘Options’ menu contains two fields, ‘miles’ and ‘hours’ that have up and down arrows presumably to allow the users to adjust the quantities in the form. It is unclear how one would use the arrows, given that the up and down controls on the keypad are reserved for moving the red selector box. Since a phone is essentially a number pad, it is suggested the up and down arrows are removed and the user just be able to type in their desired value. (If this is an artifact of a Flash widget, ignore).
6. [H7 - Provide Shortcuts - S2] - On the landing page, scrolling right or left is halt at the end of the list of categories. It should cycle back through to the beginning of the menu.
7. [H4 - Consistency - S3] - There is no clear link between the list of events and the map view. It would help if the list of events each had a number, and the map icons then displayed this number. The event closest to the user in the given category should be the cardinal number, and the number in view should be the selected event from the list, not the location of the user.
8. [H1 - Simple Dialogue - S2] - It is unclear how the events are currently sorted on the landing screen. It seems to be by time, but then time repeats in the list, suggesting a new day’s worth of data is displayed along with the current day’s.
9. [H4 - Consistency - S2] - The map interface has potentially many more functions than simply ‘Zoom’ available to it. It is suggested that the ‘Zoom’ function be changed to the ‘Options’ function, allowing for the addition of functions like ‘Find my Location’, etc.
10. [H4 - Consistency - S3] - Pressing ‘Select’ on a map location displays that location’s details page. Choosing ‘Back’ at this page now returns the user to the list, even though the map was the last location. Suggestion that the wording is changed or that the action is consistent with normal ‘Back’ behaviors.
11. [H10 - Help and Documentation - S2] - Though of limited scope, this app could use some help functionality, or interactive dialogues to help the user remember where they are in the interaction flow, or assist them in returning to a previous screen correctly.
Scenario 1
12. [H1 - Simple Dialogue - S2] - There is no method documented on how to enter time ranges, which scenario one employes. If this has been scoped out of the design, ignore.
Scenario 2
13. [H1 - Simple Dialogue - S2] - The ‘History’ function is nowhere to be found. Again if scoped out, ignore.
Comments
This is a great prototype and a great app. I actually don’t think there is a lot wrong with the concept, and suspect that most of the bugs are due to the relatively short incubation and coding time we have available. I do think that there is a tendency to emphasis the ‘location-aware’ part of the app because it seems like an interesting new technology, but would suggest that the focus be on the events when switching between the map interface and list interface. On the map, the user’s location should be highlighted, but it should not be the primary focus of using the map.
