TraveLite Heuristic Evaluation - Group Report
April 5, 2001
Evaluators: Moryma Aydelott, Kirsten Swearingen, Amity Zeh

Project Overview

Our group conducted a heuristic evaluation of TraveLite, a system for customizing travel guides that may be downloaded to a personal digital assistant (PDA). Background on TraveLite is available at http://www.sims.berkeley.edu/academics/courses/is213/s01/projects/P8/. The system itself may be viewed from http://fusion.sims.berkeley.edu/travelite/interactive_sarahV1-2/. The heuristic guidelines the TraveLite team requested we use appear at http://www.sims.berkeley.edu/%7Ejenglish/HeuristicsUsed.htm.

This system does a good job of making a very difficult task comparatively easy, of grouping information in logical ways, and of providing an appropriately uncluttered, aesthetically pleasing design for the task at hand. TraveLite also offers its users maximum flexibility and control over the guide content. Our heuristic evaluation turned up several areas in which this interface has flaws, and further areas where the design could be improved. Following is a discussion of the broad areas we identified as needing further attention; a summary appears below, followed by links to the complete heuristic evaluation logs (group and individual).

Note on Findings

At times, we found it difficult to assess whether a usability problem existed or whether the needed functionality just had not yet been implemented. In other places, because the prototype showed only a small portion of the total content, it was difficult to assess whether a user would be able to achieve his/her goals at this content-intensive site. For these reasons, our notes frequently comment on our uncertainty as to the true extent of the problem cited.

Overall Recommendations

Our recommendations concern four aspects of usability (in order of overall severity and frequency): system status, site-wide consistency, user freedom and control, and matching to the real world.

System Status

On the whole, lack of system status information presented the most serious usability problem. All three evaluators expressed discomfort with the amount of feedback the system provided during the guide-making process. The evaluators wanted to know upfront about the number of steps/screens and time involved in building a guide, as well as continual feedback as they progressed from screen to screen (e.g. "You are completing Step 2 of 3"). The numbered steps on Choose Guide Type and Size. In addition, the screens were similar to each other and a few of the screens (such as "Edit Regions") did not have labels, which added to the sense of disorientation. It was also difficult for a user to know the status of his or her information; whether or not the guide was currently "purchased," "available," or "saved."

Adding the site tour will help a great deal, but we also recommend adding location and process cues to each screen, for those users who opt to bypass the tour.

Site-Wide Consistency

The evaluators found a variety of inconsistencies--links with the same name leading different places, or with different names leading the same place. For example, clicking on Preview sometimes showed "Guide Content" and sometimes showed "Country Background Information." On the other hand both “Create a New Guide” and “Purchase Destinations” leads to “Add/Remove Destinations from Guide.” Taken individually, each such problem did not create a serious difficulty for the user; taken together, they caused a high degree of user confusion and frustration.

We recommend a close review of the site map and individual screens, to create a list of link labels and the actions each one invokes, in order to eliminate any inconsistencies.

User Freedom and Control

The problems with consistency and system status led to moments when the user had to take a next step without being certain what the consquences would be. For example, by clicking "Let Me Build My Guide," a user entered a process of undetermined length. Similarly, clicking on "Finished" from the "Edit Regions" screen implies an irreversible action, when in fact this only saves the guide for later editing. Clicking on "Edit" from the Preview page, would seem to be a harmless action, but brings the user back to the beginning of the process, to the Add/Remove Regions Screen, and discards any changes made up to that point.

Taking the steps described in the previous sections should serve to address this concern as well.

Edit Regions - Matching to the Real World

Most of the usability violations occurred on this screen, which is by far the most complex and feature-filled screen. We liked the use of tabs to separate the different facets of information. However, we had trouble understanding how to work with the tabs. Because of the graphic design, it was not immediately apparent that the tabs WERE tabs, and then that the check boxes would include or exclude the information on that tab. Additionally, the Check All / Uncheck All option at the bottom of the screen seemed to repeat the checkbox functionality. It was also not clear whether changes would be saved when moving from one tab to another (this may have been due to missing system functionality).

We recommend eliminating the checkboxes from the tabs or the Check All / Uncheck All links. More specific suggestions on clarifying the language and display appear in the summary log.

Summary of Usability Findings

Appendices

Combined log of heuristics violations: HTML or MS Excel (better for printing)

Individual evalutions
--Moryma Aydelott
--Kirsten Swearingen
--Amity Zeh