TraveLite is a web-based application for building a customized travel guide for download to a PDA.

This informal usability study was designed to identify aspects of the TraveLite system that users find difficult or easy. We also sought feedback from participants regarding how TraveLite could improve to match needs and expectations. This evaluation was conducted on the Second Interactive Prototype.

Method:

Participants: We tested four participants, all males age 30-34. Participants were chosen based on having at least some Internet and travel experience. Two of the participants are involved in digital product development for a travel content provider. One is a biotech researcher. The last is a graphic designer. Most of the participants have used the Internet for at least 7 years and consider themselves very experienced. The last participant has used the Internet for 4-6 years, and considers himself fairly experienced. Most of our participants do some research online. One participant considered himself inexperienced with online travel information, but very experienced with both print travel information and researching. The other two indicated that they were inexperienced to somewhat experienced with print travel information but fairly experienced with online travel information and somewhat experienced in researching. The fourth participant is very experienced with both online and printed travel information and with researching a trip. He also has the most extensive experience with online travel resources [Note: this participant is involved with digital product development for a travel content provider].

Equipment: We tried to choose testing locations where the participant would feel comfortable. We also required an IBM compatible PC running Internet Explorer v5 or later and access to a T1 line. For one test, we used the particpant's business office, for the other three we used the SIMS student lab.

Tasks: For the purposes of this test, we adapted elements from all three scenarios used previously. We wove the tasks into a single story line that incorporated all the primary tasks for all three personas. We wanted to ensure that each user visited every important page of the site and exercised each part of the functionality. [New Test Scenario] In the scenario, the participant is interested in creating a guide to San Francisco, they use the TraveLite system and work through the process of creating a guide, then they need to save the guide. In the next task, they return to the guide, make a few changes and then download the guide.

In the Test Scenario, we include a final task that allows the participant to create a guide of their own design. Following the first session, however, we decided that this was unnecessary and would not yield many more results than the preceding task. Therefore, we eliminated this task from later tests.

Procedure: For each session, we read through a printed script that explained the purpose of usability tests; described the TraveLite system, including a few caveats about functionality and content; and introduced each member of the team and their role in the process. We also explained 'talking aloud' and asked users to do so while performing the tasks so that we could gain an understanding of what they were thinking while interacting with the system. We explained to them that we particularly wanted to know what they found easy to use and areas that were particularly frustrating. We then had each participant fill out an Informed Consent Form and Pretest Questionnaire. For each test we had at least two team members present, one acting as a facilitator and one as a note-taker and time keeper.

Following the completion of the tasks, we held an informal discussion with the participant. We asked them what they thought about TraveLite in general. We also followed up on any elements that they found particularly confusing and asked what they would have preferred. We then had them fill out a Post Test Questionnaire.

Test Measures
For this informal usability test, we measured task time completion, difficulty ratings for each of ten subtasks recorded on the posttest survey, and clarity ratings for each of the seven major categories on the site. We wanted to ask explicitly about task difficulty and categories because it is possible to perform a task without thinking it is easy to do so. We also wanted to contrast how people felt about the system after having used it with the results of actually watching them use the system to see if there were differences.

We wanted to time task completion to get an idea of how long it would take the average user to build and maintain a guide with the system. We found that most users, however, were either having too many problems with the system or spent so much time playing and exploring the system that this measure is relatively meaningless. Since our application is meant to foster exploration and fun, task completion time may never be a useful measurement for the task of building a guide. This measure may be more applicable to our Rapid Guide feature, which we did not test.