Introduction

Method & Test Measures

Results

Discussion

Formal Experiment Design

Appendix

Work Distribution Table

 
Assigment 8: Usability Testing

Formal Experiment Design

Hypotheses
Adding a slider to the live traffic map will confuse users by blurring the distinction between predicted vs. live traffic. We expect this to decrease completion time for Task 2 (the commuter who's determining when to leave this morning). We also expect this to produce a lower subjective satisfaction rating than the current design.

Instead, adding a button that says "Show me the typical traffic pattern today" (linking to the predicted traffic map with slider), may help flexible commuters to better determine their optimal time of departure. For example: if the live map shows bad traffic, users may immediately switch to the predicted view to see if traffic typically lets up in an hour or so. We expect that this link to the "typical pattern today" will produce faster completion time for Task 2, and also higher subjective satisfaction, compared to the current design.

Factors and Levels

The factor we are testing is the appearance of the live traffic map. There are 3 levels:

  • N - No slider. This is the current design, where the live traffic map appears without a time slider. If the user wishes to see predicted/forecasted traffic, he or she must plan a new trip, selecting the "Traffic Prediction" option.
  • S - Slider. This alternative design includes a time slider beneath the live traffic map, allowing the user to fluidly switch between real-time and predicted traffic views. Specifically, the slider tick labelled with the current time will produce a live map, while every other tick on the slider will show a predicted map.
  • B - Button to "Show me the typical traffic pattern today" positioned on the same page as the live map. This button links users to a predicted map, with the slider initialized at the current timestamp.

The response variables we will measure are:

  • Time to completion of Task 2
  • Subjective satisfaction with the system

Blocking and Repetitions

Because subjective satisfaction is a key measurement in this experiment, we believe that a between-groups testing strategy is necessary: once a user has seen 1 variation of the interface, his/her overall impression of the system will be biased for future variations. Therefore, we will conduct our experiment as follows: We will recruit 15 participants and divide them into 3 groups such that each group includes a balanced variation of driving behavior, computer experience and length of residency in the Bay Area. For each block, users will complete Task 2 on one of the three factor levels. Group 1 will use level N; group 2 will use level S; and group 3 will use level B. Task completion will be timed, and all users will be asked the same set of questions in a subjective follow-up interview.