Results

We derive the following results from incident logs (see appendix C) and interviews with the subjects after the testing.

• Most subjects hesitated and asked the meaning of 'MyFav.'
• Some buttons were redundant, some were at wrong places, some are missing from where there should be. For example, there was no need of a back button on language (if cancel, go to first page without getting any card). The back button on the menu page was redundant since going back to the first page is not useful. One subject spent a long time finding the 'check out' button.
• The checkout screen needs a redesign.
• The Happy Meals page needs a redesign.
• The whole process was still not quick and easy enough.
• There was a need to be able to use cash with the system.
• There was a need to be able to use a credit/ATM card to directly purchase the food.
• The sauce selection was at a wrong place.
• Need more cues for the card swiping process.
• Need more instructions on some pages.
• Need to relocate some instructions.

Discussion

There are many interesting findings from the evaluation. The most interesting one is how much the appearance, specifically, the neatness, of the prototype seemed to really affect the subject's willingness to give negative suggestions. One of the subjects even said sorry when he told us to get rid of one main element of the system. So, we explained that it was ok and that it was something we expected. He strongly agreed that if the prototype had looked neater or been digitally-coded, he would probably have not given such feedback.

One finding reminded us to focus more on the locus of attention of the users. One user got frustrated when he faced with one page. He later told us that he had to make too many decisions on that page and we should split that screen. Another subject said the same thing and also commented that the line of decisions was unnatural. Our design prototype aims at having as fewest screen as possible so that users do not feel having to go through too many steps, but did not take into account how users' mind would work on one page. We should have placed the component such that the users' locus of attention moves in a natural way -- linear and top-down.

We also discovered we made a big mistake designing the display of toys, which come with Happy Meals. Our menu of Happy Meals look like other categories, users need to choose the meal before choosing the toys. This is actually not the sequence that happens to one of our subjects. She thought aloud that she was looking for toys, yet our design did not facilitate her with this task. What was going on in her mind was that she first looked for pictures of toys before deciding about the meal itself.

Another major finding was about how the system works. One subject did not think there was an enough incentive for users to have to buy a prepaid smart card using credit/ATM card just to have it record their favorite orders. The users should be able to buy the food using the credit/ATM cards itself. One subject would want to use cash if he could. This would lead to a major change of our design. Luckily, we can make this change at an early stage because of this testing.

Due to the overwhelming number of useful responses, we cannot discuss all other minor (yet important) findings. Therefore, in addition to the above implications in changing the design, we came up with a list of both changes we made on the fly (both during the debugging and the testing) and planned changes in our design. Note that the reason why some plans are not yet definite is because we still have to figure out how the pieces fit together.

• We eliminated the back and next buttons, change it to OK and Cancel.
• We relocated and enlarged some buttons.
• We added on the last screen that the order number is also printed on the receipt
• We changed  'MyFav' into 'My Favorite'.
• We added a plus and a minus on each quantity in the tray, and also a quantity for the sauces.
• We erased the clear button on the keypad and draw a bigger one next to the number display.
• We added some textual feedback on special request and toy.
• We added 'Add Value' and 'Cancel' buttons to the error screen that pops up when the user tries to add an item that makes the total charge exceed the current card value.
• We need to add an instruction of how to change an existing order.
• We need to add an arrow pointing to the card reader.
• We need to associate sauce selection with each order, not on the confirmation page
• We need to let people see the toys before ordering Happy Meals
• We need to make it clear that the special request choices are exclusive for each subject item
• We need to think about how to allow non-McCard users to use the system.
• We need to think about how to keep the quickest version of buying scenario quick and simple
• We need to think about a better way of presenting the menu since the menu items under 'other' category were not given upfront.

This evaluation, however, might not be able to tell how real users will respond to the real system for the following reason:

For our interface, the speed of a human computer might have an impact on the impression of the subjects. Since the system's users are likely to be hungry at the time. Speed should be given a top priority. With an incredibly slower speed, the subject might perceive the interaction to be rather slow and is led to think that the interface might not be convenient to use.

The use of fake data and graphics seem to slow down the subjects' response. Graphical representations of the menu seemed quite important for them to make decisions. With the real interface, the users could have better ideas while browsing the menu. In this testing, the subjects' hesitating gestures might originate from the lack of accurate graphical presentation, instead of from the design itself.

By specifying scenarios for the subject, we could not account for all the possible cases that can happen. For the real system, there can be surprises. Also, giving the subjects some constraints upfront, by specifying the tasks, can limit some design possibilities that can otherwise be realized if the subjects were allowed to test the interface freely. We suspect that most subjects did not complain about the payment process because we told them that they were supposed to use credit/ATM cards to add value to McCard when necessary.