McInterface: Pilot Usability Study Results and Discussion

McInterface
User Interface Design & Development Project
SIMS 213, Spring 2001

: Linda Harjono, Saifon Obromsook, John Yiu Chi Wai

Pilot Usability Study

Results

The results of the usability testing are very useful for us. These are the compiled results corresponding to the test measures.

1. Direct vs. Indirect ordering process

The testing made it very clear to us that the 'direct' version is definitely not an option. It is only advantageous by make the ordering process a bit faster (by one screen) when users do not want many options with their order (i.e. only works with 'Quick BigMac' scenario). However, the 'direct' design version is not flexible enough for other situations. In fact, it cannot handle other scenarios at all. The test subjects did not know that there are special request options since they did not know that there exists a more detailed options page associated with each 'Options' button. One test subject tried to find ketchup in the menu when asked to get more ketchup for his girlfriend ('Many Item' scenario), which we think is a very serious flaw in our design. Also, the participants think that the display of size and drink choices made the main menu screen very cluttered.

Another test subject commented that the Indirect version was "more like real-world situation," since the flow of food purchasing process was like the usual way customers talked to the person at the cashier.

2. 'Remove' button vs. the quantity changer on Order Summary

For removing an order item from the order summary, it was not obvious at all for users to do so by changing the quantity to zero. However, it was obvious when there was a 'Remove' button. For changing the quantity, one participant said that he still needed to think a little bit before figuring out that he can add the quantity of an ordered item on the Order Summary through the 'Edit' button, as opposed to through placing another order of the same item.

3. Price display

All participants did not need a complete price list (of each item's different sizes) on the Main Menu screen.

4. Menu categories' placement

All participants are comfortable with finding certain items. One suggested that the New Tastes Menu should be in an upper position in the menu bar at the left, and that there should be some animation to inform the customers about special promotions.

5. Food categories' tabs

All participants did not think that the tabs are obvious enough to tell that they can be selected. They say that using buttons might make it more obvious.

6. Special request

All participants really liked the icons. One suggested that there could be more special requests like no mayonaise and no cheese.

7. Instructions and button labeling

The label 'Options' was not very meaningful. The participants like 'Edit' better, although one suggested that it could be changed to 'Change Order.' He also said that 'Remove' could be changed to 'Cancel.'
In the direct version, the instruction on the top of the menu display is not noticable to one participant. After a while, we finally had to tell them what to do. In the direct version, the 'Order' buttons worked better, but they still cluttered the main menu screen.

8. Users' overall feelings towards the design

All participants like the idea of our system and think that it would be really useful in practice. They also think that the direct version of the prototype is usable enough to be put in real use (assuming there are no programming bugs), but there is still some room of improvement.

There are also some unexpected feedbacks from the participants:

Some food category display pages are too cluttered (for both versions), especially the Extra Value Meal page. One suggested that we could use vertical tabs or just simply separate them into two categories (Extra Value Meal 1-5, Extra Value Meal 6-10).
The pictures of the food on the main display page are too small.
There should be French Fries' pictures in all available sizes on the menu display.
The Happy Meals Toys category is confusing since it is not a food category. It should be somehow distinguished from the food menu or relabeled.
There should be feedback on toy selection in the Order Summary
The menu should have buttons of categories on one screen and items of each category in the next screen (hierarchical menu).
There should be an abbreviated order summary on the Payment Choice screen instead of showing only the total.

Discussion

We are now certain that we will start building from the 'Indirect' version of the prototype. These are the design decisions resulting from the experiment:

Use the 'Remove' button.
Display only one price on the Main Menu screen.
Revise the categories placement by highlighting the New Tastes Menu and distinguishing the toy preview page.
Change the display for categories from tabs to buttons.
Consider adding more special request options
Use 'Order' buttons next to each image instead of using an instruction on the top. We think that the reason why that instruction is unnoticable is that naturally images will catch our eyes better than text. Since that page is full of pictures, it is expected that users will overlook the textual instruction.
Consider splitting Extra Value Meal items into two categories and enlarge each picture on the Main Menu page.
Consider showing French Fries in all sizes

We will incorporate these items into our next prototype. The rest of the comments are given a lower priority.

An interesting lessons we learn from the result of this pilot testing is that designers who are, especially later in the process, deeply involved in the design can really overlook many critical flaws of the design. We also found out that, in some cases, we cannot focus too much on solving heuristic violations, since the solution can very well be in the expense of something more critical, which we sometimes overlook.