Goal:

The goal of this assignment is to learn how use low-fidelity prototyping in the early stages of UI design. You will first build a paper-based prototype and then perform a simple usability test. You will incorporate the results of the test into design changes in your prototype for the next assignment.

Prototyping:

Design and construct a paper-based prototype using the techniques outlined in the CACM article by Mark Rettig and in the lecture by James Landay. This will be based on your sketches and your task analysis from the last assignment.

Your low-fi prototype should be able to support execution of the scenario tasks that you described in the last assignment . These scenarios should give good coverage of your interface; if they don't you should redesign them in advance. They should encompass both easy and more complex tasks.

Testing:

Preparation:

To prepare for the evaluation you should assign team members to the different tasks (i.e., computer, facilitator, etc.) and practice with someone playing the participant.  Get any bugs out of your procedures. It may take about 4-8 hours to create the materials and several more hours to practice playing computer.

Participants:

Find three participants (i.e., volunteers who are not in your group) to work through your task scenarios. It is OK to ask other members of the class to help out and participate in your test, but remember it must be voluntary. You shouldn't use friends. The type of people you recruit should be based on your task analysis.

You should ask the participants to sign an informed consent form saying the test results will be confidential. Be sure to let participants know they are free to change their mind about doing the study if they do not want to sign the form, but if they don't sign the form then do not do the study with them..

Here is an example of an informed consent document used for a study on campus. You need to modify it to fit your project's description.

Procedure:

You should write-up a script of how you will proceed and follow the same script with each participant. Tell each participant what they are trying to achieve, but not how to do it. When they are finished, you will give them the directions for the next goal and so on. Each participant will perform tasks to achieve all three goals. Keep the data separate for each task and participant.

Give each participant a short demo of how to work with someone "playing computer". Do not show them exactly how to perform your tasks. Just show how the system works in general and give an example of something specific that is different enough from your scenarios.

Instruct the participant to "think aloud" to let you know what they are thinking as they work. Remember not to give them hints or ask them questions while they are executing their tasks, but you may need to prompt them to keep them talking.

After each participant has finished the tasks, "debrief" them about the interface in general, and follow up on questions and issues that arose during the course of the task execution.

Location:

It would be preferable if you can find an empty room or a space with only a few other people in it.

Time:

As discussed in class, testing the interface should take about an hour per participant.

Measurements and Observations:

Keep a log of critical incidents (both positive and negative events). For example, the user might make a mistake and you notice it or they might see something they like and say "cool". Keep track of when they hesistate and when they proceed confidently. You may want to write this on a copy of the script or of sketches showing the interface at different stages. Later you should prioritize these events and give severity ratings to the problems you observed.

You should concentrate on process data (i.e., what is happening in the big picture) rather than detailed quantitative data (i.e., time or number of errors). Observers should jot down ideas for alternatives to the design based on what they observe the participants doing.

Results:

Report your results (summaries of the process data) and draw some conclusions with respect to your interface prototype. This should be the most important part of the write-up. We want to understand how you would change your system as a result of what you observed.

Write-up:

Your write-up should be no more than five pages of text and you must turn this in on paper and on the Web in your group directory. The write-up should follow this outline (number of pages/section are approximate):

1. Introduction
• Introduce the system being evaluated (1 paragraph)
• State the purpose and rationale of the experiment (1 paragraph)
2. Prototype
• Describe your prototype (1 page plus sketches & A photo of the test "computer" is recommended -- you can borrow the digital camera from the computer lab)
3. Method
• Participants (who -- demographics -- and how were they selected) (1 paragraph)
• Task Scenarios (1/2 page)
• describe each task and what you looked for when those tasks were performed
• Procedure (1/2 page)
• describe what you did and how
4. Test Measures (1 paragraph)
• describe what you measured or looked for and why (bulleted)
5. Results (1 page)
• Results of the tests (summary of what you saw)
6. Discussion (1 page)
• what you learned from the low-fi evaluation
• what you will change in your interface from these results
• what the evaluation could not tell you
7. Appendices (as many pages as necessary - link from text into the appendices)
• Materials (all things you read --- demo script, instructions read -- or handed to the participant -- task instructions)
• Raw data (i.e., entire merged critical incident logs - typed up!)

Grading - Your write-ups will be graded out of 100 points and the points will be awarded as shown:

• Required Outline Items [70]
• Creativity of Methods & Computer [20]
• Writing [10]