Pilot Usability Study & Final Presentation

Assigned: April 8
Writeup Due: April 29 Before Class
Presentations: April 27 & 29 In Class
Poster & Short Presentation Session: May 5 at 6pm in Sutardja Dai Hall

For this assignment, you must complete 5 tasks:

  • Conduct a usability study of your interactive prototype with at least 3 subjects. Each subject should perform at least 3 tasks with the prototype.
  • Improve your interactive prototype based on the results of this user study. You should also add new features to cover typical tasks that users would want to perform with your application beyond the representative tasks of your usability study.
  • Present your final prototype and your design process in a 12 minute presentation during the final week of class.
  • Write up a short report describing your usability study. (Note that this report need not contain any details of your final prototype, which you will describe in the Process Book).
  • Present your application via a short 2.5 minute presentation and poster at a joint demo session for industry and faculty members on May 5 starting at 6pm.

This assignment is 15% of your overall grade, which has been broken down into 75 points for grading. The breakdown of points is discussed below.

Step #1: Conduct a Usability Study
Start by conducting a usability study of your interactive prototype. The procedure that you use for this study should be similar to that of the study that you conducted for the low-fidelity prototype assignment, with one important difference: subjects should not think-aloud during this study. Instead, you should focus on recording representative performance data, such as the amount of time required to complete the tasks.

Because you will not be able to get feedback from the subjects immediately when they encounter difficulties using this procedure, it will be more important to conduct a good debriefing following the study. Make sure to have observers taking notes when subjects appear to have difficulties, so you can revisit these situations during the debrief.

Optionally, you may wish to consider recording video and audio of the usability study sessions and watching this video with the subject as part of the debriefing. If your subject has time, you can ask them to talk aloud while watching the video and explain what they were thinking as the study progressed.

As with the low-fidelity prototype study, you should do the following:

  • Run at least 3 subjects
  • The subject should perform at least 3 representative tasks with different levels of difficulty: one easy, one difficult, and one in the middle.
  • Obtain informed consent from your subjects using an appropriate form.
  • Collect a log of critical incidents with associated severity ratings.

Step #2: Improve your Interactive Prototype
During and after your usability study, you must spend some time modifying your interactive prototype. You should make at least a couple of each of the following types of changes:

  • Modifications to improve usability: You should alter the design of your interface based on what you learn from your usability study. Be prepared to describe these changes in your process book.
  • Modifications to increase functionality: Most likely, the current version of your interactive prototype does not implement the full set of functionality that you envision for your application. Add some features to your prototype, and make sure they are well integrated with your user interface design. Also be sure to have a rationale for why these features will not hurt the usability of your prototype. You may also implement these features before conducting your usability study if you have time, so that you can evaluate their impact on usability as a result of your study.

Step #3: Present your Process & Final Prototype In Class April 27 & 29, 30 points
You will have 12 minutes to present your design process and final prototype to the class on either April 27 or 29. A presentation schedule will be posted on the wiki by April 22.

The focus of this presentation should differ from the interactive prototype presentation that you gave earlier. Instead of demoing your prototype, you should focus on describing your design process. For example, show screenshots and sketches that demonstrate how your design evolved over the course of the class. You may still demo your prototype of course, but try to limit the demo to showing features that the class has not already seen in your interactive prototype presentation.

You should also be sure to mention techniques that did or did not work well for your group as you designed, implemented, and studied your interface. The goal is for the entire class to be able to leverage the experiences of every group in future projects.

Your presentation should be uploaded to the wiki and linked on your group page. This may be done after class on April 29.

Step #4: Write up a short report documenting your usablity study Due April 29, Before Class, 45 points
Write up a brief report covering just the usability study. Note that details of your final prototype design should instead be documented in your process book. The structure of your report should be similar to what you already turned in for the low fidelity prototype assignment. Thus, the format should be:

  • Each team member's name and role in this assignment 5 points
  • Experimental Method 15 points
    • Participants (1 paragraph)
    • Environment (1 paragraph)
    • Tasks (1 paragraph)
    • Procedure (1 page)
    • Test Measures (1/2 page)
  • Experiment Results (1 page) 10 points
  • Experiment Discussion (3/4 page) 10 points
  • Appendices (as many pages as necessary - link from text into the appendices) 5 points

As with previous assignments, please create a page on the class wiki for your report. Link your report both from the Pilot Usability Study page and your own group page.

Step #5: Present your project via a short presentation and poster May 5, 6pm
On May 5, starting at 6pm in the CITRIS auditorium of Sutardja Dai Hall, we will have short project presentations followed by a longer poster session. These presentations will be joint with CS 160 and a class from the Center for New Media. Free food and beverages will be served.

On May 5th at 6pm you will present your final interface designs in person to the teaching staff in the Banatao Auditorium in Dai Hall. Since we have teams from 3 classes, your slot will be short: 2 minutes and 30 seconds. Only one member from your group should present, but all members should be on stage. Because of time constraints, you cannot use your own laptop.

  • You must use Google Docs Presentations to prepare your slides.
  • You can add video to your presentation by uploading it to YouTube - Google Docs can integrate YouTube clips.
  • Guideline: 5 slides maximum.
  • Cover motivation (who wants to use your interface and why?), and give a demonstration of the final interface either through screenshots or videos. There won't be time for the complete story. Focus on the needs of your users and the benefits your solution provides. Most of the audience will not have seen your project before - make the presentation understandable and compelling for them.
  • You must submit a link to your slides via the Wiki by Monday, May 3, 6pm so we can test your presentation.
  • If you use video, also submit a link to your videos on YouTube or upload a backup of your video files to the Wiki.

On May 5th at 7:30pm, right after presentations, you will demonstrate your interface to the teaching staff in a poster+demo session. You will design the poster; we will print it for you. Please make sure that your interface is in working order and ready to go for the demo. If your interface requires showing interactions between multiple users, bring multiple devices. (You can bring one device and a laptop with a simulator if you don't have access to multiple devices). If it requires some data in a database to be useful, make sure you enter enough data.

  • Design your poster and send us a 30"x40" PDF file. Be aware that if you send a PDF with smaller proportions, then the poster printer will scale it up to fit within a 30x40 box.
  • You must submit your poster file by Monday, May 3, 6pm so we can print your poster. Link your PDF on the Wiki and send it to jwnichols@us.ibm.com.
  • Practice a demo "pitch" with your friends - you should find a balance between demonstrating some functionality of your interface and letting visitors try the application first-hand.