Last revised 12/10/08 12:30 pm - broken links fixed
Skip to each month's assignments: October, November, December

Aug 28

Intro: concepts and methods

What this course will cover



Pls fill out and email both of us this student info sheet




Sept 4

User-centered design, iterative development, and planning
Methods overview

Kuniavsky, Chs.3 & 4

Courage & Baxter Chapter 3

Ben Shneiderman and Catherine Plaisant, Designing the User Interface : Strategies for Effective Human-Computer Interaction, New York; Addison-Wesley, 2004
Sections 1.1-1.4, Sections 3.1-3.4

Usability basics from process map and planning guide methods table

Methods list Old, but still applicable

Please spend a lot of time familiarizing yourself with common methods.

Assn 1: Naive usability assessment due



Sept 9

Visit from Kevin Lim, MIMS 08


Users: their concept and roles; problematizing 'users'

Multiple stakeholders: beyond end-users

Problematics of multiple groups/needs

Ethics of working with users overview learning about users

Kuniavsky Ch. 6 & 7

Oudshoorn, Nelly, and Pinch, Trevor. "How Users and Non-Users Matter," p. 1-16. In: Oudshoorn and Pinch, eds. How Users Matter: the Co-Construction of Users and Technology. MIT Press, 2003.

Hackos and Redish, Ch 2, Thinking about Users, Ch 3, Thinking about Tasks, Ch 4, Thinking about the Users' Environment

Skim these -- useful resources for ethics of working with users:

* Ethical Principles of Psychologists and Code of Conduct 2002; see especially sections on Privacy and Confidentiality.
* STC usability SIG on ethics especially the UPA Code of Professional Conduct (goes beyond working with users to usability work in general)
* UC Berkeley Committee for the Protection of Human Subjects - especially the guidelines on informed consent.




Sept 11


and experimental design

Most critical:

Rubin, Jeffrey, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley, 1994 (Rubin) Specific readings Chapter 2. Chapter 3, Chapter 5, Chapter 7, Chapter 8, Chapter 10 (This book is worth buying)

Kuniavsky, Ch. 10

Optional: Babbie, Earl R. (2001) The Practice of Social Research, 9th ed., Wadsworth. Chapter 5: Conceptualization, operationalization, and measurement (or equivalent chapter from another social research methods text).

Robert Opaluch, "Usability Metrics". In Ratner, Julie, ed. Human Factors and Web Development, 2nd ed. Lawrence Erlbaum, 2003. p. 101-144.

Optional, but recommended

Assn 2: Usability Test

Testing Slides



Sept 16


Recruiting users

Non-lab based usability testing

Mobile (moved to 9/18)


Experience sampling (moved to 9/25)

Recruiting Users

Kuniavsky, ch 6, to p 117 (on recruiting users) (added 9/15)

Sample usability testing materials

Remote usability

Susan Dray and David Siegel, Remote Possibilities: International Usability Testing at a Distance, Interactions, Mar.-Apr. 2004

Bolt | Peters Remote Testing Methodology Outline nitty-gritty of what they do for clients; not quite a how-to for researchers, but excellent outline of out to proceed.

Remote Online Usability Testing: Why, How, and When to Use It by Dabney Gough and Holly Phillips

IBM, Experience remote usability testing, Part 1


Tools to check out if you have the time/need

remote testing slides

Sept 18

Developing evaluation criteria through competitive analysis and expert inspection

Includes heuristics

Mobile usability (moved from 9/16)

Read at least one of the following:

Optional helpful tips: 10 usability tips & tricks for testing mobile applications

Competitive/competitor analysis: (a simple but powerful method)

Kuniavsky, ch 14

Some examples -- note that these range from a simple chart to a lengthy report. For your projects, you're more likely to do a chart:

mobile usability slides

competitive analysis slides

heuristics assignment


Sept. 23


Discussion of usability testing exercise

In-class heuristics exercise

Heuristics (a classic method, and Nielsen's heuristics are something everyone knows and refers to, whether they use them or not):

Muller, Matheson, Page, and Gallup, Participatory Heuristic Evaluation, Interactions, Sep.-Oct., 1998

Nielsen, Jakob. How to conduct a heuristic evaluationand Severity Ratings for Usability Problems

Nielsen's 10 Usability Heuristics The classics; everyone knows these, and many still use them. Here's Instone's piece on how to apply them to the web -- again, oldie but goodie.

Seven usability guidelines for websites on mobile devices An example of heuristics for a special situation; in this case, for websites on mobile phones. We're not saying these are the best; just good examples.

First principles, AskTog

International standards for HCI and usability

Optional, but recommended

Usability test due

heuristics slides


Sept 25


Kuniavsky, ch 6, p. 117-127 as an overview

Robert S. Weiss, Learning from Strangers - The Art and Method of Qualitative Interview Studies, New York: The Free Press, 1994
Chapter 3 - Preparation for Interviewing
Chapter 4 - Interviewing - Part 1
Chapter 4 - Interviewing - Part 2 - Examples of Interviewing
Chapter 5 - Issues in Interviewing

Interviewing slides


  Sept 30 Interviewing, cont.  

Oct. 2

Other ways of collecting data from people

Diary studies

Experience sampling


Interviews and Focus Groups

Krueger, Richard A., Focus Groups, 2nd ed. Sage, 1994; "Asking questions in a focus group." pp. 53-69 Useful in other situtations, not just in focus groups.

Rosenbaum, et. al., Focus Groups in HCI: Wealth of Information or Waste of Resources, CHI 2002

Gaver, B., Dunne, T., and Pacenti, E. 1999. Design: Cultural probes. interactions 6, 1 (Jan. 1999), 21-29. DOI=

Experience sampling

Sunny Consolvo, Miriam Walker Using the Experience Sampling Method to Evaluate Ubicomp Applications Pervasive Computing April-June 2003 (Vol. 2, No. 2) pp. 24-31

Diary Studies

Kuniavsky, pp. 369-385

Arbitron Diary Study Interesting to see how they do a paper diary and why. For radio ratings. Includes a copy of their form.

Carter, S. and Mankoff, J. 2005. When participants do the capturing: the role of media in diary studies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 - 07, 2005). CHI '05. ACM, New York, NY, 899-908. DOI= Useful for their discussion of using media other than paper. Optional, but recommended

interviewing assn

ESM slides

diary study slides


Oct 7

Understanding context through observation


Blomberg, J., Burrell, M., Guest, G. An ethnographic approach to design. In Jacko J. A., Sears A. (eds.). The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. Lawrence Erlbaum Associates, Inc.: Mahwah, New Jersey, 2003.

David R. Millen. Rapid ethnography: time deepening strategies for HCI field research. Conference proceedings on Designing interactive systems : processes, practices, methods, and techniques. ACM, 2000.

Hugh Beyer and Karen Holtzblatt, Contextual Design, San Francisco; Morgan-Kaufman, 1998
Chapter 1 - Introduction

Highly recommended:

  • Paul Dourish, "Implications for Design," CHI 2006
  • Diana Forsythe (2001). "It's just a matter of common sense": Ethnography as invisible work. In Studying those who study us: An anthropologist in the world of artificial intelligence (pp. 146-162). Stanford, CA: Stanford University Press. Originally pub'd in CSCW 1998 v20 no. 1 p 6-11

observation assignment

observation slides



Oct 9

Observation/ethnography continued

Qualitative data analysis

In-class exercise with the data from your diary study (updated 10/7) .Come prepared to play with your data!

diary study reflection due in class

qualitative data


Oct 14

Qualitative data analysis continued

Working with interview data.


  • Color printouts (if possible, black-and-white otherwise) of the diary study photos from Flickr at the default size, with titles attached. Try to print them out so that each photo and its title are on the same page. That way, we will have a standard format for working with photos in groups.
  • Your completed diary study, which we will also be working with

Interview assn due


Oct 16

Final projects

Continue discussion of qualitative data analysis

Project update and class discussion about research problem-solving


Oct 21

Sampling issues and measurement

Any thorough text on sampling and survey sampling will do; here's an online source: Statistics: Power from Data from Statistics Canada. Read the entire chapter on Sampling Methods that starts here. These concepts underlie much of data collection for a variety of purposes.

Look at: -- summary of polls for this year's US presidential election. Note especially those that have qualifiers after the data:

  • MOE+- means "margin of error, plus or minus"
  • Registered voters vs adults
  • "half sample" - only half were asked this question

revised 10/13/08


Obervation assignment due

sampling slides


Oct 23


Intro to concepts and issues of surveys

Take the Pew internet typology survey

THEN look at the survey report and questionnaire and data -- we'll use as examples in class.

Kuniavsky, ch 11

Schneiderman, Section 4.4

Dillman,Don A. (2000) Mail and Internet Surveys: the Tailored Design Method, 2nd ed. Wiley. chs. 2-3; 5. The rest of the book is recommended.

Web-based surveys: Dillman, Don, and Dennis K. Bokwer. The web questionnaire challenge to survey methodologists.

Take the Pew internet typology survey

THEN look at thesurvey report and questionnaire and data -- we'll use as examples in class.

Useful resources -- sources for questions and examples of how to present data:

Survey resources from They often reproduce their questionnaires -- a source of pre-tested questions, if they ask questions close to what you need (e.g., demographics, internet use/experience). Their reports are good at presenting data statistically and graphically and analyzing it. UCLA Digital Futures Project -- more recent report not available online. Again, useful example, and possible source of questions. and Popular online survey services. Many allow some free use that may be enough for your projects.

Optional, but recommended

Dillman,Don A. and Leah Melani Christian, Survey Mode as a Source of Instability in Responses across Surveys, Field Methods, February 2005

revised 10/13/08

Survey slides


Oct 28

Survey design and analysis continued


Survey Assignment

survey slides 2


Oct 30

survey reporting



Nov 4

Representing quantitative data

Election day

Tufte, Edward. Visualisation of Quantitative Information, Ch 46. Skim Ch 6.

Kass, Gary. Creating Good Charts: General Principles of Graphic Display.

Optional, but recommended

Reporting and general topics

Nov 6

Presenting findings and recommendations


Rashmi Sinha, Persona Development for Information-rich Domains, CHI 2003

Pruitt, John and Grudin, Jonathan. Personas: Practice and Theory. 2002.Kentaro Go and John M. Carroll, The Blind Men and the Elephant: Views of Scenario-Based System Design, Interactions, Nov.-Dec. 2004

Cooper, A. (1999). The inmates are running the asylum. Indianapolis: Sams. ch 11: Designing for people.

What is a scenario? from Information & Design.

Nokia Personas

Use cases from and from Wikipedia

Survey assn due



Nov 11



No class today


Nov 13

Qualitative data analysis and presentation





Nov 18

Reporting strategies


Kuniavsky, ch 17

AskTog, 2001: How to Deliver a Report Without Getting Lynched

Jarrrett, Caroline. Better Reports: How to Communicate the Results of Usability Testing

Common Industry Format report format (see about CIF).

Some examples: from Dialog Design


Nov 20 (tentative)

Universal usability

with guest speaker

Aaron Marcus, Universal, ubiquitous, user-interface design for the disabled and elderly, Interactions, Mar.-Apr. 2003

Schneiderman, Ben. Universal Usability. CACM 43:5 200. p. 84- 91.

Browse the site and the W3C web accessibility initiative sites

W3C introduction to web acccessibility

W3C Quick tips

WAI guidelines and techniques

Jacob Neilsen, International Web Usability Testing; Offshore Usability

Section 508 standards

Recommended: Evers, Vanessa. Cross-cultural applicability of user evaluation methods: a case study amongst Japanese, North-American, English and Dutch users, CHI '02



Nov 25

Careers in research
Guest speaker tbd (tenative)  

Nov 27




Dec 2

Project presentations
Project presentations  

Dec 4

Project presentations



Dec 9

Project presentations


Final project write-ups due Monday, Dec. 15 -- early submissions welcome.