Syllabus

Last revised 12/10/08 12:30 pm - broken links fixed
Skip to each month's assignments: October, November, December
Introduction
1

Aug 28

Intro: concepts and methods

What this course will cover

 

 

Pls fill out and email both of us this student info sheet

Slides

 

2b

Sept 4

User-centered design, iterative development, and planning
Methods overview

Kuniavsky, Chs.3 & 4

Courage & Baxter Chapter 3

Ben Shneiderman and Catherine Plaisant, Designing the User Interface : Strategies for Effective Human-Computer Interaction, New York; Addison-Wesley, 2004
Sections 1.1-1.4, Sections 3.1-3.4

Usability basics from Usability.gov

Usability.gov process map and planning guide

Usability.net methods table

Methods list Old, but still applicable

Please spend a lot of time familiarizing yourself with common methods.

Assn 1: Naive usability assessment due

slides

3a

Sept 9

Visit from Kevin Lim, MIMS 08

USERS

Users: their concept and roles; problematizing 'users'

Multiple stakeholders: beyond end-users

Problematics of multiple groups/needs

Ethics of working with users

Usability.gov overview learning about users

Kuniavsky Ch. 6 & 7

Oudshoorn, Nelly, and Pinch, Trevor. "How Users and Non-Users Matter," p. 1-16. In: Oudshoorn and Pinch, eds. How Users Matter: the Co-Construction of Users and Technology. MIT Press, 2003.

Hackos and Redish, Ch 2, Thinking about Users, Ch 3, Thinking about Tasks, Ch 4, Thinking about the Users' Environment

Skim these -- useful resources for ethics of working with users:

* Ethical Principles of Psychologists and Code of Conduct 2002; see especially sections on Privacy and Confidentiality.
* STC usability SIG on ethics especially the UPA Code of Professional Conduct (goes beyond working with users to usability work in general)
* UC Berkeley Committee for the Protection of Human Subjects - especially the guidelines on informed consent.

 

 

slides
Methods
3b

Sept 11

LAB-BASED USABILITY TESTING

and experimental design

Most critical:

Rubin, Jeffrey, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley, 1994 (Rubin) Specific readings Chapter 2. Chapter 3, Chapter 5, Chapter 7, Chapter 8, Chapter 10 (This book is worth buying)

Kuniavsky, Ch. 10

Optional: Babbie, Earl R. (2001) The Practice of Social Research, 9th ed., Wadsworth. Chapter 5: Conceptualization, operationalization, and measurement (or equivalent chapter from another social research methods text).

Robert Opaluch, "Usability Metrics". In Ratner, Julie, ed. Human Factors and Web Development, 2nd ed. Lawrence Erlbaum, 2003. p. 101-144.

Optional, but recommended

Assn 2: Usability Test

Testing Slides

 

4a

Sept 16

USABILITY TESTING, CONT.

Recruiting users

Non-lab based usability testing

Mobile (moved to 9/18)

Remote

Experience sampling (moved to 9/25)

Recruiting Users

Kuniavsky, ch 6, to p 117 (on recruiting users) (added 9/15)

Sample usability testing materials

Remote usability

Susan Dray and David Siegel, Remote Possibilities: International Usability Testing at a Distance, Interactions, Mar.-Apr. 2004

Bolt | Peters Remote Testing Methodology Outline nitty-gritty of what they do for clients; not quite a how-to for researchers, but excellent outline of out to proceed.

Remote Online Usability Testing: Why, How, and When to Use It by Dabney Gough and Holly Phillips

IBM, Experience remote usability testing, Part 1

 

Tools to check out if you have the time/need

remote testing slides
4b

Sept 18

Developing evaluation criteria through competitive analysis and expert inspection

Includes heuristics

Mobile usability (moved from 9/16)

Read at least one of the following:

Optional helpful tips: 10 usability tips & tricks for testing mobile applications

Competitive/competitor analysis: (a simple but powerful method)

Kuniavsky, ch 14

Some examples -- note that these range from a simple chart to a lengthy report. For your projects, you're more likely to do a chart:

mobile usability slides

competitive analysis slides

heuristics assignment

5a

Sept. 23

Heuristics

Discussion of usability testing exercise

In-class heuristics exercise

Heuristics (a classic method, and Nielsen's heuristics are something everyone knows and refers to, whether they use them or not):

Muller, Matheson, Page, and Gallup, Participatory Heuristic Evaluation, Interactions, Sep.-Oct., 1998

Nielsen, Jakob. How to conduct a heuristic evaluationand Severity Ratings for Usability Problems

Nielsen's 10 Usability Heuristics The classics; everyone knows these, and many still use them. Here's Instone's piece on how to apply them to the web -- again, oldie but goodie.

Seven usability guidelines for websites on mobile devices An example of heuristics for a special situation; in this case, for websites on mobile phones. We're not saying these are the best; just good examples.

First principles, AskTog

International standards for HCI and usability

Optional, but recommended

Usability test due

heuristics slides

5b

Sept 25

Interviews

Kuniavsky, ch 6, p. 117-127 as an overview

Robert S. Weiss, Learning from Strangers - The Art and Method of Qualitative Interview Studies, New York: The Free Press, 1994
Chapter 3 - Preparation for Interviewing
Chapter 4 - Interviewing - Part 1
Chapter 4 - Interviewing - Part 2 - Examples of Interviewing
Chapter 5 - Issues in Interviewing

Interviewing slides

 

  Sept 30 Interviewing, cont.  
6b

Oct. 2

Other ways of collecting data from people

Diary studies

Experience sampling

Probes

Interviews and Focus Groups

Krueger, Richard A., Focus Groups, 2nd ed. Sage, 1994; "Asking questions in a focus group." pp. 53-69 Useful in other situtations, not just in focus groups.

Rosenbaum, et. al., Focus Groups in HCI: Wealth of Information or Waste of Resources, CHI 2002

Gaver, B., Dunne, T., and Pacenti, E. 1999. Design: Cultural probes. interactions 6, 1 (Jan. 1999), 21-29. DOI= http://doi.acm.org/10.1145/291224.291235

Experience sampling

Sunny Consolvo, Miriam Walker Using the Experience Sampling Method to Evaluate Ubicomp Applications Pervasive Computing April-June 2003 (Vol. 2, No. 2) pp. 24-31

Diary Studies

Kuniavsky, pp. 369-385

Arbitron Diary Study Interesting to see how they do a paper diary and why. For radio ratings. Includes a copy of their form.

Carter, S. and Mankoff, J. 2005. When participants do the capturing: the role of media in diary studies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 - 07, 2005). CHI '05. ACM, New York, NY, 899-908. DOI= http://doi.acm.org/10.1145/1054972.1055098 Useful for their discussion of using media other than paper. Optional, but recommended

interviewing assn

ESM slides

diary study slides

7a

Oct 7

Understanding context through observation

 

Blomberg, J., Burrell, M., Guest, G. An ethnographic approach to design. In Jacko J. A., Sears A. (eds.). The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. Lawrence Erlbaum Associates, Inc.: Mahwah, New Jersey, 2003.

David R. Millen. Rapid ethnography: time deepening strategies for HCI field research. Conference proceedings on Designing interactive systems : processes, practices, methods, and techniques. ACM, 2000.

Hugh Beyer and Karen Holtzblatt, Contextual Design, San Francisco; Morgan-Kaufman, 1998
Chapter 1 - Introduction

Highly recommended:

  • Paul Dourish, "Implications for Design," CHI 2006
  • Diana Forsythe (2001). "It's just a matter of common sense": Ethnography as invisible work. In Studying those who study us: An anthropologist in the world of artificial intelligence (pp. 146-162). Stanford, CA: Stanford University Press. Originally pub'd in CSCW 1998 v20 no. 1 p 6-11

observation assignment

observation slides

 

7b

Oct 9

Observation/ethnography continued

Qualitative data analysis

In-class exercise with the data from your diary study (updated 10/7) .Come prepared to play with your data!

diary study reflection due in class

qualitative data

8a

Oct 14

Qualitative data analysis continued

Working with interview data.

TO BRING TO CLASS

  • Color printouts (if possible, black-and-white otherwise) of the diary study photos from Flickr at the default size, with titles attached. Try to print them out so that each photo and its title are on the same page. That way, we will have a standard format for working with photos in groups.
  • Your completed diary study, which we will also be working with

Interview assn due

8b

Oct 16

Final projects

Continue discussion of qualitative data analysis

Project update and class discussion about research problem-solving

 
9a

Oct 21

Sampling issues and measurement

Any thorough text on sampling and survey sampling will do; here's an online source: Statistics: Power from Data from Statistics Canada. Read the entire chapter on Sampling Methods that starts here. These concepts underlie much of data collection for a variety of purposes.

Look at:http://www.pollingreport.com/wh08.htm -- summary of polls for this year's US presidential election. Note especially those that have qualifiers after the data:

  • MOE+- means "margin of error, plus or minus"
  • Registered voters vs adults
  • "half sample" - only half were asked this question

revised 10/13/08

 

Obervation assignment due

sampling slides

9b

Oct 23

 

Intro to concepts and issues of surveys

Take the Pew internet typology survey

THEN look at the survey report and questionnaire and data -- we'll use as examples in class.

Kuniavsky, ch 11

Schneiderman, Section 4.4

Dillman,Don A. (2000) Mail and Internet Surveys: the Tailored Design Method, 2nd ed. Wiley. chs. 2-3; 5. The rest of the book is recommended.

Web-based surveys: Dillman, Don, and Dennis K. Bokwer. The web questionnaire challenge to survey methodologists.

Take the Pew internet typology survey

THEN look at thesurvey report and questionnaire and data -- we'll use as examples in class.

Useful resources -- sources for questions and examples of how to present data:


Survey resources from usabilitynet.org

http://www.pewinternet.org They often reproduce their questionnaires -- a source of pre-tested questions, if they ask questions close to what you need (e.g., demographics, internet use/experience). Their reports are good at presenting data statistically and graphically and analyzing it.

http://www.digitalcenter.org/downloads/DigitalFutureReport-Year4-2004.pdf UCLA Digital Futures Project -- more recent report not available online. Again, useful example, and possible source of questions.

Surveymonkey.com and zoomerang.com. Popular online survey services. Many allow some free use that may be enough for your projects.

Optional, but recommended

Dillman,Don A. and Leah Melani Christian, Survey Mode as a Source of Instability in Responses across Surveys, Field Methods, February 2005

revised 10/13/08

Survey slides

10a

Oct 28

Survey design and analysis continued

 

Survey Assignment

survey slides 2

10b

Oct 30

survey reporting

 

11a

Nov 4

Representing quantitative data

Election day

Tufte, Edward. Visualisation of Quantitative Information, Ch 46. Skim Ch 6.

Kass, Gary. Creating Good Charts: General Principles of Graphic Display.

Optional, but recommended

slides
Reporting and general topics
11b

Nov 6

Presenting findings and recommendations

POSTPONED TO NOV 13

Rashmi Sinha, Persona Development for Information-rich Domains, CHI 2003

Pruitt, John and Grudin, Jonathan. Personas: Practice and Theory. 2002.Kentaro Go and John M. Carroll, The Blind Men and the Elephant: Views of Scenario-Based System Design, Interactions, Nov.-Dec. 2004

Cooper, A. (1999). The inmates are running the asylum. Indianapolis: Sams. ch 11: Designing for people.

What is a scenario? from Information & Design.

Nokia Personas

Use cases from usability.gov and from Wikipedia

Survey assn due

slides

12a

Nov 11

VETERAN'S DAY

 

No class today

 
12b

Nov 13

Qualitative data analysis and presentation

POSTPONED FROM NOV 6

 

 

13a

Nov 18

Reporting strategies

 

Kuniavsky, ch 17

AskTog, 2001: How to Deliver a Report Without Getting Lynched

Jarrrett, Caroline. Better Reports: How to Communicate the Results of Usability Testing

Common Industry Format report format (see about CIF).

Some examples: from Dialog Design

 
13b

Nov 20 (tentative)

Universal usability

with guest speaker

Aaron Marcus, Universal, ubiquitous, user-interface design for the disabled and elderly, Interactions, Mar.-Apr. 2003

Schneiderman, Ben. Universal Usability. CACM 43:5 200. p. 84- 91.

Browse the universalusability.org site and the W3C web accessibility initiative sites

W3C introduction to web acccessibility

W3C Quick tips

WAI guidelines and techniques

Jacob Neilsen, International Web Usability Testing; Offshore Usability

Section 508 standards

Recommended: Evers, Vanessa. Cross-cultural applicability of user evaluation methods: a case study amongst Japanese, North-American, English and Dutch users, CHI '02

 

14a

Nov 25

Careers in research
Guest speaker tbd (tenative)  
14b

Nov 27

THANKSGIVING

 

15a

Dec 2

Project presentations
Project presentations  
15b

Dec 4

Project presentations

 

16a

Dec 9

Project presentations

 

Final project write-ups due Monday, Dec. 15 -- early submissions welcome.