IS 214: Needs Assessment and Evaluation of Information Systems

IS 214: Needs Assessment and Evaluation of Information Systems
Class Readings
Go to today's readings

Jan 17 Course Overview / Simple Usability Testing
Readings:
Laurel, ed., The Art of Human Computer Interface Design, Addison Wesley, 1990, p. 85-90, Gomoll, "Some Techniques for Observing Users"

Jan 24 What is Usability?
Readings:
Nielsen, Usability Engineering, Chapter 1, p. 1-16, Chapter 2, p. 23-42
Tognazinni, Tog on Software Design, Addison Wesley Developers Press, 1996, Chapter 25, p. 233-239, "Ease of Use"
Suggested Readings:
Shneiderman, Designing the User Interface, Chapter 1, p. 4-18
Usability Matters, Claire Rowland, WebReview.com, 3/10/2000. http://www.webreview.com/2000/03_10/
strategists/03_10_00_3.shtml
Usability Process
Readings:
Nielsen, Usability Engineering, Chapter 4
Vredenburg, Increasing Ease of Use, Communications of the ACM, May 1999, p.67-71
Suggested Readings:
Shneiderman, Designing the User Interface, Addison Wesley, 1998, Chapter 3, "Development Methodologies," p.104-107.
Microsoft Press, Windows User Experience, 1999, Chapter 2, p.30-39, "Design Methodology."

Jan 31 Usability Methods
Readings:
Preece, Jenny, ed. Guide to Usability: Human Factors in Computing, Addison Wesley, 1993; Chapter 6, "Evaluation."
Rosenbaum, Stephanie, "Not Just a Hammer: When and How to Use Multiple Methods in Usability Programs," Usability Professionals Association Conference Proceedings 2000.
Browse:
List of Usability Evaluation Methods and Techniques, http://www.cs.umd.edu/~zzj/UsabilityHome.html
The Usability Methods Toolbox, http://www.best.com/~jthom/usability/
Usable Web, http://usableweb.com/index.html

Usability Plan
Readings:
Rubin, Handbook of Usability Testing, p.81-118, Chapter 5, "Developing the Test Plan"

Feb 7 User Analysis and Recruiting
Readings:
Hackos, JoAnn T. and Janice C. Redish. User and Task Analysis for Interface Design. Wiley, 1998. Ch. 1 & 2.Rubin, Handbook of Usability Testing, p.119-139, Chapter 6, "Selecting and Acquiring Participants"
Suggested Readings:
Reeves & Nass, The Media Equation, Cambridge University Press, 1996, p.3-15, Chapter 1.
Tognazinni, Tog on Software Design, Addison Wesley Developers Press, 1996, p.215-219, Chapter 22, "Two Masters, or The Bifurcated Customer."
Users Matter, Claire Rowland, WebReview.com, 3/10/2000. http://www.webreview.com/2000/
05_26/strategists/05_26_00_1.shtml

Feb 14 Task Analysis / Scenarios
Readings:
Hackos, JoAnn T. and Janice C.Redish. User and Task Analysis for Interface Design. Wiley, 1998. Chapter 3 and p. 322-9.
Buur and Bagger, "Replacing Usability Testing with User Dialogue," Communications of the ACM, May 1999, p.63-66
Gardner, "Strengthening the Focus on Users' Working Practices," Communications of the ACM, May 1999, p.79-82
Mikkelson, Norrun, and Lee, Wai On, "Incorporating User Archetypes into Scenario-Based Design" Usability Professionals Association Conference Proceedings 2000.
Suggested Readings:
Fuccella, Jeanette and Jack Pizzolato, Finding out what users want from your Web site: Techniques for gathering requirements and tasks. http://www-106.ibm.com/
developerworks/library/moderator-guide/
requirements.html#scenario
Blomberg, Jeanette, Jean Giacomi, Andrea Mosher, Pat Swenton-Wall, "Ethnographic Field Methods and their Relation to Design." Douglas Schuler and Aki Namioka, eds. Participatory Design: Principles and Practices. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993; p. 123-155.

Feb 21 Focus Groups
Readings:
Nielsen, Usability Engineering, Chapter 1, p.214-217
Krueger, Richard A. Focus Groups, 2nd ed. Sage, 1994; "Asking questions" p. 53-69.
Suggested Readings:
Nielsen, Jakob (1997) The Use and Misuse of Focus Groups. http://www.useit.com/papers/focusgroups.html
Nielsen, Jakob (1999) Voodoo Usability http://www.useit.com/alertbox/991212.html
Fuccella, Jeanette and Jack Pizzolato, Creating Web Site Designs Based on User Expectations and Feedback. ITG Newsletter 1.1 June 1998 http://www.InternetTG.org/newsletter/june98/web_design.html
Surveys
Readings:
Shneiderman, Designing the User Interface, Addison Wesley, 1998, Chapter 4, "Surveys," p.132-145.
WebSurveyor, Survey Tips, "Survey Design Considerations" and "How Do I Conduct a Web Survey," 2000, http://www.websurveyor.com/learn_whitepapers.asp
Suggested Readings:
Singleton, Straits, Straits, and McAllister. Approaches to Social Research. Oxford Univ Press, 1988. ch 9, survey research; ch 10, survey instrumentation. Informal reserve in Computer Lab.
Nielsen on web surveys -- "Collecting Feedback from Users of an Archive." http://www.useit.com/alertbox/990110.html
Schmidt, William C. "World Wide Web Survey Research: Benefits, Potential Problems, Solutions." Behavior Research Methods, Instruments, and Computers 29 (2) 1997 p. 274-9.
Survey Resources from previous classes: http://www.sims.berkeley.edu/courses/is214/s00/resources.html

Feb 28 Contextual Inquiry and Design
Readings:
H. Beyer and K. Holtzblatt. Contextual Design: Defining Customer-Centered Systems, 1998, San Francisco, CA: Morgan Kaufmann Publishers, Inc. Chapter 3, "Principles of Contextual Inquiry" Chapter 6, "Work Models"

Mar 7 Design Exploration
Readings:
Rubin, Jeffrey, Handbook of Usability Testing, Wiley, 1994, "Exploratory Test," p.21-22, 31-37, 42-44
Hakim, Jack, and Spitzer, Tom, "Prototyping for Usability" Usability Professionals Association Conference Proceedings 2000.
User Inteface Engineering Paper, "Using Paper Prototypes to Manage Risk," http://world.std.com/~uieweb/paper.htm
Competitive Evaluation
Readings:
Nielsen, Jakob, "Test Your Competition" http://www.zdnet.com/devhead/stories/
articles/0,4413,2167696,00.html
Li, Jin, "Goal-Directed Competitive Analysis: Evaluating the User Interface in the Real World," Usability Professionals Association Conference Proceedings 2000.

Mar 14 Inspections
Readings:
Nielsen and Mack, "Usability Inspection Methods," p.170-181 in Baecker, Ronald M. et al ed., Human-Computer Interaction: Toward the Year 2000, 1995, Morgan Kaufman
Nielsen and Mack, Usability Inspection Methods, Wiley and Sons, 1994, Chapter 12, "Observing, Predicting, and Analyzing Usability Problems," p.295-339
Suggested Readings:
Nielsen and Mack, Usability Inspection Methods, Wiley and Sons, 1994, Chapter 12, "The Cognitive Walkthrough Method: A Practitioner's Guide." p.105-140
Heuristic Evaluation
Readings:
Nielsen, Usability Engineering, Chapter 1, p.20, Chapter 5, p.115-163
Interface Hall of Shame, http://www.iarchitect.com/mshame.htm
Suggested Readings:
Instone, Keith, "Site Usability Evaluation," webreview.com
Instone, Keith, "Usability Heuristics for the Web," webreview.com Microsoft Press, Windows User Experience, 1999, Chapter 2, p.25-31, "Design Methodology."

Mar 21 Usability Testing - Planning and Conducting
Readings:
Rubin, Handbook of Usability Testing, p.25-46, 67-73, 83-95, 179-182, 214-224, If you have time, read all of Chapters 4, 5, 7, & 8.
Bevin, Nigel, "ISO and industry standards for usability measurement," Usability Professionals Association Conference Proceedings 2000.
Suggested Readings:
Nielsen, Usability Engineering, Chapter 6
Julie Ratner et al, "Fun and Fast Testing Methods for Web Navigation," Usability Professionals Association Conference Proceedings 2000.Blatt, Louis, and others, "Designing and equipping a usability laboratory" Behavior and Information Technology 13: 1-2, Jan-April, 1994

Mar 27 Spring Break

Apr 4 Usability Testing - Analyzing and Reporting
Readings:
Rubin, Handbook of Usability Testing, Chapter 10, "Transforming Data into Findings and Recommendations"
Chatelain, Julianne, Sample Test Report, Usability of "Good Documents," http://www.gooddocuments.com/usabilitytest/ut1_home.htm
Suggested Readings:
Spool, Jared, et al, Web Site Usability: A Designer's Guide, Morgan Kaufman, 1997. Results of usability tests of web sites. p.3-47. See also the annotated table of contents: http://world.std.com/~uieweb/booktoc.htm
Comparative Usability Evaluation, http://www.dialogdesign.dk/cue.html

Apr 11 Guidelines
Readings:
Microsoft Windows User Experience, http://msdn.microsoft.com/library/books/
winguide/welcome.htm
Sun Microsystems Java Look and Feel Design Guidelines, http://java.sun.com/products/jlf/dg/index.htm
Lynch and Horton,Yale Web Style Guide http://info.med.yale.edu/caim/manual/
Accessibility
Readings:
Shneiderman, Designing the User Interface, Chapter 1, p.18-27, "Accommodation of Human Diversity"
Usability Interface, October 1999, Vol. 6, No. 1 - Usability and Accessibility Working Together: People with Disabilities and Computer Technology by Sheryl Burgstahler Web Accessessiblity Initiative by Whitney Quesenbery
Accommodating Color Blindness by Paul Hoffman

Browse:
IBM Accessibility Center Guidelines http://www-3.ibm.com/able/guidelines.htm
Web Accessibility Initiative - Guidelines and Checklists - http://www.w3.org/WAI/gettingstarted
Web Content Accessibility Guidelines 1.0 http://www.w3.org/TR/WAI-WEBCONTENT/
This document also appears in Interactions July-Aug 2001 p. 35-54

Apr 18 Usability Automation Tools
Readings:
Nielsen, Usability Engineering, p. 217-221.
Etgen, Michael, and Cantor, Judy, "A Comparison of Two Usability Testing Methods: Formal Usability Testing and Automated Usability Logging," Usability Professionals Association Conference Proceedings 2000.
James E. Gaskin, "User Monitoring Drives Site Designs," Inter@ctive Week, December 14, 1998 http://www.zdnet.com/devhead/
stories/articles/0,4413,2175392,00.html
US Government Bulletin on Cookies, http://www.ciac.org/
ciac/bulletins/i-034.shtml
Suggested Readings:
Bill Winnet, Tracking Your Visitors, Webmonkey, Apr. 1998 http://hotwired.lycos.com/webmonkey/
98/16/index2a.html?tw=e-business
IBM UCD Workbench, http://www-3.ibm.com/ibm/easy/eou_ext.nsf/publish/27
Thau!, Advanced JavaScript Tutorial - Cookie Introduction, Webmonkey, http://hotwired.lycos.com/webmonkey/
98/29/index1a_page7.html
Ethics
Readings:
Oliver K. Burmeister, "HCI Professionalism: Ethical concerns in Usability Engineering" & "Usability Testing: Revisiting Informed Consent procedures for testing Internet sites," Australian Institute of Computer Ethics Conference, 11/12/2000, http://www.aice.swin.edu.au/
TRUSTe Model Privacy Statement http://www.truste.org/webpublishers/pub_resourceguide.html

Apr 25 Managing Usability in an Organization
Readings:
Rubin, Handbook of Usability Testing, Chapter 11, "Establishing A Usability Program: Strategies and Tactics"
Mayhew, Deborah J., The Usability Engineering Lifecycle, Chapter 18, "Promoting and Implementing the Lifecycle," Morgan Kaufman, 1999

May 2 Final Projects

*IS 214: Needs Assessment and Evaluation of Information Systems* Class Readings Go to today's readings
Jan 17		Course Overview / Simple Usability Testing Readings: Laurel, ed., The Art of Human Computer Interface Design, Addison Wesley, 1990, p. 85-90, Gomoll, "Some Techniques for Observing Users"
Jan 24		What is Usability? Readings: Nielsen, Usability Engineering, Chapter 1, p. 1-16, Chapter 2, p. 23-42 Tognazinni, Tog on Software Design, Addison Wesley Developers Press, 1996, Chapter 25, p. 233-239, "Ease of Use" Suggested Readings: Shneiderman, Designing the User Interface, Chapter 1, p. 4-18 Usability Matters, Claire Rowland, WebReview.com, 3/10/2000. http://www.webreview.com/2000/03_10/ strategists/03_10_00_3.shtml Usability Process Readings: Nielsen, Usability Engineering, Chapter 4 Vredenburg, Increasing Ease of Use, Communications of the ACM, May 1999, p.67-71 Suggested Readings: Shneiderman, Designing the User Interface, Addison Wesley, 1998, Chapter 3, "Development Methodologies," p.104-107. Microsoft Press, Windows User Experience, 1999, Chapter 2, p.30-39, "Design Methodology."
Jan 31		Usability Methods Readings: Preece, Jenny, ed. Guide to Usability: Human Factors in Computing, Addison Wesley, 1993; Chapter 6, "Evaluation." Rosenbaum, Stephanie, "Not Just a Hammer: When and How to Use Multiple Methods in Usability Programs," Usability Professionals Association Conference Proceedings 2000. Browse: List of Usability Evaluation Methods and Techniques, http://www.cs.umd.edu/~zzj/UsabilityHome.html The Usability Methods Toolbox, http://www.best.com/~jthom/usability/ Usable Web, http://usableweb.com/index.html Usability Plan Readings: Rubin, Handbook of Usability Testing, p.81-118, Chapter 5, "Developing the Test Plan"
Feb 7		User Analysis and Recruiting Readings: Hackos, JoAnn T. and Janice C. Redish. User and Task Analysis for Interface Design. Wiley, 1998. Ch. 1 & 2.Rubin, Handbook of Usability Testing, p.119-139, Chapter 6, "Selecting and Acquiring Participants" Suggested Readings: Reeves & Nass, The Media Equation, Cambridge University Press, 1996, p.3-15, Chapter 1. Tognazinni, Tog on Software Design, Addison Wesley Developers Press, 1996, p.215-219, Chapter 22, "Two Masters, or The Bifurcated Customer." Users Matter, Claire Rowland, WebReview.com, 3/10/2000. http://www.webreview.com/2000/ 05_26/strategists/05_26_00_1.shtml
Feb 14		Task Analysis / Scenarios Readings: Hackos, JoAnn T. and Janice C.Redish. User and Task Analysis for Interface Design. Wiley, 1998. Chapter 3 and p. 322-9. Buur and Bagger, "Replacing Usability Testing with User Dialogue," Communications of the ACM, May 1999, p.63-66 Gardner, "Strengthening the Focus on Users' Working Practices," Communications of the ACM, May 1999, p.79-82 Mikkelson, Norrun, and Lee, Wai On, "Incorporating User Archetypes into Scenario-Based Design" Usability Professionals Association Conference Proceedings 2000. Suggested Readings: Fuccella, Jeanette and Jack Pizzolato, Finding out what users want from your Web site: Techniques for gathering requirements and tasks. http://www-106.ibm.com/ developerworks/library/moderator-guide/ requirements.html#scenario Blomberg, Jeanette, Jean Giacomi, Andrea Mosher, Pat Swenton-Wall, "Ethnographic Field Methods and their Relation to Design." Douglas Schuler and Aki Namioka, eds. Participatory Design: Principles and Practices. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993; p. 123-155.
Feb 21		Focus Groups Readings: Nielsen, Usability Engineering, Chapter 1, p.214-217 Krueger, Richard A. Focus Groups, 2nd ed. Sage, 1994; "Asking questions" p. 53-69. Suggested Readings: Nielsen, Jakob (1997) The Use and Misuse of Focus Groups. http://www.useit.com/papers/focusgroups.html Nielsen, Jakob (1999) Voodoo Usability http://www.useit.com/alertbox/991212.html Fuccella, Jeanette and Jack Pizzolato, Creating Web Site Designs Based on User Expectations and Feedback. ITG Newsletter 1.1 June 1998 http://www.InternetTG.org/newsletter/june98/web_design.html Surveys Readings: Shneiderman, Designing the User Interface, Addison Wesley, 1998, Chapter 4, "Surveys," p.132-145. WebSurveyor, Survey Tips, "Survey Design Considerations" and "How Do I Conduct a Web Survey," 2000, http://www.websurveyor.com/learn_whitepapers.asp Suggested Readings: Singleton, Straits, Straits, and McAllister. Approaches to Social Research. Oxford Univ Press, 1988. ch 9, survey research; ch 10, survey instrumentation. Informal reserve in Computer Lab. Nielsen on web surveys -- "Collecting Feedback from Users of an Archive." http://www.useit.com/alertbox/990110.html Schmidt, William C. "World Wide Web Survey Research: Benefits, Potential Problems, Solutions." Behavior Research Methods, Instruments, and Computers 29 (2) 1997 p. 274-9. Survey Resources from previous classes: http://www.sims.berkeley.edu/courses/is214/s00/resources.html
Feb 28		Contextual Inquiry and Design Readings: H. Beyer and K. Holtzblatt. Contextual Design: Defining Customer-Centered Systems, 1998, San Francisco, CA: Morgan Kaufmann Publishers, Inc. Chapter 3, "Principles of Contextual Inquiry" Chapter 6, "Work Models"
Mar 7		Design Exploration Readings: Rubin, Jeffrey, Handbook of Usability Testing, Wiley, 1994, "Exploratory Test," p.21-22, 31-37, 42-44 Hakim, Jack, and Spitzer, Tom, "Prototyping for Usability" Usability Professionals Association Conference Proceedings 2000. User Inteface Engineering Paper, "Using Paper Prototypes to Manage Risk," http://world.std.com/~uieweb/paper.htm Competitive Evaluation Readings: Nielsen, Jakob, "Test Your Competition" http://www.zdnet.com/devhead/stories/ articles/0,4413,2167696,00.html Li, Jin, "Goal-Directed Competitive Analysis: Evaluating the User Interface in the Real World," Usability Professionals Association Conference Proceedings 2000.
Mar 14		Inspections Readings: Nielsen and Mack, "Usability Inspection Methods," p.170-181 in Baecker, Ronald M. et al ed., Human-Computer Interaction: Toward the Year 2000, 1995, Morgan Kaufman Nielsen and Mack, Usability Inspection Methods, Wiley and Sons, 1994, Chapter 12, "Observing, Predicting, and Analyzing Usability Problems," p.295-339 Suggested Readings: Nielsen and Mack, Usability Inspection Methods, Wiley and Sons, 1994, Chapter 12, "The Cognitive Walkthrough Method: A Practitioner's Guide." p.105-140 Heuristic Evaluation Readings: Nielsen, Usability Engineering, Chapter 1, p.20, Chapter 5, p.115-163 Interface Hall of Shame, http://www.iarchitect.com/mshame.htm Suggested Readings: Instone, Keith, "Site Usability Evaluation," webreview.com Instone, Keith, "Usability Heuristics for the Web," webreview.com Microsoft Press, Windows User Experience, 1999, Chapter 2, p.25-31, "Design Methodology."
Mar 21		Usability Testing - Planning and Conducting Readings: Rubin, Handbook of Usability Testing, p.25-46, 67-73, 83-95, 179-182, 214-224, If you have time, read all of Chapters 4, 5, 7, & 8. Bevin, Nigel, "ISO and industry standards for usability measurement," Usability Professionals Association Conference Proceedings 2000. Suggested Readings: Nielsen, Usability Engineering, Chapter 6 Julie Ratner et al, "Fun and Fast Testing Methods for Web Navigation," Usability Professionals Association Conference Proceedings 2000.Blatt, Louis, and others, "Designing and equipping a usability laboratory" Behavior and Information Technology 13: 1-2, Jan-April, 1994
Mar 27		*Spring Break*
Apr 4		Usability Testing - Analyzing and Reporting Readings: Rubin, Handbook of Usability Testing, Chapter 10, "Transforming Data into Findings and Recommendations" Chatelain, Julianne, Sample Test Report, Usability of "Good Documents," http://www.gooddocuments.com/usabilitytest/ut1_home.htm Suggested Readings: Spool, Jared, et al, Web Site Usability: A Designer's Guide, Morgan Kaufman, 1997. Results of usability tests of web sites. p.3-47. See also the annotated table of contents: http://world.std.com/~uieweb/booktoc.htm Comparative Usability Evaluation, http://www.dialogdesign.dk/cue.html
Apr 11		Guidelines Readings: Microsoft Windows User Experience, http://msdn.microsoft.com/library/books/ winguide/welcome.htm Sun Microsystems Java Look and Feel Design Guidelines, http://java.sun.com/products/jlf/dg/index.htm Lynch and Horton,Yale Web Style Guide http://info.med.yale.edu/caim/manual/ Accessibility Readings: Shneiderman, Designing the User Interface, Chapter 1, p.18-27, "Accommodation of Human Diversity" Usability Interface, October 1999, Vol. 6, No. 1 - Usability and Accessibility Working Together: People with Disabilities and Computer Technology by Sheryl Burgstahler Web Accessessiblity Initiative by Whitney Quesenbery Accommodating Color Blindness by Paul Hoffman Browse: IBM Accessibility Center Guidelines http://www-3.ibm.com/able/guidelines.htm Web Accessibility Initiative - Guidelines and Checklists - http://www.w3.org/WAI/gettingstarted Web Content Accessibility Guidelines 1.0 http://www.w3.org/TR/WAI-WEBCONTENT/ This document also appears in Interactions July-Aug 2001 p. 35-54
Apr 18		Usability Automation Tools Readings: Nielsen, Usability Engineering, p. 217-221. Etgen, Michael, and Cantor, Judy, "A Comparison of Two Usability Testing Methods: Formal Usability Testing and Automated Usability Logging," Usability Professionals Association Conference Proceedings 2000. James E. Gaskin, "User Monitoring Drives Site Designs," Inter@ctive Week, December 14, 1998 http://www.zdnet.com/devhead/ stories/articles/0,4413,2175392,00.html US Government Bulletin on Cookies, http://www.ciac.org/ ciac/bulletins/i-034.shtml Suggested Readings: Bill Winnet, Tracking Your Visitors, Webmonkey, Apr. 1998 http://hotwired.lycos.com/webmonkey/ 98/16/index2a.html?tw=e-business IBM UCD Workbench, http://www-3.ibm.com/ibm/easy/eou_ext.nsf/publish/27 Thau!, Advanced JavaScript Tutorial - Cookie Introduction, Webmonkey, http://hotwired.lycos.com/webmonkey/ 98/29/index1a_page7.html Ethics Readings: Oliver K. Burmeister, "HCI Professionalism: Ethical concerns in Usability Engineering" & "Usability Testing: Revisiting Informed Consent procedures for testing Internet sites," Australian Institute of Computer Ethics Conference, 11/12/2000, http://www.aice.swin.edu.au/ TRUSTe Model Privacy Statement http://www.truste.org/webpublishers/pub_resourceguide.html
Apr 25		Managing Usability in an Organization Readings: Rubin, Handbook of Usability Testing, Chapter 11, "Establishing A Usability Program: Strategies and Tactics" Mayhew, Deborah J., The Usability Engineering Lifecycle, Chapter 18, "Promoting and Implementing the Lifecycle," Morgan Kaufman, 1999
May 2		Final Projects