INFOSYS 214
Needs Assessment and
Evaluation of Information Systems
Mondays, 3:30-6, 202 South Hall
Admission: SIMS students, or permission
of instructor
Overview/wrap-up presentation (currently not working)
BUY -- Not in campus bookstores
Jan 25 Intro to the course
ASSIGNMENTS (subject
to change)
Assignment
1 Assignment
2 Assignment
3 In-depth
investigation Final
project
USEFUL WEB SITES
(we'll keep
updating this list)
NEW
http://www.upassoc.org
Usability Professionals'
Association. "A forum to promote usability concepts and techniques
worldwide."
http://www.software.ibm.com/ucd/index.html
IBM on user-centered design.
Jakob Nielsen's Alertbox:
http://www.useit.com/alertbox
One-stop shopping for Nielsen's
view of the (web) world.
UTEST Home Page: http://tigger.clemson.edu/utest/index.html
READINGS AND SCHEDULE
The order of topics is structured
to introduce early the methods that students are most likely to need to
use on final projects. The result is not conceptually clean, but practical.
Please note that the order of the readings was changed after they were sent to Copy Central, so that the order of the reader does not match the order of the syllabus. To find readings more easily, check the reader page numbers, listed in square brackets after the articles and chapters below.
1. January 25 - Introduction to the course
If you are not familiar with measurement, sampling, and/or social science survey research methods, read about them, early in the course, in any quantitative social science research methods text. We will put on informal reserve within South Hall the following:
Buttenfield, B. "Usability Evaluation of Digital Libraries." Science and Technology Libraries (forthcoming). Will be distributed in class.
Marchionini, Information Seeking in Electronic Environments, ch 3 [Reader pp. 5-38]
Optional, but highly recommended -- in the reader: Schamber, "Relevance and Information Behavior," Annual Review of Information Science and Technology 29 (1994) p. 3-48. [Reader pp. 39-62]
3. February
8 - Usability Testing
Nielsen, Usability Engineering, ch 6, "Usability Testing"
From Behavior and Information Technology 13: 1-2, Jan-April, 1994, Special issue on usability labs: [Reader pp. 389-427]
Blatt,
Louis, and others, "Designing and equipping a usability laboratory."
Faith,
Janet L. and others, "A practical guide to using software usability labs."
Bevan,
Nigel, and Miles MacLeod, "Usability measurement in context."
Rubin, Handbook of Usability Testing, Table of Contents (a good summary of steps of usability testing); ch. 2, "Overview of Usability Testing." [Reader pp. 363-388] This is an excellent resource and well worth buying if you are serious about usability testing. $45 in paper.
Also recommended: Joseph S. Dumas, Janice C. Redish. A Practical Guide to Usability Testing, Ablex, 1993.
Rubin and Dumas & Redish are on reserve in Main Library.
4. no
class February 15 - President's Day -- however, rather than lose a week's
work due to one day holiday, we'll cover more ground for our next class
5. February 22 -- Usability Inspection; Heuristic Evaluation; Cognitive Walk-Throughs
Nielsen & Mack, chs 1, 3, 5, 6
Nielsen UE ch 5
Nielsen & Mack UIM ch 2
Instone, Keith, "Site Usability Evaluation," [Reader pp. 429-432] http://webreview.com/wr/pub/97/10/10/usability/index.html
Instone, Keith. "Usability Heuristics for the Web." [Reader pp. 433-437] http://webreview.com/wr/pub/97/10/10/usability/sidebar.html
Browse: Nielsen's Alertbox, http://www.useit.com/alertbox/
Browse: webreview.com on usability. http://webreview.com/wr/pub/Usability.
Assignment 1, due March 1.
6. March 1 - Surveys and Interviews
Singleton, Straits, Straits, and McAllister. Approaches to Social Research. Oxford Univ Press, 1988. (on informal reserve within South Hall) ch 9, survey research; ch 10, survey instrumentation.
Marchionini, G., Plaisant, C., Komlodi, A. (May 1996). User needs assessment for the Library of Congress National Digital Library. CS-TR-3640, CAR-TR-829, CLIS-96-01. [Reader pp. 103-134] ftp://ftp.cs.umd.edu/pub/hcil/Reports-Abstracts-Bibliography/3640html/3640.html. Look also at the subject of their work: http://www.loc.gov/ammem/ - which also has a survey.
Nielsen on web surveys:
http://www.useit.com/alertbox/990110.html
Assignment
1 is due today.
Assignment
2 is about
today's content.
David L. Morgan, Focus Groups as Qualitative Research, Sage, 1988; p. 53-71.
Krueger, Richard A. Focus Groups, 2nd ed. Sage, 1994; "Asking questions," p. 53 -69. [Reader pp. 145-154]
Bishop, Ann Peterson. "Digital
Libraries and Knowledge Disaggregation: The Use of Journal Article Components."
DL98: Proceedings of the 3rd ACM International Conference on Digital
Libraries. New York, NY: ACM, 1998. [Reader pp. 155-167] http://anshar.grainger.uiuc.edu/dlisoc/socsci_site/conf-dl98-ann-knowl-disag.html
Optional: in the reader:
Hufnagel, Ellen M. and Christoper Conca. "User Response Data: the Potential
for Errors and Biases." Information Systems Research 5:1 (March
1994) p. 48-73. [Reader pp. 79-91]
Assignment
2 is due today.
These readings overlap somewhat but each offers something that the others don't.
Blomberg, Jeanette L. "Ethnography: aligning field studies of work and system design." in Monk, Andrew F., and G. Nigel Gilbert, eds., Perspectives on HCI: Diverse Approaches. ch 8, p. 173-197. Academic Press, 1995. [Reader pp. 207-218]
Blomberg, Jeanette, Jean Giacomi, Andrea Mosher, Pat Swenton-Wall, "Ethnographic Field Methods and their Relation to Design." Douglas Schuler and Aki Namioka, eds.. Participatory Design: Principles and Practices. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993; p. 123-155. [Reader pp. 219-251]
Brun-Cottan, Francoise and Patricia Wall (1995). "Using Video to Re-Present the User." Communications of the ACM 38 (5): 61-71. [Reader pp. 263-273]
Paul Dourish and Graham Button. 1996. "Technomethodology: Paradoxes and Possibilities." In Proc. ACM Conference on Human Factors in Computing Systems CHI'96 (Vancouver, Canada). New York: ACM. [not in reader] http://www.acm.org/sigchi/chi96/proceedings/papers/Button/jpd_txt.htm
Hughes, John, and others. "Ethnography in Interactive Systems Design." Interactions II.2 April 1995 p. 58-65. [Reader pp. 253-262]
Jordan, Brigitte, "Ethnographic
Workplace Studies and CSCW," Shapiro, Dan, and others, eds., The Design
of Computer-Supported Cooperative Work and Groupware Systems, Elsevier,
1996, ch 3, p. 17-42. [Reader pp. 275-300]
no class March 22 - Spring
Break
9. March 29 -- Task Analysis: Contextual Inquiry
K. Holtzblatt and S. Jones, "Contextual Inquiry: A Participatory Technique for System Design," Participatory Design: Principles and Practice. Aki Namioka and Doug Schuler (Eds.), Hillsdale, NJ: Lawrence Earlbaum Pub. 1993. [Reader pp. 169-186]
K. Holtzblatt and H. Beyer, "Representing Work for the Purpose of Design," Representations of Work, HICSS Monograph (Hawaii International Conference on System Sciences), January 1994. Lucy Suchman, Editor. [Reader pp. 187-205] http://www.incent.com/papers.indx/WorkModeling.html
For list of papers by these authors on Contextual Inquiry, see http://www.incent.com/Papers.html. [Note that Holtzblatt et al.'s web papers are bad examples of web documents! The paper itself doesn't include authors and source.] Their book: H. Beyer and K. Holtzblatt. 1998. Contextual Design: Defining Customer-Centered Systems. San Francisco, CA:Morgan Kaufmann Publishers, Inc.
10.
April 5 - Logs and Monitoring; Scenarios
Cooper, Michael D. "Design considerations in instrumenting and monitoring web-based information retrieval systems." JASIS 49 (10): 903-919, 1998. [Reader pp. 301-317] Can be accessed via http://www2.interscience.wiley.com/zcgi/zti
Buttenfield, Barbara, article in IJHCS, forthcoming. Will be distributed in class.
James E. Gaskin, User Monitoring
Drives Site Designs, Inter@ctive Week, December 14, 1998
http://www.zdnet.com/devhead/stories/articles/0,4413,2175392,00.html
Nielsen, Usability Engineering, p. 217-221.
Carroll, John, ed. Scenario-Based
Design 1995 John Wiley & Sons [Reader
pp. 319-350]
Kuutti,
ch 1
Kyng
l, ch 4
Carey
and Rusli, ch 7
Constantine, Larry L. "Essential
Modelling: Use Cases for User Interfaces." Interactions II.2,
April, 1995; p. 35-46. [Reader
pp. 351-362]
CHANGED
TOPIC AND READINGS FROM HERE TO END OF SEMESTER
Marchionini, Information Seeking in Electronic Environments, ch 3 [Reader pp. 5-38]
Schamber, "Relevance and Information Behavior," Annual Review of Information Science and Technology 29 (1994) p. 3-48. [Reader pp. 39-62] Read her pp. 23-36, reader pp. 50-56.
Dervin, Brenda. "An
Overview of Sense-making Research: Concepts, Methods, and Results To Date."
Presented at: International Communication Association Annual
Meeting, Dallas, Texas, USA, May l983. Author's note: "This 1983
presentation of the Sense-Making approach is now out of date but still
provides a foundation for interested readers." 72 pages but much
of that is appendices.
http://communication.sbs.ohio-state.edu/sense-making/art/artdervin83.htm
Articles about why trust and credibility are problems on the Web:
NY
Times 3/4/99
NY
Times 2/26/99
About how to evaluate web resources
Information
quality on the web
Harvard's
Widener Library on evaluating web resources
Practical
guide to evaluating info quality, from Johns Hopkins' Eisenhower Library
Nielsen's
AlertBox on trust and web design
April
19 - Assessing larger social impacts
trust, cont.
Due April 19: assignment
3 -- Will
be discussed in class.
Eugene Rochlin: Trapped
in the Net. Princeton Univ Press, 1997: chapters 1; ch 2 p. 46
(THe Interconnected Office to
end of chapter; 4; 7; and
12. http://pup.princeton.edu/books/
The rest of the book is
recommended; some of you may find the chapters on
financial trading interesting,
if you're interested in e-commerce.
(The chapters on the military
may be of particular interest just now when
you consider what NATO is
trying to do with international forces.)
Representing work:
Suchman, Lucy. Making work visible. CACM 38:9 (Sept. 1995) p. 56-64. [RPN 495 - 502]
Muller, Michael J. Ethnocritical
questions for working with translations, interpretation, and their stakeholders.
CACM 38:9 (Sept.
1995) p. 64-65. [Reader
pp. 503-504]
Bannon, Liam J. "The politics of design: representing work." CACM 38:9 (Sept. 1995) p. 66-68. [Reader pp. 505-507]
Collecting data in cyberspace:
Thomas, Jim. "Introduction:
a debate about ethics of fair practices for collecting social science data
in cyberspace." The Information
Society 12:2 (1996) p. 107-118.
[Reader pp. 523-533]
King, Storm A. "Researching
internet communities: proposed ethical guidelines for the reporting of
results." The Information Society
12:2 (1996) p. 119-128.
[Reader pp. 535-543]
May 10 -- Recap and synthesis.
Review of methods,
timeline of approach, who does usability; discuss course in general
and what we learned individually..
May
12 – Final deadline to turn in projects