SIMS 214 - Needs Assessment and Evaluation of Information Systems

This course will introduce methods and underlying concepts of needs assessment and evaluation for information systems. We  will follow two main threads: 

(1) Learning about and practicing some  major methods of needs and usability assessment (broadly defined). 

Current practice tends to focus much more on usability than needs assessment -- which can result in band-aid approaches to flawed designs, rather than incorporating considerations of use and users early in the design process.  We will investigate the major methods of both needs and usability assessment currently being practiced,  in the context of an iterative approach to design and assessment.

(2) Investigating, and possibly developing, appropriate methods of needs and usability assessment for information systems. 

Appropriate needs and usability assessment matches methods to conditions and purposes.  We will consider which methods and approaches are appropriate for various circumstances.

Most usability assessment is focused on interfaces. But, to be useful and used, an information system must have useful content and functionality as well as a usable interface. It must fit the work practices of the community being supported. One of our major concerns will be investigating the appropriate use of various methods for incorporating usability concerns into systems.

Much of our discussion will address web usability and digital libraries in particular. Digital libraries and other systems with a major information function differ from systems that are used for other purposes.  We will be particularly interested in how to choose and apply methods for such information or knowledge-intensive purposes. 

Underlying Orientations

(1) A broad focus on understanding people's purposes and practices in creating, finding, and using information.  

Information systems are used in contexts and for purposes; needs and usability assessment must be similiarly grounded.   

(2) A close partnership with and respect for users 

Work practices and tools (including information systems) are mutually-constituted. We can never entirely understand users' current practices.  Nor can we foresee how an information system, product, or service will be used and its effects.  We assume that the design and evaluation processes must respect the user's expertise; be grounded in the user's experience; continually consult with multiple points of view; and, wherever possible, include users in the design and evaluation process.

(3) High methodological standards

To be valid, reliable, and defensible, needs and usability assessment must be grounded in good social science research methods.  However, we also have to be aware of the constraints imposed by these methods, and the need to be flexible in terms of the time and resources required.

(4) Critical practice, that is, both learning about and taking a critical (i.e.,somewhat skeptical) stance toward our work

This is an area where much work is being done, some useful, some not.  Much is being said about user behavior and about design principles for the Web in particular.    Much of it is overly-narrow in its focus, and consists of de-contextualized pronouncements.   We will examine critically these discussions and the field of needs and usability assessment.