In describing the course below, at the end of each section, I have
posed questions which students should be able to answer at the end
of the section. The questions are often posed as specific situations
that students might face in the course of research or while working
An introduction to various quantitative methods and the appropriate
of design and analysis:
- Posing a
the related concepts
the Independent, Dependent, and Controlled Variables
I have been asked to test the usability of web retail site. How
should I begin to test it? What aspects of the site should I test?
the testing method (experiment, observation, surveys etc.)
- What kind
of method is appropriate for the current situation?
- Choice of
testing method as a tradeoff between control and realism (often,
the more controlled the study is, the less realistic it is)
Quasi-Experimental and Non-Experimental Methods.
- Other aspect
of testing: Ethical issues in using human subjects.
Consent and Debriefing.
What testing method should I use? What are the pros and cons
- The art
of finding and recruiting participants (taking into account Random
Selection and Random Assignment)
- A practical
view of randomization: Randomization and Pseudo Randomization.
issues about sample size and statistical power.
a participant database.
How will I find participants for my studies who are representative
of the product's typical users? How should I recruit them and assign
them to various experimental conditions? How many people should
Analysis of data
- Basic statistics
- Levels of
measurement: nominal, ordinal, interval, and ratio
- Mean, median,
and non parametric statistics
mean differences, significance levels and what they mean
- Type 1 and
Type II errors
representation of data
of experimental designs
- Single Factor
of Experimental Error
of Treatment Effects
of the Null Hypothesis
of the factorial design
- The two
- Higher Order
Analysis for Higher Order Factorial Designs
- Simple Effects
- Other possible
designs: e.g., Latin Square Designs
company is redesigning their web-site to support faster navigation.
I have been asked to compare the old and new designs to find out
which supports faster navigation. What experiments can I run to
find this out?
of Non Experimental Studies
methods for analyzing correlational data
- Brief Introduction
to Factor Analysis, Cluster
Analysis and Multidimensional Scaling
- The design
of surveys and questionnaires
- How to frame
- Kinds of
scales: Likert, Semantic Differential etc.
need to design a questionnaire to test users' satisfaction with
the personalization features for our website. How should I go about
| Part II:
A closer look at some usability testing methods and Group
- What is
- The different
flavors of GOMS analysis.
- A keystroke
level GOMS analysis.
Registered users can personalize their stock portfolios on my company's
website. However, the method to create the personalized portfolio
takes too long and is prone to errors. I have been asked to try
to reduce the average time and errors in creating a portfolio. How
can I achieve this goal?
the mental models of users for designing large-scale websites.
Card Sorting and Similarity Ratings.
Analysis: Cluster Analysis and Multidimensional Scaling.
I have been asked to restructure a large consumer website. How should
I develop the product categories so that it will be easy for users
to find items on the site?
- How to find
out if there are significant individual differences between groups
- What kind
of individual differences variables might exist: demographic variables
such a age, sex etc. situational variables such as motivation,
level of interest, fatigue etc., cognitive variables such as memory,
cognitive style etc.
- How to analyze
existing data to identify individual differences?
- How to design
studies to test for individual differences?
- Item Response
I suspect that my company's product is heavily biased towards expert
users, How should I demonstrate this? I think that the women visitors
do not like a particular website. How can I find out if this hypothesis
Web Usability Evaluation
- What are
the various kinds of automated methods in web usability evaluation
(e.g., automatic capture of usage data, Log file analysis)?
- What kind
of usability problems can such methods help identify?
the Group Project
an interesting usability problem
the problem in terms of more specific usability questions
at least two method of testing, develop the testing method
and testing subjects
data, write research reports.
stage of the project, the groups will give short presentations. This
allows other groups to get exposure to the complete research process
for each testing method.