California Digital Library
Assignment 9: Pilot User Study
APPENDIX A:
Materials used with the Participants
|
California Digital Library
|
[ Home | Introduction | Prototype | Method | Test Measures | Results | Discussion | Formal Design | Appendices ]
[top]We will be testing an interface to the California Digital Library. Before we begin, we have a brief Consent Form for you to sign.
[ Consent Form ]
This simply says that you understand the potential risks of this study – none – and the potential benefits – also none.
The California Digital Library has been created as a "tenth library of the UC System" and is accessible via the Web. Today we will be testing a revised interface for the top-level Web pages of the CDL site.
Before we begin our test, let me ask you a few brief questions about your background and experience with this and similar systems:
Now, let’s begin our test.
The re-designed screens of CDL are displayed infront of you. I will give you three tasks which will start from this CDL prototype. Some of the pages of this prototype are yet to be developed and these pages will appear as non-functional screens with explanatory blurbs for example: [example: What's New at CDL].We have tried to integrate our redesigned interface, where possible, with existing, lower-level pages of the current CDL.
You may now take a few minutes to get familiar with our prototype
[ wait a few minutes.... ]
Do you have any questions before we begin?
As I mentioned before, your tasks will begin with our CDL prototype. You may then continue to navigate the site until you complete your task.
_________________________________ will be observe and take notes.
Remember, we are not testing you – we are testing the interface. If anything is confusing or difficult, it is helpful for the designers to know that so they can improve the interface design.
We will be giving you three specific "tasks" or "goals" to try as you use the system. Each task should take about 5 or 10 minutes. Following the test, we will ask you a few questions about the system to help assist the designers in their work.
While you are using the system, we will ask you to "think aloud", by speaking both what you are doing and why; for example, "Now I’ll click on the triangle for finding aides because I think it will get me to a collection at the Bancroft Library." This will help us to know what actions you are taking and will help us judge how understandable the system is. "Thinking aloud" may seem unnatural at first, so if you forget, I will try to remind you to continue talking aloud.
If you want to take a break or discontinue the test at any point, you are certainly welcome.
Do you have any questions before we start?
Scenarios:
[read aloud, give handout of overviews to participants]
First: Imagine that you are a UC Berkeley student writing a comparative literature paper. You want to compare one of your class readings with Dr. Seuss' famous book Green Eggs and Ham. You want to find a copy of this book at UCB.
Second: Imagine that you are a UC [graduate student? faculty member?] studying 20th century theatre history. You are looking for a specific journal article that was recommended by a colleague; unfortunately, you are unable to remember the exact citation. You know that the article concerns the discovery of two "lost" puppet (marionette) plays from the Spanish Civil War (1936-39). The article was published in a leading English-language theatre research journal in late 1997 or early 1998 and was written by J McCarthy.
Please attempt to identify this article. If it is possible, you would like to obtain the full text of the article on-line; if not, you would like to identify libraries where you could view the physical journal.
Third: You are a UC Berkeley graduate student in the School of Information Management and Systems. You are expected to write a report on the National Information Infrastructure (NII). You want to find out about the history of NII and where it is headed. Your arguments about the future must be supported by past and current trends seen in the information industry. You have already browsed the WWW and now you want to see if there are any published materials in this area.
Please identify a few (top level) CDL sources that might be relevant to your research on this topic. Then choose a couple of these sources and delve deeper to find specific resources and papers on the history and development of the NII.
Before we finish, we have few questions to ask you about your experience with this system:
[top]We are a group of graduate students in the School of Information Management and Systems at UC Berkeley who are conducting a study to test on a redesigned interface for the California Digital Library.
If you agree to participate, you will be asked to use a mock-up of a Web site to search for informational resources. You will then be asked a few follow-up questions.
This study will have no direct benefits to you; it will also have no direct risks.
All personal information from the study will be kept completely confidential to the extent permissible by law. You will be represented by an ID number and will not be identified in any reports or publications.
Please read and sign the following statement:
I have read the above terms and agree to participate in this study.
Sign: _________________________________________
Date: _________________________________________
Participant # _________________________
(read and completed by interviewer)
Participant #________
1. In what year of your program are you?
2. Have you ever used CDL before?
And, how often do you use it?
- If not, have you ever heard of it?
3. Have you ever used:
4. What types of library resources do you usually search for—journals, books, online material, archives, etc.?
[top]Scenario #1: Comp Lit student
Find: the book entitled Green Eggs and Ham by Dr. Seuss at UCB
Scenario #2: UC faculty member studying 20th century theatre history
Citation of the recommended article:
Topic: discovery of two "lost" puppet (marionette) plays from the Spanish Civil War (1936-39)
Journal: leading English-language theatre research journal ?
Author: J. McCarthy
Date: late 1997 or early 1998 ?
Scenario #3: SIMS student
Find information on the history and current trends of the National Information Infrastructure (NII).
First: Identify a few relevant top level sources
Second: Choose a couple of these sources and find specific resources and papers on the history and development of the NII.
[top]Participant #_________
1. Given that our UI is in a rough design phase, what is your general impression of the User Interface presented to you today?
2. Did you experience any frustrations when navigating through our site?
3. Did you feel like you learned about the scope of the CDL as you navigated through the site?
4. Are there any parts of our CDL interface you feel can use improvement? if so, where and how?
[ Home | Introduction | Prototype | Method | Test Measures | Results | Discussion | Formal Design | Appendices ]