Group 3 - Method

California Digital Library

Interface Design Project

Low Fidelity Prototyping

Method

Participants

Three participants were selected to test our prototype. The script contained pre-test questions to collect information on their backgrounds and experience with CDL as well as with other library systems. All three were masters students (one first-year and two second-year) in the SIMS program. Their level of expertise with the CDL library system and with other online library resources at UC Berkeley ranged from virtually no previous exposure to significant but not expert-level knowledge. Further details are available in demographics.

Task Scenarios

#1. Imagine that you are a UC undergraduate student. You are using the California Digital Library for the first time. Rather than looking for a specific resource, you are browsing the system to determine what resources might be available related to your major, Art History. You are interested in both general Art History resources as well as in Renaissance Italian sculpture of the 16th century.

We looked to see if new participants would use an overview of site (Site Map) or a guide (User Guide) to begin. And, was our combination of the Search and Browse functions useful to them. Could they learn quickly how to narrow their searches?

#2. Imagine that you are a UC graduate student in the Genetics Department. You are looking for a specific journal article that a professor recommended. You don't remember the exact title or reference, but you know that the article was published within the past couple of months, is about mitochondria DNA in sea urchins, and is by somebody named Roberti. Please see if you can find this article.

This presupposed some prior knowledge of the system (that could be gained simply from performing the first scenario.) We looked to see if it was obvious where to find an entire (recently published) article in a journal.

#3. You are a faculty member in the History Department studying religious minorities in the 19th-century United States. You are currently working on a research paper about Jews in the American South during the Reconstruction period. It is important for you to stay up-to-date on related developments in your field, so you are using the CDL to look for recent related scholarly journal articles.

This scenario tested whether our immediate jumps into the system would work. And, whether using the Profile function was clear or presumed prior knowledge.

Procedure

There were three testers: Paper "Computer", Observer, and Test Facilitator (who also manipulated the real computer when necessary.)

We set up the paper "computer" (our prototype pages) on a table in front of a computer monitor. The computer monitor was loaded with two browser windows tiled side-by-side. The Leftside contained our bookmarks page (which simluate links out of the prototype to save time if applicable.) The Rightside contained the working results browser (beginning as a blank screen.)

The script welcomed them, a Consent Form was signed, and a set of Pre-Test Questions were read aloud to gather prelimineary background information on the participant.

The scenarios were first dictated in full by the Test Facilitator and then the participants were provided with a handout of the scenario particulars (reminders of what they were looking for.) They were instructed to use our initial Home, Search & Browse pages. When the "search" went beyond those pages, the Test Facilitator moved through the CDL site online as per the participant's "thinking out loud." (In some cases, the participant was allowed to use the mouse and move through the site on their own. All the while being prompted to think out loud.)

Each scenario was considered complete when the participant felt satisfied they had reached their goal. The Testers did not prompt them.

At the end of the third scenario, a page of Post-Test Questions was given to the participants to complete by hand. To answer Question #4 on the paper, they were also provided with a hardcopy of the existing CDL Home and Search/Browse pages for reference. In two cases, the answers to #4 were captured more fully on photocopies of the prototype pages by the Observer. These visual notes indicated some of the good and bad features of the interfaces as compared to the existing CDL UI.