Appendix D � Critical Incident Logs


Participant 1, April 21, 1999, 6th year history graduate student

Summary

� Link names within the Meet the Suffragist page [and throughout the site]
� Blue triangles (in Dynaweb) are confusing
� In search results, doesn�t see the arrows (surrounding the results in the text) that link to the next hit
� Need complete book citation
� Might want to break down Bibliography by subject
 

Details

User starts at the Home page and clicks on the About this Site link (�because it�s blue�) [Other links are pink for some reason.]

He reads the table, and doesn�t realize at first that it refers to the left frame. He�s not sure what he should do, and expects there to be links in the table. He realizes fairly quickly, however, that the table is just an explanation of the frame.

He wants to get an introduction to the site, so he clicks on home [he seems a bit surprised at where that takes him]. He seems to want the intro paragraph, but doesn�t read the intro on the Home page underneath the image and introductory links. He clicks on Meet the Suffragists. He likes the page, says it looks nice. He asks about linking names to other suffragists on the page.

He goes to the Alice Paul oral history. When asked about what he sees, he understands the TOC in the left frame and the sub-topics, but asks what the blue triangles are for (says he would�ve figured it out). On the oral history, he looks for who the interviewer is, but doesn�t find it. He asks about the format of the document and whether it comes from the Bancroft.

He wants to go back to where he selected Alice Paul (the Meet the Suffragists page) but doesn�t know how to get there. He�s told to click on the Oral History link on the left frame, but this brings him to the oral history opening page, which just has photographs and titles.

He says he�s mainly interested in primary sources. He clicks on the oral history link for Sara Bard Field and says he would look for anarchism or Emma Goldman (for his own research purposes). He asks about Search. We tell him to go to the search page by clicking on the Search link in the left frame. He enters Alice Paul (?? �I think) and gets 26 hits on oral histories. He asks which oral history, and says �great.�

SCENARIO 1: (Jane Addams)
He goes to the search page and types Jane Addams in the full feature search with no limits. He gets 154 hits (�wow�), which he feels is a lot. He clicks on the oral history links and understands that the red number is the number of mentions within the document. He tries to see in what type of context Addams appears (just a passing reference or real information). He says he would go back to the previous screen and would scroll down looking for more red-highlighted Addams. We explain the arrows next to the hits (he doesn�t see them on his own), and he uses one to go to the next hit. He sees that he�s in another section. He notes that it will say �2 hits� even if it�s just �Jane Addams� because it�s searching for both names. He would go through all of the search results (speeches and secondary sources-books). He notes that he likes speeches as a resource because it�s very direct (as opposed to the reminiscences of oral histories). When he does click on the Secondary Source book search results, he is surprised at what he finds because he thought he would get something from the bibliography [mismatch between left navigation bar and contents??]. What he gets is a chapter of the book, but the complete citation is missing. He says he feels he�d get everything he�d need by doing a search (�Is that right?�).

SCENARIO 2: International Women
Says that he needs a section that talks about England. He goes to Home, then clicks on Meet the Suffragists (�Not what I want.�) Then goes back to home and questions Intro to the Era (is doubtful because it sounds too general), but clicks on it anyway. He scans for the word England or Great Britain or UK. He then goes to Search, and thinks about searching for Suffragette (what Suffragists were called in England) or England. He picks Suffragette, and gets 12 hits on oral histories (Alice Paul, specifically). He goes to the next hit using the left frame contents and sees Great Britain! He then sees the name Pankhurst, which is what he was looking for. When asked what he would do after getting the name, he says he would do a search on the name.

SCENARIO 3: Alice Paul, 1917
Questions for a moment how to get to the Alice Paul bio (which he remembers seeing from browsing). Goes to Meet the Suffragists page from the Home page, then to Intro to the Era, where he would scan for 1917. He says he would then go to Alice Paul�s oral history because he would like to get her specific words. To find 1917, he would look for contents items about the US entry into WWI. He goes to the Search page and says he would search for �National Women�s Party� or �World War I� or �White House�. His search for National Women�s Party yields 182 hits, of course not just Alice Paul, but fortuitously, Alice Paul�s is the first link. He goes to the Alice Paul oral history and questions how he would get to 1917 (unclear). He searches on World War I, which also brings up the Alice Paul oral history first. Unfortunately, World War II is mistakenly included in the results.

Post-study questions:
Timeline: why didn�t he use it? He didn�t think it would be useful. He looks at the timelines and likes them, but asks why one heading is purple and notes that you lose the headings when you scroll down.

Bibliography: he thinks it might be useful to break down into topics.

Overall: useful site, varied sources are good. He thinks that the site provides creative options for teachers, and feels that he could come up with interesting course assignments from the sources on the site.
 


Participant 2, April 22, 1999, 6th year history graduate student (was one of our low-fi testers)

Summary

� Questioning hiding timelines (using a large screen)
� Wants annotation of contents: How much is here? Where does it come from?
� Likes division of Primary and Secondary sources
� In search results, doesn�t like being brought to the middle of a primary document
� Need to add complete citation to the Secondary Source book pages
� Add a note to the periodicals page that explains that theses are �selected� periodicals
 

Details

From the home page, she clicks on About this Site and says that she always starts by going to a site�s guide to get an overview of the collection. She scrolls all the way down (even to the acknowledgements section). She goes to the left frame and clicks on Show Timelines. She clicks on 1910 and looks at the content. [We suggest that she click on 1913 for a year that has more information.] She says she�s more interested in the Suffragist column, and doesn�t think the Literature and Arts information is so important.

She clicks on Photographs and says that she likes the ability to see larger images. She clicks on the browser�s back button.

She asks �why would you ever want to hide the timeline?�

She clicks on Correspondence and goes to the first letter. She wants to know how much is here (on the site) and where it comes from. She asks where she would find letters to (from?) Alice Paul. We point out the Sort by Author option.

She clicks on Songs and listens to a song. She then clicks on Interviews and listens to an excerpt. She says it�s great to have a voice with it; that it�s a unique contribution to have it all in one place.

She clicks on Speeches and looks at the speech.

She likes the division of Primary and Secondary sources. She mentions that she likes the left frame rather than the pull down menus she saw during the low-fi prototype test.

SCENARIO 1 (Women reformers and Jane Addams):
She first goes to Oral Histories but doesn�t look there for very long (because she doesn�t see Jane Addams?)
She then goes to Correspondence and sorts by author.
She clicks on Books, then on Periodicals and sorts by Suffragist. Still no Jane Addams.
She goes to Interviews, then asks for help.

We tell her to go to the Search page. She types �Adams, Jane� and gets 33 hits. The search only returned hits on Jane because Addams was misspelled. She tries again, typing Addams, Jane, and gets 46 hits. She goes to the Speeches and Secondary Books search results and sees Jane Addams mentioned there. She says the 40 Oral History hits are overwhelming, and doesn�t like being brought to the middle of a primary document. She says she would scroll up to the beginning to get context.

She says she would look at all search results, and would check out the secondary books from the library (not read them online). She suggests adding a full citation to the book result screen.

SCENARIO 2 (International Women from memory):
Went to search page; entered �British suffragist� in quick search. Got lots of hits (because the search engine retrieved hits for both terms). Then entered British and suffragists in search box in the bottom Dynaweb frame and got 5 hits. She liked the Dynaweb search form. She didn�t find the answer to our question, thought, so she went back to the search form. She wondered what other search options there were. She did not read the search tips. She thought it would be good to search on an organization�s name. She entered the organization�s name in full feature search and chose limit to: organization name. She wanted to do a Boolean search. She then clicked on Browse terms even though she said she wasn�t sure what she would find. She was still confused when she saw the list of terms. When asked why she did not read the search tips she said that she thought she could figure it out, that the search page is in a familiar layout so didn�t think she needed the tips.

Says that she�s used to the Melvyl model of lastname, firstname, and thinks we might want to use that format as a default (since scholars may be used to that).

Participant 2 did not go through SCENARIO 3 because she had gone through the scenario when testing the low-fi prototype (and because we ran out of time). We did ask the user about her overall impressions. She thinks the Ephemera is great because in a physical archive you would have to get permission, possibly pay for access, and the document might not be readily available.

She notes that we might want to add that the periodicals are �selected� periodicals so that people won�t be confused into thinking we have everything.
 


Participant 3, April 23, 1999, 6th year history graduate student

Summary

� Doesn�t know whether to click on About this Site or Guide to the Site (same thing)
� A �scope� statement about the site might be worthwhile to include, reminder to people of limited scope of the site
� Interview (audio): include date of interview and interviewer�s name; include source and that it�s a piece of a larger interview
� Introductory material to the suffragist at the beginning of the oral history (or a link to the biography on another page of the site)
� Notes that another way to use the site is as a study of oral histories
� Periodicals: include editorial note about �selected� articles/journals
� Search is confusing, overwhelming at times
� Consider adding links to related sites (women�s and suffragist)
 

Details

She starts from home and quickly reads the Introduction. She is confused about whether to click on About the Site or Guide to the Site, but chooses About and realizes that she�s seeing the guide.

She clicks on Show Timelines, asks why hide them, but then says maybe if you�re working on a smaller screen� (She does not look at any year at this time.)

She looks at the choices on the left bar and says she would not ordinarily go to anything that involved audio (due to lacking the appropriate software) or photographs (because graphics are too slow).

She looks carefully at the Guide to the Site, looks at the explanation of Search, and says she assumes you could search on names too.

Her questions at this point are: Where there any males involved in the Suffragist movement. Only women mentioned here. Her comment is that the site seems great for teaching.

She clicks on Speeches and says �that�s neat.�
She clicks on Songs, and really likes that they�re there (although there�s not sound on the computer we�re using). She says they�d be good for teaching.
She clicks on the interviews and says �Great! Alice Paul�s voice! Wow!� She likes having the text and being able to hear what she says. She suggests including the date of the interview in the text. [Good idea.]

She mentions the issue of memory (since the women were remembering back 50-60 years) and noting the context of the 1970s, when the interviews took place. She also suggests making clear the source of the interviews (eg: from the Bancroft), whether they�re pieces of longer interviews, and who conducted the interview.

She goes to photographs and clicks on one to make it larger.
She then goes to Oral Histories, and says she sees familiar and unfamiliar names. She goes to Mabel Vernon�s Oral History. She understands that the left frame is a chapter outline. She wants an overview of Mabel Vernon, and clicks on Introductory Materials to find out who Mabel Vernon was. But there�s not really anything there except a chronology, and she would rather have a summary that highlights Vernon�s activities.

She notes that another way for a historian to use the oral histories is as a history of the interviews themselves.

She clicks on Periodicals, then clicks on The Suffragist to go to articles from that periodical [very exciting because no one else seemed to see those options]. She suggests adding a brief explanation of why we have just those two periodicals; at least something noting that these are just selected periodicals. She then clicks on Sort by Subject Terms [again, very exciting to us]. Her suggestions at this point are that as the site grows to maybe pick some selected subjects and/or popular journals�to make editorial selections for the user rather than overwhelming them with all the information. She says that she misses the context of an actual periodical, but appreciates the easy access to the information.

She clicks quickly on Correspondence, but the time for browsing is over. We briefly show her Meet the Suffragists and Intro to the Era. Her comments are that Meet the Suffragists should be linked to within the site (we agree) and that the Intro to the Era looks like a good length. She also says we might want to remind people of the limited scope of the site (and maybe limit periodicals to just articles related to the specific women covered on the site).

SCENARIO 1 (Jane Addams):
She goes to Search, stating that she�s not good at searching and it makes her uncomfortable. She types �Jane Addams� into (??Full Feature??) and gets 154 hits, which she considers a lot. She asks whether she needs a boolean �and�. She says she probably would have browsed the terms list if she got back so many results.

We tell her to try the boolean using the Advanced Search option (we also tell her that should could do exact phrase). She now gets 44 hits. When asked about the numbers on the search results page, she says she didn�t notice them and she is confused by them. She goes to the Timeline results, but says it�s not so useful (vague information). She asks about the triangular buttons at the bottom of the page�unclear on what they are. She likes the personal accounts better than the timeline results.

SCENARIO 2 (International mother/daughter):
She starts with Search again, but this time goes to Browse. She scans for duplicate names, thinking these might mean mother and daughter. She then goes to Correspondence (thinking they might�ve written to each other), and the back to search. She seems hesitant to search, and doesn�t seem to know what search term to use. We encourage her to go ahead, and she enters �England�. She gets 252 hits, and says that she might not continue at this point. She looks at the resulting books and oral histories, but admits she�s lost. She searches on �British� and gets 97 hits.

SCENARIO 3 (Alice Paul and 1917):
She starts at Meet the Suffragists to get an overview of Alice Paul. She clicks on the Oral History (from the bio) and says she would try to find her way through the Oral History to find 1917. She sees 1917 in the Table of Contents. She clicks on Back (?) to see the Timeline. Clicks on 1917 to get an overview of the period (she says). She likes the contextual material, but wonders why the specific items are chosen. She likes the side-by-side material.

She clicks on Periodicals to get more information (for example, if she knew Alice Paul had been arrested she would go to that subject), and she sorts by date to get 1917. She also goes to Correspondence and sorts by date. She clicks on Books and asks what she will get. She says at this point it may be worth doing an actual search of Alice Paul, maybe limiting by date if possible.

She clicks on Ephemera because she hadn�t seen that before. She says that she does like to browse terms lists.

Overall, her comments were to include links to other related sites (women�s and suffragists projects). She says the site could provide focused study for undergraduates, such as concentrating on one oral history.

Her questions are whether there were any men involved and if there is anything about the anti-suffragist movement. She suggests emphasizing that the site contains information on leaders and activists who weren�t necessarily famous.