• Test Procedure
• Task Instructions
• Demographic Questions
• Follow-up Survey
• Informed Consent
• Test Logs
• Test Measures Results
• Follow-up Survey Results

Test Procedure

1. Ask test user to sign the informed consent
2. Explain the purpose of the test
3. Ask test user to fill out the demographic questions
4. Introduce briefly about the application
   • Elements of the interface (menu bar, tool bar, folders, views)
   • Main, Map and Calendar views
   • Limitations of the current interface
     • Java is slow
     • drag & drop is not implemented
5. Introduce the user test procedure ·
• How to do the test - think aloud
• Three scenarios briefly
6. Conduct test.
• A. Simple adding, saving and sorting bookmarks and folders
• B. Browsing with History Map view
• C. Power search history records in calendar view
7. Follow-up interview (questions to fill out + brief interview)
8. Thank the test user!

Task Instructions

In this test, you are going to perform three different tasks using Webmarking, an application to organize web bookmarks and history files. Steps that you take in each task are given in the following. Please feel free to ask questions and make comments at any time during the test.

A
You organize your bookmarks using folders. Recently you have visited several newspaper web sites and saved these sites as bookmarks. Now you are viewing The New York Times homepage.

1. Add this page to your bookmark using the Webmarking interface
2. Create a new folder to store several newspaper web sites. Name the folder "News".
3. Put the New York Times bookmark to the "News" folder.

You remember having visited The Financial Times homepage yesterday. But the site is not bookmarked.

1. Open a history folder for yesterday.
2. Find The Financial Times site in the folder.
3. Move the site to the "News" folder.

B
You would like to view your history files using the "Map" view in Webmarking. The "Map" view allows you to view your history files (pages you've visited) in a graphical format showing not only the pages you've visited but which pages you visited them from as well. In this case, you would like to view all the pages that you visited from The New York Times home page.

1. Go to the "Map" view.
2. Find and select The New York Times home page as the root document in the "Map" view.
3. Find the article of The New York Times you read which is titled "How to Separate Good Data from Bad".

C
Webmarking allows users to conduct searches over their history and bookmark files while also offering them a navigational interface to facilitate browsing where searching is simply insufficient. In this scenario you would like to perform a search to find the page from The New York Times that you visited last week. The problem, however, is that you are not sure when last week. You are certain that it was not Friday nor was it Wednesday. You decide that the best way to find this page is by having the search results displayed in the "Calendar" view so that you can go to last week's files and see for yourself when it was.

1. Open Search dialog box.
2. Search on all pages from The New York Times. Select Calendar view as a result view.
3. Change the view to "Week" and determine which day (not Friday and not Wednesday) it was when you visited The New York Times.

Demographic Questions

Gender

• male
• female

Education/Major

• undergraduate
• masters
• phd

  major: ..................................................................

How often do you use web browsers?

• everyday
• few times a week
• once a week
• less than once a week

Do you bookmark the web sites you like or often visit?

• often
• sometimes
• never
• I didn't know the bookmark function

Do you organize your bookmarks using folders?

• Yes. I have created many folders
• Yes. But I only have a couple of folders
• No. I don't use folders
• I didn't know that I could use folders.

Do you use the history function of the web browser?

• often
• sometimes
• never
• I didn't know the history function

Follow-up Survey

Overall, how do you feel about the interface?

easy to use 5 - 4 - 3 - 2 -1 difficult to use

Are the icons intuitive?

easy to use 5 - 4 - 3 - 2 -1 difficult to use

How do you like the following features?

• Viewing history and bookmark folders in the same window

I like it! 5 - 4 - 3 - 2 - 1 I hate it...

• Map view

I like it! 5 - 4 - 3 - 2 - 1 I hate it...

• Calendar view

I like it! 5 - 4 - 3 - 2 - 1 I hate it...

• Viewing search result in different views

I like it! 5 - 4 - 3 - 2 - 1 I hate it...

Informed Consent

A group of Masters students in the School of Information Management and Systems at UC Berkeley is conducting a study to assess the usability of a new system for managing web bookmarks and browser history files. 

If you volunteer to participate in this study, you will be asked to perform some tasks using a paper prototype of a computer interface.  You will be also asked to answer questions orally before, during and after the test.  A test observer will write down your comments and behaviors while you are participating in the test. 

There are no direct benefits to you for participating other than what may be an educational experience in viewing a new interface to organizing web bookmarks.  There are also no risks to you; you will simply be playing with a paper prototype and answering some questions.  The results of the test will be kept confidential and not shown to anybody in a form where you can be identified. 

If you accept these terms, please write your initials and the date here:  
 
  

...............................................................  
(initials) (date)

Test Logs

Test Measures Results

The following situations are counted as "errors" in this measurement;

• select incorrect menu item or icon
• cannot find the right way to complete a task and ask the evaluator how to do it

Since test users and an evaluator talk while users perform tasks, we couldn't measure the accurate task completion time in this testing. Instead, we measured it by relative measures, "quick" "moderate" and "slow," to see how easy to complete a certain task or how intuitive our interface is.

Numbers in the following table represents the number of users that fall into each category.

Follow-up Survey Results