1. Introduction
2. Prototype (Images)
3. Method
4. Test Measures
5. Results
6. Discussion
7. Appendix

1. Introduction

The purpose and rationale for this experiment is to determine how well the mental model embedded within this initial prototype fits with users' current mental models of how they can manipulate and view bookmarks and history files. In addition to that it is also important to determine how well the different history views facilitate accomplishing the task of finding pages that were once visited. 

2. Prototype 

See images of the paper prototype

In order to be able to test the low-fi design on actual users, we have created a paper-based prototype as described by Mark Rettig and James Landay. The prototype consists of a fixed application window containing menus and icons, as well as frames defining two empty windows used for navigation and presentation purposes.

Because the prototype has empty navigation and presentation windows, it is possible for a user testing the prototype to choose views dynamically according to the tasks they are asked to perform. The navigation window can be set to show bookmark folders, history folders or both at the same time. The user may choose to view the presentation window using map, calendar, tree or frame view. In the paper-based prototype, we have ignored the possibility of resizing windows, as this would have been very difficult to implement.

In addition to the different views, several dialog boxes are included in the prototype. These are low-fi versions of a search dialog, as well as dialogs for creating folders and bookmarks. The dialog boxes are placed on top of the application window as they are required. Some static browser windows are included with the prototype as well. These are used to make it possible to emulate the interaction between a browser and the proposed application, hence making it easier for the user to visualize the bookmarking process.

Finally, to be able to simulate the application behavior, the prototype includes a make-shift mouse-pointer and transparent colored bars used to mark folders and bookmarks as selected. Paper pull-down menus are displayed if the user accesses a menu item or a combo box in a dialog. Colored icons representing folders and bookmark icons are placed in the navigation and presentation windows as needed. The map views used are static and cannot be changed by the person serving as "computer" because it is difficult to change a map on-the-fly. Tree, frame and calendar views are displayed using paper icons and labels.

The user communicates with the "computer" using the mouse pointer. The "computer" has to display the views selected by the user, as well as refresh these whenever the user clicks on a command. Additionally, dialog boxes and pull-down menus must be displayed when appropriate, and folders and bookmarks must be highlighted when selected by the user.

3. Method 

Task Scenarios 
Tests were conducted using the following three scenarios; 

A.   Simple adding, saving and sorting bookmarks and folders 
B.   Browsing with History Map view 
C.   Power search history records in calendar view

1. Add this page to your bookmark. 
2. Create a new folder to store several newspaper web sites.  
    Name the folder "News". 
3. Put related bookmarks to the "News" folder.

4. Open a history folder for yesterday. 
5. Find The Financial Times site in the folder. 
6. Move the site to the "Newspaper" folder. 

You would like to view your history files using the "Map" view in Webmarking.  The "Map" view allows you to view your history files (pages you've visited) in a graphical format showing not only the pages you've visited but which pages you visited them from as well.  Pages viewed in this view are connected then by other pages that may have links to them.  For example, if you visited Amazon.com by accessing a link on the Yahoo home page then it would look like this: 

In this case, you would like to view all the pages that you visited from The New York Times home page. 

1. Go to the "Map" view.
2. Find and select The New York Times home page as the root document in the "Map" view. 
3. Find the second page of one of the articles you read.

Webmarking allows users to conduct searches over their history and bookmark files while also offering them a navigational interface to facilitate browsing where searching is simply insufficient.  In this scenario you would like to perform a search to find the page from The New York Times that you visited last week.  The problem, however, is that you are not sure when last week.  You are certain that it was not Friday nor was it Wednesday.  You decide that the best way to find this page is by having the search results displayed in the "Calendar" view so that you can go to last week's files and see for yourself when it was. 

The "Calendar" view allows view your history files in a graphical way quite similar to the look and feel of a personal date book.  In this view you can zoom in on particular days, weeks, or hours in order to see which files you've visited during those times.  

1. Go to the "Calendar" view.
2. Open a search window to start a search on all pages from The New York Times.
3. Then go to last week (Feb 28 - Mar 6) and determine which day (not Fri and not Wed) it was when you visited The New York Times.

1. Ask test user to sign the informed consent.
2. Explain the purpose of the test and introduce briefly about the application.
3. Ask user-type questions.
• How often do you use web browsers?
• Do you bookmark web sites you often visit or you like?
• Have you ever organized your bookmarks using folders?
• Do you know that you can go back to the sites you recently visited using history files?
4. Conduct test.
A.   Simple adding, saving and sorting bookmarks and folders
B.   Browsing with History Map view
C.   Power search history records in calendar view
5. Debrief.
6. Thank the test user!

4. Test Measures 

• How intuitive and easy the system was to use 
• Whether or not the participant could understand the various navigation modes 
• Which features were found useful and which were less so 
• Which useful features were not provided 
• Whether or not the system was useful both to novice and experienced users 
• To which degree our application was successfully integrated into the browser (on a conceptual level)

5. Results

Scenario A:  

Adding new bookmarks seemed to be a problem - it was not obvious whether or not you should add bookmarks using the browser or our application. The participants wanted a simple click-and-drag feature. Furthermore, whenever a dialog box for new items pops up, focus should automatically be transferred to the "Name" field for input. Regarding the other dialog box fields, "Date" was generally seen as unnecessary (no need for editing it), while "Description" seemed somewhat unclear (what does it actually mean and do?)  

When the participants started moving files between folders, there seemed to be a feedback problem - it was not obvious whether or not the file was actually successfully moved. Also, when bookmarks are created from history files, the files should perhaps be copied rather than moved (so it would continue to exist as a history file). 

The map view was generally well received, though it did perhaps seem to have a conceptual flaw: how is the map displayed if a site is not visited by link? Furthermore, the tabs for changing view modes were not visible enough. There should also be more information about the WWW sites in this view than what was currently available (perhaps such info could be displayed when the icon is selected). Finally, opening a page required a double-click, which was not found entirely sufficient - an "Open Page" button and menu choice should also be available. 

This was perhaps the most popular feature - one participant found it "amazingly cool!" What seemed like the most significant problem was how to switch between month, week and day view. As with the map view, there should be more information about the WWW sites in this view than what was currently available. On a more specific level, the week view spaces for Saturday and Sunday should certainly not be any smaller than those for weekdays.  

The search dialog box did not have a way of specifying view-mode, which caused some confusion. In addition, the "last visited" criteria needed far more options - default should perhaps be 'anytime'. A feature for specifying the range yourself (like in a database query - e.g., Mar 12 1999 to May 15 1999) would certainly be useful for the more advanced users. 

The effective difference between single and double mouseclick was not always obvious. Perhaps more importantly, the integration between the browser and our Webmarking application was not adequate - they functioned more like two separate systems. To make the system more user-friendly, we might consider importing icons and buttons directly from the browser - this way, it would seem more like an extended browser feature than a system in itself. 

6. Discussion

The results from the "Map" viewing scenario were that the display did not contain enough information about how the navigated through the files being displayed. Users felt that the connectors between history records didn't show direction as well as a way of showing whether or not the user was behaving in a breadth intensive or depth intensive pattern. If they viewed a page from a portal site, then returned back to that portal site and viewed another page, there was no way of representing the amount of breadth based browsing carried out by the user. After discovering this, we decided to divide the page connectors into "from" and "to" types including directional arrows. Along with that, we decided that applying a shading scheme to both pages and connectors in our map to determine how often a particular page was accessed and from where would provide a better mental model of how users perceive their browsing behavior.  

As for the calendar view, we felt that users were generally pleased with being able to view their history files in this manner. However, we discovered that if a particular day had several history files, simply providing an icon to view those files was not sufficient enough. We decided to include more information in the month and week and day displays. In a particular day or portion of a day contained several files, then the icon representing those files should be shaded by number of files that it represents and even possibly displaying the number of files above the icon. Therefore if a user was viewing their history files by week and they noticed an icon on a particular day, they would also know how many files to expect to find if they were to zoom in on that particular day or hour even.  

7. Appendix

• Test Procedure
• Task Instructions
• Evaluation Sheet
• Informed Consent
• Test Logs