The lack of a central repository (akin to ERIC for researchers in Education or the ACM digital library for Computer Scientists), plus the absence of an information exchange forum, creates the need for an open online meeting place where academics and field workers can exchange information and comment on each other's work. This central repository would include both field projects and research documents. It would allow users to easily store information on field projects, document their progress, and make comments on projects and documents. Such an application would be especially useful if layered with a function to allow annotations.

If these annotations can be indexed, they can be more of an asset to users. Users can add or view comments and references which are specific to certain blocks of text. They can define new terms within the text on the website. They will also be able to start new threads of discussion that link into documents. Some key elements to the success of such a system include creating the right mix of openness, to encourage use of the system, devising mechanisms to monitor content, and designing a system that is scalable, which is essential in an area of research that is constantly evolving.

Our solution was to implement a system with an open structure. We wanted to improve upon the two main existing databases on IT and Development: the World Bank's Development Gateway and the World Resources Institute's Digital Dividend. To do this, we designed our system around the user. We created a category structure based on user testing. Comments on documents and projects can be added very easily, without registration, and thereby encouraging participation and exchange of ideas. The structure and features of the site were also determined through user tests. Our goal was to design an system that would make the process of collaboration in IT and Development more effective, open, and enjoyable.

Jeff is working on his latest economic cooperation plan, involving two well-known organizations. He finds a document in the Collaboration Repository about building a small business in the town of Mbale, Uganda. After reading through the document summary, he clicks on the paragraph number on the left side of the document to view all the comments made for that paragraph. A small pop-up window appears, showing all the comments. He reads a few of the comments and finds them useful for expressing brief ideas to specific parts of the text. Jeff closes the pop-up window and continues on to the user reviews.

After reading a few user reviews, Jeff decides to add his own comment to one of paragraphs. He scrolls back up to a paragraph talking about NGO strategic planning and organizational development and clicks on the "Add comment" button. A new window pops up with the paragraph text as well as some form fields for him to fill in and add his comment. He fills in all of the fields and types in a short blurb about the program he is currently developing, then submits the comment. The pop-up window closes and the document page refreshes, showing the updated comment count for that paragraph. Jeff feels satisfied that he has contributed his knowledge, and found some valuable information for his economic cooperation program as well.

Tamara Sen has been working on a project with her crew of graduate students on creating a GSI map to submersible borewells in rural Bangladesh. Opinions on the use of borewells are highly contentious, especially in the flat plains of Bangladesh, which are areas highly prone to drought. Tamara has a disadvantage in being several thousand miles away from her test bed. Thus, she is very keen on receiving information from the field on what development professionals and planners think about this project.

She goes to the Collaborative Repository and answers a series of short questions about her project. The purpose of the questions is to classify and categorize the project. She clicks on the continue button and is taken to the second half of the add form. This page contains the basic information about her project, like the title, summary, contact person, etc. After filling this form out, she is taken to a confirmation screen. Tamara is happy that finally she has a found a way to receive comments and feedback about her project quickly and easily. She awaits eagerly to read what other people will say about her project.

After endless hours of searching the web for information about healthcare technologies in Colombia, Stephanie has found little credible and relevant information. Realizing she needs a much more trustworthy source, she turns to the Collaborative Repository after hearing a colleague mention that it helped their organization. Stephanie goes to the website of the repository and clicks on "Search." Presented with a set of categories options, along with a specialized keyword search prompt, Stephanie can't contain her smile. She selects the most relevant choices encompassing healthcare and Columbia, types in 'technology' in the keyword prompt, and hits the search button. The search results contain a series of documents and projects from which she can peruse at will. With so many hours saved, Stephanie now has more time to write her PhD thesis.

View Interaction Diagram

We used Dreamweaver, and PHPMyAdmin to create the look and feel of the interface. Dreamweaver allowed us to use libraries, CSS, and templates to help make our pages more consistent. It also allowed us to test pages instantly by automatically uploading them when we saved them, saving us uploading time and speeding up our testing process. We encountered minor problems setting up the connections, but once those were complete, it was not difficult to use the application.

We used PHPMyAdmin, a PHP-based GUI for MySQL, to look up table information used to write the search queries. This tool was helpful in that we did not need to know all the SQL commands in order to modify or view database properties.

For the View/Add Comments page, we used Microsoft FrontPage and JavaScript to rapidly prototype the document view screen and the popup windows for adding and viewing comments. FrontPage helped by providing a quick and easy way to mock up the pages, and JavaScript was a convenient way to simulate the sequence of popup windows for adding and viewing comments. These tools did not help to create an actual connection to the database to work with real data, but this was acceptable for the initial prototype.

Using the Collaborative Repository, users can enter document or project information online, search for documents and projects, and have others rate and comment on the text. Users can comment at a paragraph level of each document or project, as well as comment on the entire document.

The News page pulls in news from around the world concerning IT and Development as soon as they are created, allowing users to stay updated on current events within the field. The home page lists a few new documents and projects recently added to the Collaborative Repository system. On the home page, users will be able to see what is new in the system at a glance, or they can use the quick search to find documents of interest to them. Finally, users can browse through categories on the Browse page if they are interested in a certain topic, but they are not looking for a specific document or project.

We did not implement a dynamic Browse page that connects to documents associated with the categories on that page. We wanted to present an example of what our Browse page would look like if it were functional, so we typed in the categories on that page, but did not connect them to a corresponding search in through the database. Our primary focus remained on the commenting feature's user interface.

We did not add a capability to print to PDF. Users can print by printing out the HTML page if they need to, but we wanted users to do most of their work online, and keep returning to the online representation of the document to make the system truly collaborative.

Lastly, we did not implement a capability for users to edit or remove their documents, comments, or projects. In order to build up a critical mass, we wanted documents and projects to remain in the system, even if the authors had received negative comments or if they were finished working on the project or document. Instead, we created an email notification system through the My Account page to allow moderators to receive emails requesting changes or deletions for documents and projects. This would allow users more time to consider whether they really wanted to remove a text, and also the moderator could monitor these user activities as a way of obtaining user feedback on a project or document.

The initial sketches of the design mainly focused on the commenting interfaces. These initial sketches show our ideas of how the document and viewing comments would be frame based. There would be a pop-up window to add a comment. These ideas were later incorporated into the paper prototype. When viewing a comment, the top half of the frame would have a list of all the comments, while the bottom half contained the actual comment. The add a comment pop-up basically remained the same throughout the whole iterative process. This screen had fields for the username, email, text to comment on, and a field for the comment.

For the low-fi prototype, our team focused on three main sections: the search page, the commenting page, and the edit a document page.

Some of the major differences from the initial design sketches were that we built out the interfaces for the search and add document pages. Our search form consisted primarily of checkboxes and text fields, some of which responded to user input immediately (for example, selection of an Area such as 'North America' opened a list of countries in 'North America' to search). The commenting screen showed how the documents and comments would appear in different frames of the same browser: one on the right, the other on the left, respectively. We used small pieces of colored paper within the document to denote an icon. Users could click on the icon and view all the comments associated with it. Finally, the edit document screen was a simple form containing three main sections of data to fill out. Users clicked on one of three large navigation buttons to get to each section of the metadata (such as author, date), research information (such as participants) and search and categorization to allow users to enter keywords and categories to classify their document.

We made several changes to our prototype as a result of the low-fi testing. For the search page, we separated the document and project search pages. The users felt that they would be either searching for documents or projects, and having a search which combined both would provide too much information. We also changed the categories according to the results of the card sorting task done by our low-fi test subjects. We provided examples after each top level category of its sub-categories because the top level categories were too vague. We added a pull down menu of countries instead of having the countries categorized according to the continent (Central America, South America, North America, etc.). Some of the users were not sure where to look for a certain country using the categories, so we decided to give them a complete listing instead.

For the commenting page, we renamed the "Annotate" button, which was unclear to many of the users, to read "Add Comment" instead. All of the users said that they preferred a popup window rather than frames to add and show comments. They wanted to be able to associate the document they were reading with the comment they were adding. They felt that a separate pop-up window, rather than a frame side-by-side with the document, would offer more user control. One user expressed a strong desire that his workspace should not be disrupted, resized or obscured in any way by the UI, but felt that a single pop-up window was acceptable because he could easily move it out of the way or minimize it. We did not require the user to login in order to add comments. Users expressed differing views of requiring users to be logged in before they were allowed to comment on documents. One view was that logging in would help ensure that those adding comments would feel like they were a member of a community. In addition, logging in would promote better quality information sharing. Another view was that the ability for anyone to add comments freely would encourage wider participation in sharing of ideas and opinions on the site.

For the Add a Document page, we decided to implement one of the user's suggestions for avoiding potential misclassification of a project or document. This was done by adding a section where the user must answer a series of questions. Based on the answers, the project or document would be categorized automatically by the system. In order to avoid copyright infringement, we added a disclaimer and privacy statement that the user must agree to before adding a new document.

http://dream.sims.berkeley.edu/collabrepository/

Road Sage provided us with numerous points to improve and change for this iteration. For the second interactive prototype, we focused on fixing most of the heuristic violations, mainly ones with a severity rating of 3. Some were simple, such as increasing font size and color, but other changes required simplifying repetitive tasks and providing clearer instructions.

The rest of the changes surrounded areas about error prevention and consistency. We found it necessary to clarify what fields in form pages were required, and which were optional. This is an improvement, because it saves the user time when accomplishing their task, along with less confusion. Error messages that occurred from using the site were modified to give the error more feedback, and prevent further errors. For the search, we eliminated the case where no search results existed, and provided them with the opportunity to refine or search over. We also added some help text to all of the pages. However, the help section is an area that will be much more detailed and improved overall in future iterations.

The following are some of the major changes we made and their justifications:

Add Comment Page

• Added instructions on how to add a comment – "To add a comment, please select the word or phrase you wish to comment of from the text and click 'Add Comment.'" And we added an error message if the user tries to add a comment without first selecting some text. We also implemented the javascript that actually allows the user to select text and have it appear in the "Add Comment" dialog box – select text, click Add Comments, and the Add Comments dialog box will display the text you selected in the "Text to Comment" field.

Add Document Page

• The document entry page was broken up into two pages: one page with the metadata and category information, and the other with document general details. If the user has an unreliable connection, she will not lose much information, or she can wait until the connection is better and then continue entering information.

• The user will only now see the disclaimer/privacy page upon registering with the system.

Search Page

• If there are 0 records returned, it tells the user to submit an empty search form to see all available records in the system. The search form is also displayed so the user can quickly search again without going back to the previous page.

Changes to Overall Site

• Included help text on search page and add document page.

• The font size was made larger, and the font color was made brighter so that it provides more of a contrast against the navigation bar.

http://dream.sims.berkeley.edu/collabrepository/production/

The changes in the third interactive prototype involved fixing issues brought out by the pilot usability test:

• Provide more clarity between top navigation links.

• Make editing document process more intuitive since users are unlikely to read the help.

• Combine the separate search pages for projects and documents into one page and then separate the results by projects and documents.

• Indicate required fields on search pages.

• Change the name of My Stuff to My Account.

• On commenting page, change pencil icon to read Add Comment.

• Make documents easier to read (change background color).

• Implemented toggling to show or hide unwanted sections, like hiding project search results if you are only interested in documents, show/hide individual comments on the view comments page.

• Built login mechanism to restrict uploading and rating of documents to registered users. We wanted to ensure better quality documents in our system, while still promoting the free flow of ideas.

• Added news page which receives RSS feeds. The purpose of this page is to provide an incentive for returning to the site and build up a critical mass.

There are several reasons this prototype was the most effective. Firstly, this technique allows us to gain immediate and direct feedback. Since it was a paper-based model with post-it notes, construction paper buttons, and so on, our users were not hesitant at to criticize and suggest new features. This then led to the most amount of feedback relative to any other technique. We learned to avoid the term ‘annotate’ and use ‘comment,’ and were very surprised to learn that a frame-based model was shunned by everyone. This made us focus on building a site that worked with pop-ups. One person even suggested ways of outlining comments in these pop-ups.

We also used a card sorting technique that involved terms relevant to information and communications technology in the developing world. We realized we had too many cards, and saw how time consuming it was for the user. Fortunately, we gained new insight to better construct a hierarchical browsing framework to search for documents and projects in the system.

In comparison, the heuristic evaluation and pilot usability test were focused on our interactive prototypes. Though effective in other ways, such as different feedback based on error message placement and icons, these two evaluation techniques either reinforced issues from the low-fi prototype, or focused on features that were not in the system yet. Nevertheless, we learned a great deal from the two techniques, but they dealt more with visual and feature oriented problems, which weren’t an issue with the low-fi approach.

Low-fi prototyping definitely has obvious advantages over the other techniques, but it may be inaccurate to compare them because this technique occurs at an earlier stage. It provided us with an earlier opportunity to compare our proposed ideas, and saved a lot of time for the implementation stages that may also explain our favoritism.