SIMS 213 Assignment 3: Task Analysis

By Haydee Hernandez, Qun Liang, and Hailing Jiang

Group Names and Manager Roles [top]

Problem and Solution Overview [top]

Existing KM sites fail to offer a good solution to the problems mentioned above. First, they generally presume an acceptance of the discipline, so there is no endeavor to convince skeptics or address their concerns and misunderstandings. Second, they are geared towards amassing links rather than analyzing possible user tasks associated with those links. This inhibits information chunking and potentially results in information overload. A KM thesaurus is wanting. A third common problem with the existing KM sites is that they presume an interest in only a small set of the KM sub-fields rather than the whole discipline, thus leading to sites lacking in either breadth or depth.

The improvements we suggest in fulfilling those various user demands are threefold. First, new content has to be generated and presented to the user. It should at least answer the following basic questions: What is KM? Why should we care? How is it different from pre-existing disciplines? What are the hot topics in KM? Second, we propose to provide users with a conceptual framework within the UI and allow them to browse our thesaurus to find their interested information. This has proven a good starting point for novices in a field. Lastly, we will integrate our thesaurus with the search engine. The thesaurus can be used not only to organize information items and automatically reformulate user queries, but also to present the search results in the context of a hierarchical structure to give users some clue as to which pieces of information can better satisfy their information needs.

Task Analysis [top]

Target users and tasks

• Information managers or assistants to managers, who need information to solve the KM problems in their corporate environment;
• University professors or researchers who are interested in this area;
• Graduate students, taking courses on KM;
• Average business professionals or general information seekers who have a general interest in KM.

• Find out general information on KM;
• Survey KM domain, e.g.., find out the difference between KM & other related disciplines;
• Find specific information on subjects in KM;
• Find specific answers to a vaguely stated question;
• Find contact information or people, organizations, and companies related to KM domain;

Interview questions and results of interviews

1. Have you heard of the term "Knowledge Management"? If so, where?
2. What do you think KM is? What is the difference between KM and information systems, KM & CS?
3. Have you ever used any KM site? Which ones? What do you like and dislike? What unprovided features would  you like to add?
4. Where do you do most of your online search? Home or Office? Modem or high speed access?
5. If you are asked to search for information on KM, what would you do and where to go? Why? e.g.. go to friends, colleagues, online search?
6. What steps do you take to conduct a search?
7. What is your biggest complaint when using search engines?
8. How do you gauge a good website from a bad one in terms of reliability , organization, quality of provided information?
9. Do you prefer to have the full URL typed out in the search result or is a hyperlinked title satisfactory?
10. How do you like to save your online work? Print? Email? Bookmark? Save file or others?
11. How often (if at all) do you contact people or organizations that you find online?

• Interviewee 1 -- Harry is a consultant and publisher in the KM community. Survey work he has conducted (the largest in the KM community as of today) reveals that KM professionals value the ability to answer their specific questions with specific answers. For instance, they may want to find a project management product capable of riding on top of their NT system. They don't want irrelevant results about how to do project management. The basket addresses this need by allowing users to find all subject terms matching their criteria. This should only bring valuable records to the foreground. Harry also liked the idea of a thesaurus to add user browsing and exploration to get a sense of the entire discipline. His main concern was whether the thesaurus would use the proper terminology and be organized in an intuitive fashion. He also brought to our attention that there is a gap between the terminologies used by the business world and the academia. We can address this need within the Gotcha group through further usability testing.

 
• Interviewee 2 -- Bob, a PhD student in Computer Science, demonstrated a sound understanding of Knowledge Management and the distinct difference between it and other disciplines. While he didn't use specific KM sites, he did provide useful information on his search methods. For searches on a new domain he jumps to Yahoo because its subject orientation provides a better understanding of a new discipline. This reinforces the needs for a Browse Subjects page for Gotcha. When gauging the credibility of information he uses referrals and domain locations (.edu) to judge the caliber of work. He also stated he prefers to immediately do a search upon entering a new site. The presence of a full-text query field at the top of each tab addresses this need. He saves his on-line work using all options at some time or another. We currently support all options in our prototype. He rarely contacts people or organizations. Lastly, he raised the desire to have more expressive search criteria such as support for alternative terms. Even though, we don't provide the flexibility he wants, the basket could be indirectly used for this purpose.

 
• Interviewee 3 -- John is a project manager in a company. He does not know much about KM discipline but he recognizes its importance as the winning power of a company in competition. He said that there are no people or divisions dedicated to KM in his company. Knowledge management is conducted in an informal and non-integrated way through technical seminars, internal paper dissemination, meetings and conferences. He expressed his interests in our system. When asked for online search experiences, he said that he usually goes to Yahoo and Amazon.com. He thinks that the way of organizing information in Yahoo is simple, intuitive and clear-cut, which makes life easier for novice users. He uses Amazon.com not only for ordering books, but also for finding information or monitoring a subject area. He likes the title search,  related subjects and "the other books the customers bought with this book" features in Amazon.com. He said that it is so easy to use. He also admitted that he did not try other search engines very much because if he tried once and felt frustrated, he won't try twice. When gauging a web site, he ranked simplicity, clarity and efficiency to convey information as the most important things. He usually saves his online work by Print or Save.

Scenarios of example task sequences

• Scenario 1 -- John works as a project manager in a company. He is new to the KM domain and wants to find some general information on KM. Since he knows little about KM, he decides to read information in FAQ's on KM first to get familiar with the area. With enough background information in hand, he decides to browse the KM thesaurus subjects and navigate down hierarchies to get a better understanding of items that were interesting from the FAQ's.

• Scenario 2 -- Mary is a graduate student in the Business school. She is working on a class project, in which some topics are  related to KM. She wants to search more information on these topics. Mary has some keywords in mind for these topics. So she finds that the full-text query option provided by the system may be the fastest way to find information she needs.  She types in the keywords in the full-text query box. The search results present some records of interest that she subsequently selects and emails to herself. She iterates through more searches by clicking on the hyperlinked subject categories of interesting records to find other records that might be useful. She keeps doing this until she either runs a new query altogether or she has enough information.

 
• Scenario 3 -- Susan works as an information manager in a company. Her boss asked her to do a study and evaluation on the options and issues of developing KM systems. She needs to find information on the existing systems, the companies behind the systems, etc.. Since the question is pretty vague, she has no good keywords to start with. She decides to navigate the thesaurus hierarchies to get some ideas. After navigating the thesaurus and checking the items in some of the categories, she may have a clearer picture on the problem and find some interesting information. In this stage, she could also use the full-text query option to conduct further searches on the topics that are particularly interesting. As she continues throughout her search process she keeps emailing herself the bibliographic records that are the most useful.

Suggested Solution [top]

Functionality

1. Search - Users can search bibliographic records through a full text index query, a structured subject search using thesaurus descriptors, and by field names in bibliographic record such as title, author, or date. Three search options have been provided to help users with different information seeking contexts. The first option lets users perform boolean queries the full text index. The second option limits search results catalogued with user-selected descriptors. A shopping cart metaphor is used here in the form of a basket, enabling users to add thesaurus provided descriptors to their query basket. The third option is available to both full-text and basket queries. It lets power users refine query results possessing bibliographic information.
2. Save user work - Users can save their work by saving a file to their hard drive, printing a file, printing bibliographic records, bookmarking desired pages, and by emailing selected bibliographic records. All these options are provided to serve users with different preferences/needs. The email option also supports collaboration enabling users to send their search results to other parties such as a supervisor or group mate.
3. Browse - Users can browse the site using exploratory techniques or they can quickly jump to their desired location by using the site map. Users wishing to browse subject descriptors can explore available subjects by navigating through the Browse Subjects page. They can also use the Quick Jump feature on that page to go directly to the selected subject's page and view its subheadings and associated records. Browsing through exploration and quick jumping provide a nice learning curve for novices. If a novice has never used our site before, she can explore. If she has used our site, she can jump to where she left off without wasting needless time.
4. Contact us - Users wanting human contact can use this feature to email the knowman group feedback on bugs, areas of confusion, etc. The contact feature is good fodder for improvements in future versions.
5. Help - Help is provided in two ways. Users can use the Help feature always found in the top right hand corner to get general help. Users wanting help using their basket are provided that within the context of using the basket to eliminate the need to switch back and forth between general help and subject searching. The left hand column of the Browse subject page remains constant with information on "Using your basket". Users viewing the Basket page also have a constant left hand column except this one contains hyperlinks to common basket tasks and questions.
6. Read - Novice users with no domain knowledge can read original content on the FAQ's page. This establishes an easy learning curve for novices trying to gain an understanding of KM.
7. Establish authority - To establish our site as an authoritative site on knowledge management, the home page will include a description of the manual process used to filter and catalog content. We will also place links to press releases about our site such as the upcoming KM World article. This should persuade users to consider our site more credible.

User interface

Four interface pages are not assigned tabs. They include: the Site Map, Help, Basket, and Search Results. The first three are always accessible in the upper left hand corner on every page within the site. These are pages can be viewed on an as needed basis. Providing tabs for them would have cluttered the screen unnecessarily since they are functions which every novice may not want prominently displayed.

The concept of a basket was inspired from e-commerce implementations of a shopping cart. The Basket page supports the need to modify selections within the basket such as clear all, delete a specified item, or add another item. But since the basket is actually a subject query transaction, it also supports common query activities. It can refine the query through an options hyperlink or by selecting records that are limited by their resource type (print, electronic, or both). The default resource type is both because we assume that users would want access to both unless they have a specific preference for one over the other. To eliminate user confusion on how to perform a boolean query, the terminology Find All (representing AND) and Find Any (representing OR) were used.

In addition to subject searching in the guise of a basket, our site supports full text index queries. These are available on each tabbed page right underneath the tab. This layout enables users to quickly type a query without navigating to the basket if they don't understand its purpose or simply because they prefer the results generated from this type of query.

Navigating subject categories feels like Yahoo. Categories are hyperlinked so that users can travel down the hierarchy. The navigation page displays the category name and the hyperlinked path the user took to reach it. Subcategories are displayed first followed by resources classified to that category. The only difference between Yahoo's style and our presentation is that the facet names presented on the Browse Subject page possess scope notes (definitions) for each facet. This gives users a quick understanding of the type of information they should find listed in that facet. Scope notes are not provided for any other listings.

Drawings of proposed interfaces

Experiment Outline [top]

The experiment we plan to do in a formal study focuses on figuring out the most effective way to facilitate searches by making use of the thesaurus in the user interface. In this experiment, novices will be given a general KM problem and asked to search for articles that answer the given question. The relevant articles in the collection are pre-determined. For this experiment, the independent variables would be the different ways in which our thesaurus could be used to facilitate searches. One way to use the thesaurus within the UI would be to allow structured searches that only utilize the terms contained in the thesaurus. The UI for such searches could be implemented using the shopping cart/basket metaphor, allowing users to select the terms simply by clicking the ones that interest them and choose whether they want the query to include all or just any of the terms picked. Another way to utilize the thesaurus would be to allow ad hoc searches that can use any keyword, and present the subject descriptors manually assigned to a retrieved record and the hierarchical structure in which it is catalogued along with the record itself, so that users can modify their searches at that point. The dependent variables would include the average precision of the retrievals and general personal satisfaction level. The following table lists some of the possible interpretations of the results:
 

Average Precision	Post-interaction survey	Interpretation
Relatively lower	Relatively higher satisfaction score	UI is good, but we must improve descriptors or better understood.
Relatively lower	Relatively lower satisfaction score	Bad UI design, and the thesaurus needs improvement or we need to make the categories better understood
Relatively higher	Relatively higher satisfaction score	Good system
Relatively higher	Relatively lower satisfaction score	UI problem. Determine methods for improvement.
About same as unstructured full-text retrieval	Relatively higher satisfaction score	Good UI design. Seek ways (if any) to improve performance
About same as unstructured full-text retrieval	Relatively lower satisfaction score	Improve UI and thesaurus.

[top]