Test log

[Participant 1 | Participant 2 | Participant 3]

[Return to Assignment 9]

Participant 1

Task1 (20 minutes)

• Start from "Home" page.
• Read the contents in "Home" page, want to see what the site is about.
• Click on "About KM" page.
• Read the contents in the "About KM" page. Assuming that all the questions were on the same page, so just scrolled down the screen to read all the questions, did not click the imagemap on the left column.
• Click on the link "An interview with Larry Prusak", thought that there was probably something relevant to the introduction of KM.
• Go back to Gotcha site.
• Click the link next to the interview "Knowledge Management ...", read the article. Commented that it was a pretty good review in KM.
• Go back to Gotcha site.
• Read the contents in other sections that seem interesting on the "About KM" page and click on some links in the sections.
• Click on  the "Knowledge Management ..." link again, check some links inside that page.
• Read the contents listed under "hot topics", comment the topics are not general, too specific or detailed. he was looking for some general staff on KM, some reviews.
• Click on Resources, Products, Search, trying to see what is under each category.
• Go to Resources page, assuming that there may be some articles in this category.
• In the Reading list page, click on "Sveiby's FAQ on Knowledge Management". Read the contents on the page, found that it was pretty interesting, just the character fonts change a lot on the page.
• Click the "Introduction" link on the Reading list page, comment that this article is a good history review on KM.
• Go to Web sites.
• Click on Brint. Browse the information in Brint site, trying to find some articles.
• Go back to Gotcha.
• Proceed down the web sites.

• Go to Search page, because he already know what he is looking for.
• Type in "intellectual capital" in the search box, then click on go.  Comment that he usually does not touch any other search options.
• In the "search results" page, click on the first result returned: glossary. When seeing the glossary page, comment not very interesting.
• Click on the next two hits, still trying to find articles.
• On the "search results" page, found that the message "Your query is capital" was confusing.
• Go back to Search page.
• Type in phrase "intellectual capital" in the search box, click on go.
• Click on some of the top hits, and read the corresponding pages
• Click on page 2.
• Click on some of the hits, and read the corresponding pages. Comment that the search results did not give him much information he wanted.
• Go to "About KM" page.
• In "Hot topics", found "Intellectual capital". Comment, Hey, this is cool.
• Click on the link "interview with Tom Steward". Comment, he would probably bookmark this page because it contains some references.
• Go to "Resources" page and click on "Web sites".
• Click on "Brint" to brint.com.  Search "Intellectual capital" in Brint because he did not want to spend too much time going through the articles.
• Click through the search results returned by Brint. If nothing interesting there, go back to "Web sites" and check other web sites.
• Go to "Professional organizations", hoping that they may have links to their journals or something.
• Check "Periodicals", does not like periodicals because the contents are not available online.

• Notice that there is a link to case studies in the Home page, click on it.
• Read the cases to see if they have the same problem as his, check the cases in the order as they were listed.
• Check the "Quckie case studies", read the titles of the case studies, trying to find something about sales and customer services.
• Check "Anatomy of a failure". Comment, it is pretty interesting. Read this article and note down some keywords.
• Check some other links listed in the "Quickie case studies".
• Go to Search. The mouse-over windows poped up by accident. Comment that he does not like pop-up things. They are annoying.
• Go to Resources.
• Check web sites, search the same query.
• Go to Products.
• Check the Customer Service Support category. Check the web sites of companies listed under this category. Remember names of the products or some keywords.
• Go to Search. Type in the product names or keywords found in Products page and company web sites. Try to find some reviews for comparison.

Post-test questionnaire

1. Satisfied
2. Resources -- web sites page most useful. It links to other websites.
3. Glossary page the least useful.
4. Satisfied.
5. Very easy.
6. Moderately irrelevant.
7. No.
8.

• Home page is ok.
• "About KM" page. Thought all the questions are on the same page and expected that "Who are the gurus" is at the bottom.
• confusing.
• Resource page. The categories should be ordered by the frequency of use. He thinks that the order should be Reading list, followed by Case studies ...
• Products page. It may be helpful to have descriptions of each product category. What functionalities in each product category? otherwise may be confusing.
• Search page. Personally he never uses the enhance query feature. Never wanted to figure out how it works. Never reads Search tips or such things.
• Like the look of the site.

Participant 2 [top]

Task1 (5 minutes)
Go to "About KM" page.
Read the contents in the page. Scroll down the screen.
Click on and check some links inside the page.

Task2 (10 minutes)

• Go to Search.
• Type in "intellectual capital" as a phrase.
• Finding the search is too slow, stop it.
• Limit the sites to Gotcha sites and do search again.
• Realize that the scenario asks for books. Go back limit the search sites to Gotcha library and do search again.
• The system did not find any books in Gotcha library that are about intellectual capital.
• Go back to search, add Gotcha sites.
• Go to Resources.
• Check Reading list page.
• Find "intellectual capital" in the page.
• Check periodicals.
• Comment that she usually goes to Amazon if she wants to get book info. She uses web sites usually because there are some  information available online and She can print them out.

Task3 (8 minutes)

• Go to Search.
• Type in "customer service" as keywords. Knowing that the search is slow, limit the sites. Usually if the search is not so slow, she would use the default setting.
• Go to Resources.
• Check Case studies. Comment that it would be nicer if she can search in the page. She does not like to read through the contents.
• Find two sites useful. Maybe print them out.

1. Satisfied.
2. Case studies is a good idea.
3. Resources page the least useful.
4 Satisfied.
5. Easy.
6. Relevant. It would be nice to have summary.
7. No.
8.

• In Search page, the mouse-over pop-up windows are distracting. It would be nicer to have the description of each site on the side.
• In Search page, the option "Web in general" is confusing, "Search the web" might be better.
• Add  "What's New" in Home page, notifying the users what are the latest updates, new contents in the site.

Participant 3 [top]

Task1 (18 minutes)

• Look around the interface. Comment, "looks like there are plenty of resources there".
• Read the contents in the Home page to get a better understanding of the site.
• Try out each tab.
• Click on "About KM".
• Read the contents in the page. Comment, "I may print this page out to my boss. Looks like I have fairly robust content. This page gives coverage to a lot of aspects about KM. Pretty good bibliographies, articles, books".
• Comment that the questions in "About KM" are very helpful typical questions. They are very nice and answer the scenario quite a bit, but as a project manager, he wants to find some specific information about how KM can help him in his project in his business domain. Up to know, has not found anything directly related to project management. He really likes to see a model of how KM can be used to speed the product activity or expectations of the customers or clients
• Check the "Fortune". Comment, "can be used to convince my boss the importance of KM".
• Try "What are the hot topics", hoping that there may be something related to project management.
• Check "Management control concept". Comment, "Probably bookmark this page. This is good, we do these staff. This is something maybe we can do".
• Check  "Glossary", did not find anything related to project.
• Check "Products". "Pretty good classification of products".
• Go back to "about KM" page. "Pretty good definitions. Very useful. Maybe print it out to next meeting. Or probably make some PPT slides out of it"
• Comment "Got general knowledge about KM, but could not find something directly related to project management."

• Look around the Home page for a text entry box to do a search. Did not find it. Then saw the Search tab.
• Click on Search. Then changed his mind.
• Go to "about KM". Found "Intellectual capital" in this page. There was only one link under it. From his previous experience, he thought this link is not very useful.
• Recalling that he saw "glossary" somewhere, try to find "intellectual capital" in the glossary. After clicking on it, got the message "Gotcha library does not have ... ".
• Try Search.
• Type in "intellectual capital" in the query box and click on the go button. Then "Oops, just saw the message 'put quotation marks for phrases'".
• Read the search results. Then go back to Search, type in "intellectual capital" as a phrase.
• Read the results on the first page. Did not find what he is interested.
• Check next page. Got a lot of "untitled" staff.
• Check some search result links. Still want to search for practical definition of intellectual capital. Not much interest.
• Go back and try new searches. Because the search process is so slow, do two searches simutaneously. one is to search for "intellectual capital" " income statement". The other is to search for "What is intellectual capital" as a phrase.
• After clicking on Go button, noticed that there are some other check boxes on the screen.
• Try another search on "business "intellectual capital"".
• Read the search results returned. The first hit is very good, seems that it answers the scenario.

• After looking at the scenario, "like to see some case studies to explain some KM technology components".
• Click on the link "Case studies" in the Home page.
• "Basically I have a front office problem, an interface problem". check the titles of case studies.
• Decide to check HP because they have a reputation of good services. Maybe they have something about sales and customer services. Very general article. Cultural article. Did not find cases here.
• Decide to try a search. Search on  "front office automation".
• Comment, "Personally prefer the check boxes uncheck by default".
• Check the search results returned.
• Found an CIO article that looks like a good case study. Based on this link, may call some people named in this article and ask them what they feel the important things are.

• Expand the usage of graphics. It is more popular and makes the site a little more attractive and pleasing.
• Different interfaces for business, academic and management people. It is difficult ot have one site for all.
• Add a little bit advertising. Implies sponsorship.
• Add a calendar. Current events in this area.
• In home page, imply that it is part of a greater project. it is not end of the story.