Interactive Prototype 2 Test Script

[Return to Assignment 9]

1. Administer pre-test questionnaire

• Have you heard of the term "Knowledge Management"?

• Where have you heard the term before?
• What do you think KM is?
• What is the difference between KM and information systems?
• In the past week, how often have you used the Web to…
• view web sites? (multiple times a day, once a day, a few days a week, once a week)
• conduct searches? (multiple times a day, once a day, a few days a week, once a week)

2. Describe the purpose of the observation.

• Thank you for giving us an hour of your time.
• You’re helping us by trying out this user interface.
• We’re testing this product; we’re not testing you.
• We’re looking for places where the user interface may be difficult to use.
• If you have trouble with some of the tasks; it’s the interface’s fault, not yours. Don’t feel bad; that’s exactly what we’re looking for.
• If we can locate the trouble spots, then we can go back and improve the interface.

3. Tell the user that it’s OK to quit any time.

• Remember, this is totally voluntary. Although we don’t know of any reason for this to happen, if you become uncomfortable or find this objectionable in any way, feel free to quit at any time.

4. Explain how to "think aloud."

• We have found that we get a great deal of information from these observations if we ask people to think aloud as they work through the exercises.
• It may be a bit awkward at first, but it’s really easy once you get used to it.
• All you have to do is speak you thoughts as you work.
• If you forget to think aloud, I’ll remind you to keep talking.
• Would you like me to demonstrate?

5. Describe the task and introduce the product.

• The user interfaces we’re testing are web pages to a knowledge management portal.
• We don’t expect you to have any background or understanding of knowledge management. In fact, we are working mainly with novices who have no background in this discipline.
• I will pass you written instructions on tasks we want you to perform. Use the Gotcha web site, to the best of your ability, to satisfy the goal we give you. Let us know when you feel satisfied that you have met that goal. If you feel that the site cannot help you meet that goal, tell us and we’ll move on. During the whole exercise, Hailing will be taking notes keeping track of features that appear to be intuitive as well as the interface’s problem areas.

6. Explain that you will not provide help.

• As you’re working through the exercises, we won’t be able to provide help or answer questions. This is because we want to create the most realistic situation possible.
• Even though we won’t be able to answer most of your questions, please ask them anyway. Hailing will note you questions and I’ll answer them later.

7. Ask if there are any questions before you start.

• Do you have any questions before we start?

8. Have the participant sign the Informed Consent Form.

9. Give participant a demo of the site.

• Gotcha is a web portal to information resources on knowledge management. You’ll find both print and on-line resources within the site. Print resources will have bibliographic information whereas on-line resources will have a hyperlink.
• To navigate, just click on the image maps. Clicking the tabs – Home, About KM, Resources, Products, and Search – take you to the appropriate section. The blue column on the left-hand side is also clickable so that you can drill down further. For instance, if you are on the Products Page you can click Consultants to jump right to that information.

10. Start test.

Our guess:

• Since she knows little about KM, he decides to read information in About KM first to get familiar with the area.
• She may follow some of the links provided after the brief explanations.
• She’ll then move on to explore the Resources section and see if there’s anything of interest there.

Our guess:

• The user has some keywords in mind for these topics so she’ll go directly to the Search tab.
• She’ll type the phrase "intellectual capital" and not select the query enhancement feature.
• The search results present some records of interest that the user may explore The user keeps clicking interesting records until she either runs a new query altogether or she has enough information.

Our guess:

• Since the question is pretty vague, she has no good keywords to start with. She decides to explore the About KM section for a brief overview of the subject.
• She explores the Resources tab next.
• She thinks that the Case Studies page would provide a good starting point.
• She scans the case study titles and clicks those that seem interesting.
• She continues reading these studies until she either runs a query on the Search page or feels confident that KM will/ will not meet her needs.

11. Administer follow up questionnaire

12. Conclude the observation.

• You’ve just finished you last exercise.
• We appreciate the time you’ve spent helping us understand how novices might search for information on our site by either browsing or employing the full-search interface. We also wanted to see whether novices might take advantage of the folder feature to compile and save their on-line work.
• Answer any questions the participant had during or after the test.
• Discuss any interesting behavior we would like the participant to explain.