Method
Participants:
The three participants were selected because they are the prospective users of the Metadata Tagging tool.
| Participant 1 | Participant 2 | Participant 3 | |
|---|---|---|---|
| Gender | M | M | M |
| Occupation | Post-doc, lecturer, researcher | Computer science instructor, graduate student | Sr. lecturer, UCWISE principal investigator, researcher |
| Usage of the current UCWISE metadata tagging tool | rarely | rarely | Some times |
| Comfort Level with Computers | very comfortable | very comfortable | very comfortable |
| Number of Times that have participated in the UC-HIPO user study previously | 2 | 2 | 1 |
Apparatus:
We brought a consent form for the each participant to sign. We used a tape recorder to record the entire session after we asked if they feel comfortable with the session being tape recorded. The studies were conducted at several locations: a laptop in the office of participant 1; a desktop computer in the Soda Hall second floor lab of participant 2; and the faculty office in Soda Hall of participant 3. We also brought a follow-up interview questionnaire for each participant to fill out.
Task Observations:
Scenario one: Bookkeeping Metadata Tagging
- Are the users able to find the correct section of our Metadata Tagging Tool to place these types of tags? In other words, are the navigation bar and the corresponding descriptions causing major amounts of exploration before the users start their task?
- Are the users able to select from the different activity forms using the activity selection pull down menu?
- Do the users realize that they can add metadata to the selected form using the Additional metadata checklist?
Scenario two: Instructor Notes Metadata Tagging
- Are the users able to find the correct section of our Metadata Tagging Tool to place these types of tags? In other words, are the navigation bar and the corresponding descriptions causing major amounts of exploration before the users start their task?
- Are the users selecting the metadata from the checklist without confusion?
- Do the users understand that the personal notes input form can be used to attach personal notes to the selected metadata?
Scenario three: Cognitive Learning Metadata Tagging
- Are the users able to find the correct section of our Metadata Tagging Tool to place these types of tags? In other words, are the navigation bar and the corresponding descriptions causing major amounts of exploration before the users start their task?
- Are the users clicking the expand icons to view subcategories of the cognitive learning metadata?
- Are the users selecting the metadata from the checklist without confusion?
- Do the users understand that the personal notes input form can be used to attach personal notes to the selected metadata?
Link to the Actual usability study script on the 3 scenarios
Procedure:
We presented each user with the same interactive prototype and scenarios to conduct the testing.Steps:
- Informed the user of what we were testing.
- Gave a brief demonstration of the interface, showing the user the areas of the tool that are developed.
- Asked the user to perform the given tasks as we read them through each scenario, encouraging them to think aloud as they go.
- Presented the user with a follow-up questionnaire to fill out and encouraged them to think aloud as they answered each question.
- Asked the user additional follow-up questions and allowed for any additional comments regarding our tool.
Screenshots
Overview of the Metadata Tagging Tool
The Bookkeeping Metadata Tagging Section
Selecting the Quiz Template in the Bookkeeping metadata tagging section
The actual quiz template in the Bookkeping metadata tagging section
The Cognitive Learning Metadata tagging section
The Cognitive Learning Metadata tagging section with the metadata: modification expanded
The Instructor Notes metadata tagging section
