Participant 3 focused most of his time on the metadata template feature, trying to figure out what it can do for him, and trying to explore as many possibilities as possible. It seems Participant 3 had in mind very specific tasks that he wants to accomplish, and he wants to make sure our metadata tagging tool allows him to do these tasks easily.

Participant 1 followed through most of the scripts, but he was also interested to explore many possibilities with the interface, and gave us ideas on how to improve the interface.

Participant 2 was good at following through all the scripts. He explored some of the features that the system has to offer. He gave some good ideas in the end. He commented that he is the most likely person to add metadata being the UCWISE researcher, and instructor.

	Tasks	Participant 1	Participant 3	Participant 2
1	Using the multi-step navigation interface to tag metadata for a given lesson unit	- Finally understood the multi-step navigation concept in the end + When understood it, he find it clever - it was not intuitive to him that he can jump on the steps by clicking on the circles	* this feature did not stand out to him in the beginning, but later he figured out how to use it	wanted to tag the specific metadata difficulty to the quiz +played around with one screen to the next and back to try to figure out the system, this showed that he understood the features
2	Adding a new metadata that is not shown on the default metadata tagging page to a given lesson unit	- Find it hard to navigate the page to go back to the basic quiz info page;	N/A	- after a lot of playing around, he finally find how to add the tag difficulty
3	Using the basic Quiz Info page	+ it was clear to him that this page contains a default set of metadata, and not all the metadata that he was looking for; - not sure how to delete metadata, can’t find the remove feature	+ it seems to be clear to him	- Doesn't find the Basic Quiz Info title to be intuitive
4	Using the Advanced Quiz Info page to add more metadata	+ like the detailed metadata presented on the page, recognize it is the Dublin core standard - recognize that not all instructors may understand the Dublin core standard	- Find it hard to navigate the page to go back to the basic quiz info page	N/A
5	Using the free form feature to add a new metadata	+ like this feature + Love the synonym ring feature; - notice that the interface does not make it clear to him when the typed in metadata matches the exact name of an existing metadata, or if it does not have any synonyms associated with it	N/A	+ Love the synonym feature
6	Customizing the default page for adding basic quiz metadata	+ it was clear to him to use the template feature for this	+ clear to him to use the template feature	+ clear to use the template feature
7	Creating a lesson unit (e.g. quiz) template	+ find it clear	- Confused about what to put in the metadata tag input box - Find it hard to locate all the desirable metadata to place on his template	+ find it clear
8	Using a lesson unit (e.g. quiz) metadata template	- He was confused about whether the template he applied will replace the default page for a particular lesson unit, or for all lesson units of a given type (e.g. quiz);	- He did not notice there was an edit button that allows him to edit a given quiz template	+ understood the flow -had difficulty in finding the exact metadata difficulty +finally found it and was happy
9	Editing a lesson unit (e.g. quiz) metadata template	- Confused about how to add default values to the different metadata tags	- find it not intuitive to find all the metadata that he wants to add to his lesson unit template	+Does not have any difficulty understanding the template design
10	Tagging a lesson unit with domain-specific (topic) metadata	+ the interface was intuitive and clear	+ the interface was intuitive; nice!	- does not find the faceted arrangement of the topics intuitive; feels that a hierarchical drop down would be better.
11	Previewing metadata	+ find it straightforward + love the preview feature	+ find it useful	+ Love being able to see all the metadata attached to a lesson unit on the side window, and side by side during authoring
12	Tagging a lesson unit with instructor info	+ Find it intuitive and clear	- find it hard to find a particular metadata by looking at the “instructor notes” metadata facets	- does not find the arrangement of the topics in the instructor info. page intuitive. - finds browsing through the topics time consuming.
13	Locating the lesson unit authoring page using the multi-step navigation interface	+ Find the interface clever	N/A	+likes the authoring page along with the preview button.

Suggested New ideas

	Participant 1	Participant 2	Participant 3
Having a search box to search for new metadata	X	X	(This would be useful for him since he had a hard time finding the domain specific metadata he was looking for.)
Show recently used metadata	X	X	N/A
Have a final confirmation page that shows all the metadata on a given metadata	X	N/A	N/A
Able to edit the metadata in the Preview window directly	X	X; want to have a light weight interface to add topics to the lesson unit, (e.g. a multi-layer pop up window that lets him select metadata quickly)	N/A

Other comments

	Participant 1	Participant 2	Participant 3
Find the topic and instructor notes layout confusing	no	X, It is not an interface problem, but an organization problem	no comments
Each metadata element should have a date associated with it	X	N/A	N/A
Every page should have a back button	X	N/A	N/A
Likely to skip all the buttons, and just go to the authoring page directly	* so the solution is to make the metadata tool relatively easy to use	X; * I am the most likely person to add metadata,	(made no comments about this)
Should be able to add/revise metadata after the instructor have started authoring; (e.g. being able to jump back and forth easily to edit the metadata)	X; must have feature	X; must have feature	(made no comments about this)