The UCWISE Instructor Portal metadata tool

Addressing the Heuristic Evaluation Usability Problems

Here we describe the things we changed/ added/deleted to address the heuristic problems pointed out by the HE team. We first have the description of the problem as put forward by the HE team followed by our response for each problem.

1. [H1 Visibility of system status] (Severity 4)

The sequential circular images at the bottom of each page, indicate that there is some process flow involved between these tasks. However, there are no real visual cues other than the preview table that they are connected. As the user progresses through the stages, there is no feedback on what he/she is doing or what quiz he or she is editing. Additionally, the breadcrumbs on the top of the page signal something totally different -- that there are two levels that don't have a sequential connection to each other.
Our Response:
The circular images is a reference to our progress bar. We addressed this issue by putting the progress bar on top of the screen to make it more visible and create a hierarchical feel for the user. These were suggestions made by UI Patterns. Although, we don’t have intermediate navigation within stages, we are going to keep breadcrumbs implemented in order to allow for future versions of our tool to utilize multi-level navigation.

2. [H3 User control and freedom] (Severity 3)

There are no clearly marked exits when the user is completing tasks. For instance, when the user changes templates to edit the metadata template, there is no clear exit or cancel to the operation.
Our Response:
We combined the templates page with the initial section of Bookkeeping Metadata so there is no longer a secondary level of navigation inside Bookkeeping. Within the newly implemented Complete List of Basic Metadata, we have a button called Update which allows the user to select or deselect metadata to be associated to a given activity.

3. [H4 Consistency and Standards] (Severity 4)

There is no consistent placement of headers, forms or action buttons on each page. Specifically, when moving from one screen to the next, the navigation/action options (edit template, switch template) are inconsistently placed on the page and the design treatment changes. This is confusing as it forces the user to reorient themselves to the page and find where the options are to work with the data. For a specific example, the edit link changes location and treatment between the two screens.
Our Response:
Since we do not have the separate templates area anymore, this problem does not exist on that particular page anymore. However, we did address some of the screen movement by placing all of our metadata selection areas on the left, and the active metadata areas to the center of the screen. We are attempting to make this arrangement consistent amongst all sections of our metadata tagging interface.

4. [H7 Flexibility and efficiency of use] (Severity 3)

It is unclear what the prev | next links correspond to. If this is related to the clickable graphic at the bottom of the page as a way of moving the user through a process, consider a UI pattern called the process funnel in which all steps are visible and the current step is highlighted and you have only one clickable element to move through the process. Also, Navigation components are rarely at the bottom of a page. It needs to be visible to remind users where they are and allow them to easily click to the next step. The current location is not intuitive and will move off the page if it ever gets longer.
Our Response:
We did away with the Prev | Next links because we designed the navigation to be conducted through our progress bar, and breadcrumbs.

5. [H10 Help and documentation] (Severity 2)

The help button at the top right is clear, but it is not clear that it will provide contextual help for the task stage the user is in. There should be help or infomation for the particular items on the page -- using template.
Our Response:
We have not decided what to do with Help. Our users expressed the need for a general Help area, so we are not sure what the context of the Help will be yet.

Scenario 1: Bookkeeping Metatdata

6. [H1 Visibility of system status] (Severity 3)

Once the user sets a default template for a quiz, there is no indication of that. The form option disappears. Which templates are set as defaults should always be visible to the user.
Our Response:
We did away with the separate templates page and combined it with the primary page of Bookkeeping Metadata. The new implementation will keep track of the selections that the user makes for a given Activity, in order to present the user with those selections as the default for the next time they author an Activity of that same type. For example, if a user authors a quiz and adds Format and Source to their Activity, then the next time the user wants to author a quiz the default selection of metadata will include Format and Source.

7. [H1 Visibility of system status] (Severity 2)

Once the user sets the default, the screen changes to something else but there is no confirmation of what happened. The interface should provide some message like Default set or Changes Saved.
Our Response:
Since we do not have a templates page, there is not much to address here. We do, however, have a Save button that provides immediate feedback by placing the metadata into the Preview.

8. [H1 Visibility of system status] - (Severity 4)

On the Bookkeeping Metadata stage, there are no cues about what this page is for. The user is dropped into a Daily Quiz template, but a user would no idea what that means. There is a home page above this page, and I hope that there is a dashboard on that page with more information about the use of templates and default settings.
Our Response:
We have included descriptive information about this page that is intended to be informative and help guide the user.

9. [H1 Visibility of system status] - (Severity 3)

There is no visual connection between changing the template and adding bookkeeping metadata.
Our Response:
We addressed this issue by combining the two separate areas into one in order to simplify our design and make the tagging process easier throughout our interface.

10. [H2 Match between system and real world] (Severity 3)

Language like change the template does not communicate what task it is associated with. Additionally, the use of the terms bookkeeping and metadata are both confusing. I associate bookkeeping with accountants, so in this context bookkeeping metadata to me means information about an account, etc
Our Response:
As of now, we are still holding true to Bookkeeping because that was the term suggested by our users. We do understand that it might not be the best term to associate with the task, but for now, we are staying with the term until we re-examine this through the formal usability study.

11. [H4 Consistency and Standards] (Severity 2)

The Daily Quiz form is laid out differently on the switch template screen, which makes it seem like they are different templates.
Our Response:
This problem got absorbed with the decision to get rid of the separate templates page.

12. [H4 Consistency and Standards] (Severity 3)

Edit, Delete, and the Set As Default Template links are unusual because usually actions like these are buttons rather than links. Do these links lead to pages for editing or deleting or does the action take place right away? The | symbol between both sets of choices should not be the hyperlink color.
Our Response:
This problem got absorbed with the decision to get rid of the separate templates page.

13. [H6 Recognition rather than recall] (Severity 2)

On the Generic CS3 Quiz form, pre-populate the Date Modified field with the current date to minimize burden on the user.
Our Response:
We have already addressed this on the primary page of the Bookkeeping Metadata area.

14. [H6 Recognition rather than recall] - (Severity 3)

The action of changing the template to provide metadata on another quiz should not be in a list amongst actions for the same template. It should be moved up to the same level as using template: DailyQuiz because these actions are switching state between templates while the other actions pertain to the template a user is in.
Our Response:
This problem got absorbed with the decision to get rid of the separate templates page.

Scenario 2: Instructor Notes

15. [H4 Consistency and Standards] (Severity 3)

Standard form layout and functions are not used. The cancel button is at the top of the form, the save button at the bottom. These elements should be together.
Our Response:
We had the cancel button and the save button at different places as they work on different sections of the instructor notes. The cancel button is to cancel the entire tagging activity and the save button was to save the instructor notes. We addressed this by changing the button names. Instead of save we added reset text and a save button at the bottom to save the entire instructor notes metadata tags. For canceling function we added check boxes in front of the tags which can be unchecked to remove the tag from the Active Instructor Notes metadata tagging area.

16. [H4 Consistency and Standards] (Severity 3)

The cancel button does not appear on other forms. This should be consistently applied throughout the application.
Our Response:
We removed the cancel button from this page and changed the other buttons such that the buttons were consistent on all the pages.

17. [H2 Match between system and the real world] (Severity 4)

It is unclear what the difference is between Need to Fix and Need to Edit. Labels need to be more clear. All fields related to editing the same record should be available on one page so users won’t have to switch back and forth.
Our Response:
Need to fix and need to edit are metadata tags that the user can use to tag a particular activity. We realized that with the previous prototype the fact that Need to edit and Need to fix were tags and not any editing feature was not clear to the users. To address this we put checkboxes in front of the metadata tags to indicate that the user can check one or more to tag instructor notes. We also added an Active Instructor notes metadata Tagging area that reflects the tags selected from the list of tags. It also allows the user to add personal notes.

18. [H4 Consistency and Standards] (Severity 3)

On the Instructor Notes Need to Edit page, the breadcrumb does not include the current page, and the previous Instructor Notes page is not hyperlinked. This make it harder for the user to know her current location or back up to the previous step.
Our Response:
As of now, we do not have multiple levels of navigation within individual sections of our interface. However, we are keeping the breadcrumbs implemented so that future versions of our tool can take advantage of their functionality. That is most likely when the context problems regarding the breadcrumbs will get resolved.

19. [H8 Aesthetic and minimalist design] (Severity 2)

Minimize clutter by not repeating Need to in every option on the menu. If you do, then capitalize the word to consistently.
Our Response:
Need to fix and need to edit are metadata tags that the user can use to tag a particular activity and not any editing feature. Thus we could not remove the need to part of the tag. But to make it more clear to the user that they were tags and not editing feature we added checkboxes in front of the tags.

Scenario 3: Cognitive Learning Metatdata

20. [H2 Match between system and the real world] (Severity 1)

The colored bar moves vertically as the list expands. This might falsely indicate something is going on with the data on the screen. The bar should be in a fixed position unless it does signify some change on the screen.
Our Response:
To address this we completely removed the colored vertical bar and replaced it with check boxes for each metadata.

21. [H4 Consistency and standards] (Severity 3)

It might be clearer to put the preview window on the right side of the page. Since all the other actions proceed from left to right, it is inconsistent to have the results of actions show up on the left side of the page.
Our Response:
We moved the preview window to the right side of the page to address this issue. We also changed the color of the preview page as compared to the rest of the page to indicate that the preview is not editable and has a different purpose from the rest of the page.

22. [H5 Error prevention] (Severity 4)

It is unclear what the goal of this screen is. What are the actual tags being applied to the quiz questions? The labeling and instructions need to be more clear.
Our Response:
To address this we added a one line of text describing the purpose of the page. We completely removed the idea to displaying the activity questions as of now.

23. [H5 Error prevention] (Severity 3)

It is confusing to first click a word and have the + symbol become a link later. This is very difficult to see and it is unclear what happens when you do this beforehand.
Our Response:
We replaced the + sign with twisties (as suggested by the HE group). The twisties in front of the metadata indicate that the metadata is expandable. Now clicking on the metadata tag or the twisty results in the metadata being expanded to display the sub categories of the metadata. We also put checkboxes in front of each metadata tag to indicate that the checkboxes can be checked in order to tag it with the desired metadata.

24. [H7 Flexibility and efficiency of use] (Severity 4)

The task flow is not clear on this page. The relationship between the columns on the page is not clear. For instance, does each of the steps of the quiz have the same optional elements in the learning spectrum? If so, there should be a more efficient way to associate these tasks with actions.
Our Response:
We completely removed the display of activities window (a suggestion from our instructor) and made the tagging process simpler. Tagging individual activity questions can be made a part of an extended future project for this project we narrowed the scope to tagging the activities with an overall metadata for an activity.

25. [H7 Flexibility and efficiency of use] (Severity 2)

This scenario requires a large number of clicks to get through it. It would be ideal to minimize the steps required to tag quiz questions. Reduce clicks by allowing the user to edit directly within the context of where they are clicking instead of a separate space on the page for all editing functions.
Our Response:
To address this we simplified the entire tagging process and removed the feature of tagging individual questions with metadata. Now the user can tag an overall activity with few clicks.

26. [H7 Flexibility and efficiency of use] (Severity 1)

No clear affordance on the Modification link. Probably better to have a set of links next to it that say edit or something like that.
Our Response:
We resolved this by making the selection of metadata consistent with all other forms of selection throughout the interface. We put checkboxes, and we also added hierarchy arrows to indicate that there are subcategories that can be selected. Having this arrangement is also a test on our part to see if having the both of these will prove useful and not be confusing. We have a predicament where the user can tag the activity with just Modification, for instance, or they can tag the activity with a sub-category of Modification. So, at the moment, this is our solution.

27. [H7 Flexibility and efficiency of use] (Severity 1)

The blue panel displays the same text as what is in the edit box as well as what is in the Learning Spectrum list. This seems redundant and potentially confusing. Allow the user to edit the phrase directly and once instead of displaying it as editable and not editable in the same panel.
Our Response:
We moved the cognitive learning spectrum metadata to the left of the page.

28. [H8 Aesthetic and minimalist design] (Severity 1)

The rainbow of color on this page seems to be clutter unless it conveys some information. A rainbow of clutter is really attention grabbing and should only be used if it helps the user somehow.
Our Response:
We removed to rainbow colored vertical column as it was confusing.