MaNIS Interface Project
Assignment 6 - Heuristic Evaluation
Contents
Individual Evaluations of Paparazzi
Mayjane Co's Heuristic Evaluation of Paparazzi
1. [H1 Simple & Natural Dialog] (Severity 2)
The interface uses color to indicate where a post begins and ends but the color contrast is not sharp enough to be effective. Increase the color contrast.
2. [H1 Simple & Natural Dialog] (Severity 2)
Are Activity and Context related? If so, the Results Page does not readily convey this. Group together related sections by using color or enclosing them in a box.
3. [H1 Simple & Natural Dialog] (Severity 2)
Posted date is usually grouped with title and author for many publications. Move posted date in the proximity of author and title.
4. [H2 Speak the User's Language] (Severity 2)
Unclear what the word Activity means as a selection in the Sort Dropdown. Change it to something more explicit like "Number of Related Postings."
5. [H2 Speak the User's Language] (Severity 2)
Unclear what the phrase "1 link in this post" means under Activity. Does it mean the current post is linked to 1 other post or 1 other post is linked to the current post? The direction is not apparent.
6. [H4 Consistency] (Severity 3)
Inconsistent use of terms. The Context Page says "linked to..." while Results Page says "responses to..." Unclear if these phrases are supposed to mean the same thing.
7. [H5 Feedback] (Severity 2)
The "Go" button associated with the Sort feature seems unnecessary. User selection of a sort parameter should trigger the sorting immediately.
8. [H5 Feedback] (Severity 2)
Unclear how matching of a search word like "Clarke" is done for the different sections - Blog, Topic, and Post. Highlighting (in bold or red) the occurrence of the search word in the result set might help users clearly see the difference between a "Blog that matches Clarke", "Topic that matches Clarke", and "Post that matches Clarke."
9. [H6 Clearly Marked Exits] (Severity 2)
No way for a user to go back to the home page. Provide a way to get back to the home page.
10. [H7 Shortcuts] (Severity 2)
Hyperlinks to navigate subsets of a large result set are provided at the bottom of the page. A user who does not scroll to the bottom may never become aware of the feature. Put navigation both at the top and bottom of the page.
Note: Couldn't figure out how to execute task 1.c.
Denise Green's Heuristic Evaluation of Paparazzi
1. [H1 Simple and Natural Dialog] (Severity 3)
Initially, it's unclear how to tell which post has been linked to by the most bloggers (task 1b). Eventually I found it under "show context of this blog" (while I was browsing through the interface).
2. [H1 Simple and Natural Dialog] (Severity 3)
The blog statistics are hard to find. The tooltip for the icon says "Profile," which doesn't necessarily suggest statistics.
3. [H1 Simple and Natural Dialog] (Severity 1)
The profile and context icons are hard to interpret without clicking on them.
4. [H1 Simple and Natural Dialog] (Severity 2)
The Post context page could benefit from a graphical display of the relationships between the different blog entries.
5. [H2 Speak the User's Language] (Severity 3)
The 3 section headings on the search results page are too terse: Blogs, Topics, and Posts. It would be nice if there was more information about how these sections are related to the search term (e.g. Posts containing the term "Clarke").
6. [H2 Speak the User's Language] (Severity 2)
Unclear what "Activity" means in the "Sort results by" control.
7. [H2 Speak the User's Language] (Severity 2)
Unclear what "Registered Topics" means on the blog statistics page.
8. [H2 Speak the User's Language] (Severity 2)
Unclear what "Paparazzi Rank" means on blog profile page.
9. [H2 Speak the User's Language] (Severity 2)
Unclear whether "Average number of links per post" on the blog profile page refers to links to the blog, or links from the blog.
10. [H4 Consistency] (Severity 2)
The profile page shows some summary information about context (about the blog in general), while the context page shows detailed information about context (about this particular blog entry). Should all of this information about context be related/connected/grouped in the interface?
11. [H4 Consistency] (Severity 1)
Previous/Next controls at bottom of page are easy to miss. They should probably also appear at the top of the page.
12. [H6 Clearly Marked Exits] (Severity 1)
It might be nice to provide links to get back to the search results page from the context page (rather than using the browser Back button).
Jane Lee's Heuristic Evaluation of Paparazzi
1. [H1 Simple and Natural Dialogue] (Severity 2)
It may be difficult for users to tell when one post begins and one ends. The background color change is too subtle.
2. [H1 Simple and Natural Dialogue] (Severity 2)
The title and source hyperlinks should open new windows. Currently, they open in the current window. Users may not want to leave Paparazzi and may not know how to get back.
3. [H1 Simple and Natural Dialogue] (Severity 3)
The visual similarity of the text makes it difficult for users to distinguish between different elements of content.
4. [H1 Simple and Natural Dialogue] (Severity 3)
Users may find it difficult to navigate the site. Is the topics hierarchy meant to aid in navigation of the site? The breadcrumb structure seems to indicate that it does, but I'm not certain that was the intention.
5. [H1 Simple and Natural Dialogue] (Severity 1)
I don't think the "Go" button next to the "sort results" pull-down menu is necessary. The selection of a menu item could automatically trigger the event.
6. [H1 Simple and Natural Dialogue] (Severity 1)
The source URL at the bottom of a search result is not labeled. Users may not realize what it refers to.
7. [H2 Speak the Users' Language] (Severity 3)
Newcomers to the blogging world and to the site may not know what all the terms mean. For example, what's the difference between blogs, topics, and posts?
8. [H4 Consistency] (Severity 2)
Activity seems to be related to or the same as Context. Should the two be combined somehow for the sake of consistency?. Users may miss the relationship otherwise.
9. [H5 Feedback, H3 Minimize Users' Memory Load] (Severity 2)
Currently there is no indication of what the user entered as a search term. It would be helpful for users to know what they entered when looking at a result set.
10. [H10 Help and Documentation] (Severity 3)
Tool tips for hyperlinked icons would help users figure out what they are. Without some sort of help documentation, they may confuse some users. In general, a good help section will be essential for Paparazzi. A tagline under the site name may also help users figure out what the purpose of the site is.
Rebecca Shapley's Heuristic Evaluation of Paparazzi
1. [H1 Simple and natural dialogue] (Severity 3)
Icon for Paparazzi profile is cryptic. Also, the natural tendency is to click on the noun itself, the blog name, which goes outside of Paparazzi. If the most desired behavior is first to see the Paparazzi profile, then I suggest either a) placing the icon in front of the blog that it refers to b) making the blog name the link to the Paparazzi profile, and providing an icon link for going to the real thing, or c) replacing the icon with the text link "Paparazzi profile" or "profile".
2. [H3 Minimize user's memory load] (Severity 3)
Reducing the screen real-estate taken up by the post summary so more results fit on a page without scrolling. Suggest re-arranging post date, activity, and context information at the bottom of each search result. Using less space allows the user to scan more results and assess current search without scrolling. Think Google results, perhaps especially the 4th line.
3. [H5 Provide feedback] (Severity 3)
Provide feedback about why a particular posting is included in the search results. For example, if I search for "bush" show "Bush" in bold in the text of the resulting blog entries. KWIC. If I follow activity links, show the part of the posting that links to other posts, or the part of the responses that reference the current post.
4. [H4 Consistency] (Severity 2)
On the search results display page, under the Blogs heading, a "Paparazzi profile [ICON]" link is displayed. Under the Posts heading, the link to the profile for the blog is only available via the icon, and the underlined link before the icon leads outside of Paparazzi.
5. [H4 Consistency] (Severity 2)
On the Blog Profile screen, the two columns provide two different lists; on the Topic Details page, the two columns under "Frequent bloggers on this topic" provide one list.
6. [H1 Simple and natural dialogue] (Severity 2)
Post Context page: User thinks chronologically, but summaries are posted by category. ... perhaps "links to" comes first? Suggest that 'link to' and 'responses' could have some good icons. Even just ---> and ----> , used consistently in the page layout, could convey following the "flow" of the conversation.
7. [H3 Minimize user's memory load] (Severity 2)
The "context" icon does not provide immediate recognition. Requires memorization and/or text interpretation. Suggest simple text link to access the feature: "context" or " context of this post".
8. [H6 Clearly Marked Exits] (Severity 2)
From the Blog Profile page for The Joe Hill Dispatch, there is no clear path back to the front Paparazzi page. On the Topic Profile:News page clicking the Paparazzi icon works. Also, "Search" is available from any page. These are both good functionality, but recommend also providing an explicit link to "restart" using topic terms.
9. [H9 Prevent errors] (Severity 2)
Remove "go" button for "Sort results by" feature. Sort should act once menu item is selected. Otherwise menu, which doubles as a display of the active choice, could be out of sync with the type of sort displayed for the results.
10. [H2 Speak the users' language] (Severity 2) - applies to task scenarios only.
From task scenarios, I was confused about terminology in 1b: "open this blog" - assumed that it meant something to do with the prototype, but kept getting led out to the msn.com site. Took several tries before found icon linking within Paparazzi.
11. [H4 Consistency] (Severity 1) - applies to current data only.
On the Blog Profile page, the linkers and respondants are entered as "BloggerOne" etc. The distinction between "blog" and "blogger" is confusing, and seems to refer to the same thing for the purposes of the application.
12. [H1 Simple and natural dialogue] (Severity 1)
On the Blog Profile page, there is a lot of visual layout for the user to understand. Suggest trying a re-ordered format to avoid creating more visual lines on the page than necessary: (47 posts) BloggerOne[ICON] or (47 posts) [ICON] BloggerOne
13. [H1 Simple and natural dialogue] (Severity 0) - feedback requested by message within prototype
On the Topic Details: News page, I unchecked "politics" and clicked "filter posts". I expected the screen to refresh with the more limited results set, and the still-checked subtopics to be implemented at the top of the screen. In this case, I'd see News>US. Later, within the News>politics subtopic, I saw only one check box, for politics. I expected that unchecking it would take me back to the News topic, that is, it would undo the limitation I had selected by clicking the news>politics subtopic link. (Or I could click the News link, but that's all the way at the top of the page.)
Heuristic Evaluation of Paparazzi by Manis
Heuristic Violations
Please note that this is a summary of the various violations found by the evaluators. Their individual reports should be consulted for finer details and suggestions.
| Heuristic | Violation | Severity | No. Evaluators |
|---|---|---|---|
| H1 Simple and Natural Dialog | Initially, it's unclear how to tell which post has been linked to by the most bloggers (task 1b). Eventually I found it under "show context of this blog" (while I was browsing through the interface). | 3 | 1 |
| H1 Simple and Natural Dialog | The blog statistics are hard to find. The tooltip for the icon says "Profile," which doesn't necessarily suggest statistics. | 3 | 1 |
| H1 Simple and Natural Dialog | The profile and context icons are hard to interpret without clicking on them. | 2 | 2 |
| H1 Simple and Natural Dialog | The relationships between the different blog entries on the Post Context page are confusing. | 2 | 2 |
| H1 Simple & Natural Dialog | The interface uses color to indicate where a post begins and ends but the color contrast is not sharp enough to be effective. Increase the color contrast. | 2 | 2 |
| H1 Simple & Natural Dialog | Posted date is usually grouped with title and author for many publications. Move posted date in the proximity of author and title. | 2 | 1 |
| H1 Simple and Natural Dialogue | The title and source hyperlinks should open new windows. Currently, they open in the current window. Users may not want to leave Paparazzi and may not know how to get back. | 2 | 1 |
| H1 Simple and Natural Dialogue | The visual similarity of the text makes it difficult for users to distinguish between different elements of content. | 3 | 1 |
| H1 Simple and Natural Dialogue | Users may find it difficult to navigate the site. Is the topics hierarchy meant to aid in navigation of the site? The breadcrumb structure seems to indicate that it does, but I'm not certain that was the intention. | 3 | 1 |
| H1 Simple and Natural Dialogue | I don't think the "Go" button next to the "sort results" pull-down menu is necessary. The selection of a menu item could automatically trigger the event. | 1 | 2 |
| H1 Simple and Natural Dialogue | The source URL at the bottom of a search result is not labeled. Users may not realize what it refers to. | 1 | 1 |
| H1 Simple and Natural Dialog | The format of columns of linked blogs on Blog Profile page may be adding unnecessary visual confusion. | 1 | 1 |
| H2 Speak the User's Language | The 3 section headings on the search results page are too terse: Blogs, Topics, and Posts. It would be nice if there was more information about how these sections are related to the search term (e.g. Posts containing the term "Clarke"). | 3 | 3 |
| H2 Speak the User's Language | Unclear what "Registered Topics" means on the blog statistics page. | 2 | 1 |
| H2 Speak the User's Language | Unclear what "Paparazzi Rank" means on blog profile page. | 2 | 1 |
| H2 Speak the User's Language | Unclear whether "Average number of links per post" on the blog profile page refers to links to the blog, or links from the blog. | 2 | 1 |
| H2 Speak the User's Language | Unclear what the word Activity means as a selection in the Sort Dropdown. Change it to something more explicit like "Number of Related Postings." | 2 | 2 |
| H2 Speak the User's Language | Unclear what the phrase "1 link in this post" means under Activity. Does it mean the current post is linked to 1 other post or 1 other post is linked to the current post? The direction is not apparent. | 2 | 1 |
| H3 Minimize User Memory Load | Currently there is no indication of what the user entered as a search term. It would be helpful for users to know what they entered or clicked on when looking at a result set. | 3 | 2 |
| H3 Minimize User Memory Load | Reduce the screen real-estate taken up by the post summary so more results fit on a page without scrolling, allows the user to scan more results and assess current search. | 3 | 1 |
| H4 Consistency | The profile page shows some summary information about context (about the blog in general), while the context page shows detailed information about context (about this particular blog entry). Should all of this information about context be related/connected/grouped in the interface? | 2 | 1 |
| H4 Consistency | Inconsistent use of terms. The Context Page says "linked to..." while Results Page says "responses to..." Unclear if these phrases are supposed to mean the same thing. | 3 | 1 |
| H4 Consistency | Activity seems to be related to or the same as Context. Should the two be combined somehow for the sake of consistency?. Users may miss the relationship otherwise. | 2 | 2 |
| H4 Consistency | On the search results display page, under the Blogs heading, a "Paparazzi profile [ICON]" link is displayed. Under the Posts heading, the link to the profile for the blog is only available via the icon, and the underlined link before the icon leads outside of Paparazzi. | 2 | 1 |
| H4 Consistency | On the Blog Profile screen, the two columns provide two different lists; on the Topic Details page, the two columns under "Frequent bloggers on this topic" provide one list. | 2 | 1 |
| H6 Clearly Marked Exits | Provide links to get back to the search results page and home page (to start topic navigation) from the other pages (rather than using the browser Back button). | 1 | 3 |
| H7 Shortcuts | Hyperlinks to navigate subsets of a large result set are provided at the bottom of the page. A user who does not scroll to the bottom may never become aware of the feature. Put navigation both at the top and bottom of the page. | 2 | 2 |
| H10 Help and Documentation | Tool tips for hyperlinked icons would help users figure out what they are. Without some sort of help documentation, they may confuse some users. In general, a good help section will be essential for Paparazzi. A tagline under the site name may also help users figure out what the purpose of the site is. | 3 | 1 |
Total Violations Per Heuristic
| Heuristic | Total | % of Total |
|---|---|---|
| H1 Simple & Natural Dialog | 12 | 43% |
| H2 Speak the User's Language | 6 | 21% |
| H3 Minimize User Memory Load | 2 | 7% |
| H4 Consistency | 5 | 18% |
| H5 Feedback | 0 | 0% |
| H6 Clearly Marked Exits | 1 | 4% |
| H7 Shortcuts | 1 | 4% |
| H8 Precise Error Messages | 0 | 0% |
| H9 Prevent Errors | 0 | 0% |
| H10 Help and Documentation | 1 | 4% |
Total Violations Per Severity
| Severity | Total | % of Total |
|---|---|---|
| 4 | 0 | 0% |
| 3 | 9 | 32% |
| 2 | 15 | 54% |
| 1 | 4 | 14% |
| 0 | 0 | 0% |
Overall Recommendations
Overall we like the idea of Paparazzi, and we think that you are moving in the right direction with your prototype. You made good decisions in the re-design of your paper prototype. We like the fact that entering a search term gives three different types of information about the term: blogs, topics, and posts. It's a nice way of revealing the complexity of searching in this domain, without overwhelming the user with details at the beginning. It's also clear that you've thought about what details will be meaningful to users, and how to provide tools for leveraging those details.
Paparazzi is clearly trying to solve a complex problem, and so we'd expect that several iterations of the interface are needed. As you mentioned in your project proposal, helping the user develop a meaningful mental model of the archived blog domain is the key challenge. Important areas for refinements in the interaction model of the current interface include providing feedback for the user's search and navigation criteria, streamlining navigation, and continuing to reduce user confusion about terminology.
The majority of heuristic violations that we found were in the area of Simple and Natural Dialog. Most of these had to do with trouble navigating through the various pages of the application. It wasn't obvious how to get to the Profile page, and it wasn't intuitive to tell what was happening in the Context page. The relationships between the different pages were unclear. Better use could be made of search criteria feedback, color, font, and grouping to help direct the interaction flow.
The next largest groups of HE violations are in the area of Speaking the User's Language and Consistency. Some terminology needs to be clarified and made consistent, and in some cases (for example, the three main sections: Blogs, Topics, and Posts) the terminology is terse and could be improved by additional explanation in the interface.
We think that the key areas for improvement are the navigation and terminology. It also seems important to provide good user documentation. We encourage you to take a look at streamlining the display of blog post search results in order to provide the user the ability to evaluate the match between the current search results and their search intentions. We also encourage you to continue to develop ways to help the users develop good mental models of the relationships between the various postings they are navigating.