Healthy Communities

Heuristic Evaluation

Evaluators

Hong Qu

Kari Holmquist

Paulette Pan

Severity rating key

0. Not a usability problem.

1. Cosmetic problem

2. Minor usability problem

3. Major usability problem

4. Usability catastrophe

Overall Evaluation

The site does a decent job of supporting the three scenarios specified, although it does not support them completely. You may need to take a fresh look at your scenarios and test with sample users from your actual target user groups (rather than asking people to imagine that they are from another profile). We like the graphical nature of the site, and the fact that users can get an overview and drill down into the site for more information. Also, the concept of the Web dialog potentially helps build the community, as it gets some people talking. How will you guarantee that the people posting are from that community? How will you encourage people to post?

Specific Heuristic Issues

Following are the issues that we found to be of greatest concern from a heuristics perspective. More specific details and other examples can be found in the Appendix, which presents a complete list of our evaluations.

Log-In/Registering

The log-in/registering process and functionality of the site should be very clear. Right now it is very unclear. None of us could determine how the log-in process flow worked, or which features required a log-in versus which didn’t. Can one still surf information before logging in, or only after logging in? We didn’t know. We also felt that after registering or logging in, the user should be taken back to the page he/she was working on. Right now, the registration process goes to a dead end.

When the user clicks on “Registration” (from the left hand column), he/she is taken to a page that makes them choose “Register” again from a list whose options don’t make sense (Login, Register, Retrieve lost password). Why is the user looking at logging out or finding a password here when they presumably have not even signed up or logged in yet?

We noticed that there are two places for users to log in, and that the top log-in link didn’t really work. The log-out link is found in the left menu, which is problematic as the user has to look in different places to find log-in and log-out.

Email/Posting Features

Because this is one of your major scenario items, we felt it was important to make it clear from the home page how to accomplish these desired tasks. However, we couldn’t figure out how to email information or post without first having to drill down for information. And some pages didn’t have an email link at the bottom. After we emailed a page, there was no confirmation page. We weren’t sure if our email was sent out.

Search Function

The search function is a crucial part of any website. We found the search delimiters to be techy and confusing (_ALLWORDS , _ANYWORDS; Search all words [AND] and Search Any Words [OR] ). Also, we felt that the search function should be only the simple box on the home page, with perhaps an option for an advanced search. We also weren’t sure what we were searching for – Indicators? Articles? Locations? Currently, the search results page from the search box query on the home page is another search box and list of categories under which to search. It’s too much work to enter a term in the home page search box, then get another search page where the user has to refine the search. The user should have the choice upfront whether to do a simple or more advanced search.

Consistency and Navigation

We found that some links that leave the site are not consistently labeled. For example, the data source: scorecard.org link indicates the user will go to a third party site; but the view list links did not tell us that we were leaving the Healthy Communities site.

We were also confused in some cases by the right sidebar. For example, when viewing the “Compare Across Communities” page, the “Review Past History” link on the right sidebar is confusing. Does this refer to past comparisons of all counties? Because the right sidebar isn’t visually/graphically separated from the page contents, the tendency is to think the two are related. Perhaps by separating the sidebar more forcefully from the page, this link would make more sense and make the user see that the sidebar elements relate to the Air Pollution category as a whole, rather than the county comparison page.

Terminology/Labeling

We thought the “Community Indicator Dashboard” as mentioned in your report is not the greatest choice of terminology. First, it seems very wordy. Second, “Community” is rather misleading; we felt “Quality” might be a more accurate word. Also, “dashboard” in part indicates that it’s the interactive control center for these issues/”indicators”, but in fact, it’s really a diagnostics readout.

We found the labels on the graphs and scales to be insufficient. What do the numbers mean, what are they being compared against, what is the unit? (E.g. GDP “average value is 256” is pretty meaningless). Why are the dials scaled with little notches per Red-Yellow-Green section rather than just Red, Yellow or Green? Those notches imply that those measures are being compared against something. What are they being compared against?

Graphic Design

We had a number of comments on graphic design issues that might affect navigation. For example, the visibility of the items in the left column is very poor. Already, the column melts into the background and so seems like secondary or unimportant information. On top of that, the blue text disappears into the blue background.

APPENDIX

COMPLETE LIST OF EVALUATIONS

Evaluator			Evaluation
	K		1. [H1 Simple and Natural Dialogue] (Severity 4) How does the log-in process work? Not sure what the distinctions are between log-in features and non-log-in features.
	K		8. [H1 Simple and Natural Dialogue] (Severity 4) It’s unclear that one needs to log in before posting to the discussion board. Also, how do people view other postings?
		P	3. Simple and natural dialog (4) Login and Logout are both shown on the page.
		P	8. Speak the user’s language (4) Much of the vocabulary seems vague and confusing. Think about your target population: communities are often made up of old people, immigrants (may not speak English very well), and many others who would not be familiar with some of your terms. For example: - “indicators”: Your testing already pointed out that “indicators” was not the best terminology, but you kept it. What does it mean? Indicators of what in the community? - “promising practices” what does this mean? It’s completely cryptic to me, even after clicking on the link. - “Master list” (e.g. resources master list, promise practice master list) is also cryptic. - “Administration” has multiple meanings, especially when you are talking about organizations. I think, based on your specs, that you mean the system or Website admin, but it also means the office staff and policy makers. - your search delimiters are also kind of techy and confusing (_ALLWORDS , _ANYWORDS; Search all words [AND] and Search Any Words [OR] ) - “Community Indicator Dashboard” as mentioned in your report is also not the greatest choice of terminology. First, it seems very wordy. Second, “dashboard” in part indicates that it’s the interactive control center for these issues/”indicators”, but in fact, it’s really a diagnostics readout.
		P	11. Minimize user memory load (3.5) The labels on the graphs and scales are insufficient. What do the numbers mean, what are they being compared against, what is the unit? (E.g. GDP “average value is 256” is pretty meaningless). Why are the dials scaled with little notches per Red-Yellow-Green section rather than just Red, Yellow or Green? Those notches imply that those measures are being compared against something. What are they being compared against?
H			12. [H1 Simple and Natural Dialogue] (Severity 3) There are two places for log in. The top login link does not really work. Then the logout link is found in the left menu. User has to look in different places to find login and logout.
	K		3. [H1 Simple and Natural Dialogue] (Severity 3) On the Compare Across Communities page, it seems out-of-place to show the Marin air quality indicator. I just want to see the comparison bar graphs. Having the indicator there made me have to look at it to see if it showed something different than the previous page. Made me do extra work.
	K		4. [H1 Simple and Natural Dialogue] (Severity 3) The “Review Past History” link on the right sidebar is confusing. Does this refer to past comparisons of all counties? I think this is because the right sidebar isn’t separated from the page contents, so the tendency is to think the two are related. Perhaps by separating the sidebar more forcefully from the page, this link would make more sense and make the user see that the sidebar elements relate to the Air Pollution category as a whole, rather than the county comparison page.
	K		5. [H1 Simple and Natural Dialogue] (Severity 3) For the History View page, again, shouldn’t have the current air pollution indicator. The page is for historical data, not current data. Having the current Marin indicator confuses this issue.
	K		6. [H1 Simple and Natural Dialogue] (Severity 3) “Indicators” shouldn’t be on the Other Resources page – this page needs to have only non-site sources. The sidebars should be for within-site navigation.
	K		10. [H1 Simple and Natural Dialogue] (Severity 3) On the View All Indicators list, the expanded list on the main content part of the page doesn’t follow the same order as the category heading list directly above. The order should be consistent for both.
	K		11. [H1 Simple and Natural Dialogue] (Severity 3) Listing an icon for each item on the expanded View All Indicators list seems overly information-heavy. I would eliminate these. The icons on the home page do a great job of orienting the user as to what to expect. This goal has been accomplished by the time the user Views All Indicators. That way, also, all Indicators could be listed; right now, there isn’t enough room because of the icons.
	K		12. [H1 Simple and Natural Dialogue] (Severity 3) I think the search function should be only the simple box on the home page. The search results page should list hits in each category and let the user select from there. It’s too much work to enter a term in the home page search box, then get another search page where the user has to refine the search.
		P	1. Simple and natural dialog (3) Can’t determine task flow order, especially when Login to site takes place. Can you still surf information before logging in, or only after logging in? I tended to forget about the login box while I was skimming the information, focusing mostly on the central white space. I then could not figure out what the relationship was of the blue column to the left. It seemed very secondary as the text and column almost disappear right into the background design. I became more and more confused by the site because the initial view listed the contentious, negative side of the community (the community indicators are mostly negative things people don’t typically like to discuss, like pollution, teen pregnancy, obesity, etc.) but then Resource Centers is everything about the community, including arts, libraries, festivals and other positive topics, as well as the negative ones. The Resource Centers page ended up feeling more like the home page, as it showed different macro facets of community life. Whereas the actual home page seemed like the headline news, and a deeper granularity of information.
		P	7. Simple and natural dialog (3) I couldn’t figure out how to email information on pages that didn‘t have that link at the bottom of the page. This is one of your major scenario items, so it was surprising not to have this addressed (e.g. Compare status across communities. Incidentally, it would be nice to be able to choose which communities to compare here.)
		P	10. Minimize user memory load (3) The visibility of the items in the left column is very poor. Already, the column melts into the background, so seems like secondary or unimportant information. On top of that, the blue text disappears into the blue background. In fact, those are major navigation terms.
H			2. [H2 Speak the Users' Language] (Severity 2) In the dashboard indicator, red means danger and green means good. But some users may not associate the color with the rating quality. Maybe add a text label or key.
H			6. [H3 Minimize User Memory Load] (Severity 2) After I register or login, I should go back to the page I was working on. Right now, the registration process goes to a dead end.
H			8. [H5 Feedback] (Severity 2) After I emailed the page to a friend, there was no confirmation page. I am not sure if the my email was sent out.
H			9. [H4 Consistency] (Severity 2) Links that leave the site are not consistently labeled. First, the data source: scorecard.org link tells me I'll be going to a third party site. But the view list links do not tell me that I will leave the Healthy Communities site.
	K		2. [H1 Simple and Natural Dialogue] (Severity 2) What is the main point of entry for Community Indicators? Should only be one – either on the main page (recommended, since the dashboard looks so good) or the left sidebar.
	K		7. [H1 Simple and Natural Dialogue] (Severity 2) Not sure how effective the “Community Indicators” category is. The icons are very vivid and immediately convey the relevant information. One notices them first, not the category text above. If conveying this category information is important, for example, to link the function of the icons to the “Community Indicators” left sidebar text, you might consider making the category heading more visible. Otherwise, I would eliminate the “Community Indicators” link from the left sidebar and make the icons the only point of entry.
	K		9. [H1 Simple and Natural Dialogue] (Severity 2) The Indicator icons are, again, so vivid that I didn’t realize that a whole other menu of indicators was available for research. I would change the “View All Indicators” text to “View More Indicators” to let the user know that more research options are available to them and that the chosen indicators on the home page are perhaps the most popular ones.
		P	2. Simple and natural dialog (2) Why is it that when I click on “Registration” (from the left hand column) that I’m taken to a page that makes me choose “Register” again from a list whose options don’t make sense (Login, Register, Retrieve lost password). Why am I looking at logging out or finding a password here when I presumably have not even signed up or logged in yet?
		P	4. Simple and natural dialog (2) what is the difference between clicking the Registration link in the blue left hand column, and clicking “member” just below? (none!) I’d rather find out in the latter link what the benefits are of registering.
		P	5. Simple and natural dialog (2) last 10 registered users table has many sections, but I’m not sure what each of the sections show; there are no labels. There are also some random documents attached. What are they?
		P	6. Simple and natural dialog (2) It was not clear at first (home page, other levels) that users can post . How do you do this (and why not blog? )
		P	9. Speak the user’s language (2) I wasn’t sure what the Contact List was at first. Was it the members of site, Contact information to get to relevant organizations and special interest groups, Contact information to communicate with government officials and other policy makers, or the way to communicate with the Web site owners and Webmaster?
		P	12. Consistency (2) I think that the measure of the “indicators” are actually portrayed too consistently. Why are these very different issues treated/measured in the same way with those speed dials? Are they percentages? What does it mean when it’s all blue rather than red, yellow and green?
		P	13. Consistency (2) The template page is a little too similar to the Marin County specific page. I didn’t realize at what point I had moved off of the Marin site and onto the template. Isn’t the template supposed to be used for creating similar sites for other communities? Where does that template use/site creation happen?
		P	14. Prevent errors (2) many errors might be made by the misinterpretations of the terminology, as stated above
		P	15. documentation (2) The site isn’t straightforward regarding its purpose. You gave the scenario of people finding the site through Google. They might find the site through other search terms, and not realize what the site is truly about.
H			1. [H1 Simple and Natural Dialogue] (Severity 1) The color of the text in the left menu makes it hard to read. Maybe make the text brighter so it stand out in a dark background.
H			3. [H1 Simple and Natural Dialogue] (Severity 1) The color of the text in the left menu makes it hard to read. Maybe make the text brighter so it stand out in a dark background.
H			4. [H7 Shortcuts] (Severity 1) From the homepage, the user has to click on the text below the indicator icon. Some users may prefer to click on the icon itself, which is a bigger target.
H			5. [H1 Simple and Natural Dialogue] (Severity 1) The color of the text in the left menu makes it hard to read. Maybe make the text brighter so it stand out in a dark background.
H			7. [H3 Minimize User Memory Load] (Severity 1) The home button is the first item on the left menu. I expected it to be in the top left corner.
H			10. [H1 Simple and Natural Dialogue] (Severity 1) The other resources box on the bottom right is vague. Maybe list out a few related indicator or article titles.
H			11. [H1 Simple and Natural Dialogue] (Severity 1) The search box does not give enough instructions. Are users searching for indicators, articles, locations?