IS213 *** Content Management System *** Second Interactive

*** Content Management System *** IS213 USER INTERFACE DESIGN ***simS

2nd interactive prtype

Pilot Usability Study

Final Presentation

Final Prototype

project vocabulary

System schematic

work distribution

2ND interactive prototype

Responses to Heuristic Evaluation
Responses to Summary Findings & Recommendations
Summary of Changes Made and To Do List
Demo

Responses to Heuristic Evaluation

Compiled Heuristic Evaluation Worksheet
#	Severity	Violation	Problem/Answers	# evals
1	2	[H 1 ] Visibility of System status	Login- No description of system	1
			We plan to put a short description of the system on the first page, but remember this system is designed for perpetual intermediates. The company webmaster has set up each users account and rights. Thus users will have an understanding of what they are suppose to do with system in terms of their work responsibilities.
2	3	[H 10] Documentation and Help	Login-No help with login password.	1
			We plan to put instructions in on the log in page. It is also important to note that a company using this system would have a webmaster that would set up logins for other employees and rights to different parts of the system. Thus, a human would be involved in setting up the initial log in for each person.
3	2	[H 2] Match system to real world	"Editor's Console" confusing vocabulary. All users are editing content. One of the choices on the persistent menu bar is "editors consol". This option should be relabeled to reflect the tasks it supports, rather than the "role" of editor. Users with editor privileges, such as the HR administrator in scenario 4, may not consider themselves to be "editors" and therefore may not think to select that menu option. Suggested labels: "authorization console" or "make public console".	1
			We mentioned in our presentation that we planned to change the name of the editors console because of this reason. We are planning to call the editors console the verification console.
4	2	[H 2]Match system to real world	Why have to go through category before selecting file. Seems like an extra step. Files could be in directory.	1
			We are not planning to change this. The categories reflect the names of the web pages on the company site. In addition, people see different categories based on their rights within the system
5	1	[H 4]Consistency and Standards	Documents--Inconsistent action link for 3. Delete document. 1 & 2 have file selection, then button. The links on the main page clearly marked as hyperlinks. Users shouldn't have to guess where links are on a web page. The design should follow convention by underlining links.	1
			*We will make this more consistent. We made delete a button.*
6	2	[H 4] Consistency and Standards	Documents--Inconsistent operation. Verb-noun, instead of noun-verb in 1 & 2.	1
			*We e-mailed SF MOMA because we couldn't find this error on our UI*
7	2	[H 2] Match system to real world	1, 2 & 3 Seem like steps, not separate operations.	1
			We plan to take out the step 1, 2, and 3. We plan to provide instructions on each page to make the system more like a dialog. We aren't going to put dialog boxes because the system is designed for intermediate users who want to quickly do their task without annoying boxes they have to click through.
8	3	H2 Match system to real world [H 5] Error Prevention	Danger of overwrite. What are the standards for creating new documents rather than modifying existing ones? Possibility for error in deleting relevant document information by modifying document content.	2
			There are two things in place to prevent overwrite: documents have to be verified before changes are accepted and users only see the specific documents, etc. that they have rights to. Someone like Mekka will see everything, but Heather will only see the category and documents dealing with press releases and employee information. Heather will not see the style sheet section at all for example. For your work environment to be efficient you don't want everyone working on the same part of the web site. Different people will be working on different sections and will have different rights. In addition, each person’s work will be checked in the verification console.
9	2	[H 5] Error Prevention	Text entry editing screen--slow processing. Text does not appear as user types. Creates errors. Doesn't meet user expectation.	3
			*This is because the system was running off of Chris's home computer. Once we have the system running off of Sandbox it won't be slow. Sorry about that.*
10	3	[H 9] Error recovery	Icons on editing page return errors-"Web page not found" --no instruction.	2
			The icons in the DEM came with Ezedit. We did not have time to make changes to them, as this was our first interactive prototype. We realize that these are not ideal. We would like to take many of the options out and simplify the icons. We are looking into the technical feasibility of removal. There are several other text editors out on the market that don't have this problem but they cost a lot of money. Ezedit is free. If we can't change Ezedit, this will be one section of our interface that we will not be able to implement and we will just have to describe how we would like to design it.
11	3	[H 4] Consistency and Standards	Ezedit introduces interaction design problems that the content management team may not have any control over. For example, after "rolling over" the unconventional icon on the lower left of the edit page that looks like a magnifying class on a piece of paper, a pop-up note revealed that it was a "Edit Source" icon. After selecting it I realized that it is toggle that hides/shows HTML tags. At the very least, the CMS system should include a guide to using the Ezedit system so that all unconventional, nonstandard features are accessible to users who are unfamiliar with them.	1
			Once, again as mentioned (#10) above the icons in the DEM came with Ezedit. We did not have time to make changes to them, as this was our first interactive prototype. We realize that these are not ideal. We would like to take many of the options out and simplify the icons. We are looking into the technical feasibility of removal. There are several other text editors out on the market that don't have this problem but they cost a lot of money. Ezedit is free. If we can't change Ezedit, this will be one section of our interface that we will not be able to implement and we will just have to describe how we would like to design it.
12	2	[H 4] Consistency and Standards	Inconsistent rollover titles on menu icons	1
			The only place we found this is in the Ezedit section. We will try to make these consistent if it is possible. As mentioned in (#10) Ezedit has some problems we may not be able to fix.
13	2	[H 2] Match system to real world	Confusion over Make it Live and Searchable-terminology "Searchable" not clear.	1
			*We planned to take these out before you received the prototype. Sorry, we will take them out before the second prototype.*
14	4	[H 1]Visibility of System status	The "make it live" and "make it searchable" check boxes exist both in the Ezedit window, and at the foot of the "editor" page (and the modify document) page - see below). It isn't clear if the check boxes control the same function. Confusing which to use.	2
			We planned to take these out before you received the prototype. Sorry, we will take them out before the second prototype. There will be information after users make changes/additions in the system telling the user that they will be notified when the change/addition goes live or doesn’t go live. Ideally we would also like the verifying party to communicate why changes didn't go live (if they didn’t go live), but this may be something we don't have time to implement and will just have to discuss.
15	4	[H 1] Visibility of system status	After saving document user receives "Congratulations you've added a document" AND graphic/stylesheet selections. Does user have to select graphic or stylesheet?	2
			We plan to put instructions in to help users walk through this. The instructions will explain that a text chuck has been added to the system and that they can now choose a style sheet and graphic to go with the text or if they don’t choose then the defaults will apply. In addition, the instructions will tell them about the verification process as mentioned above.
16	2	[H 2] Match system to real world	No apparent default stylesheet	2
			*As mentioned above the instructions will help clarify this.*
17	4	[H 1] Visibility of system status	Unclear feedback. After adding document, user receives "Congratulations you've added a document" when is it really added? When I save it initially, or after I select style sheets and graphics?	2
			As mentioned above the instructions will help clarify this. Users will be informed that they have added the text chunk to the system. They can choose style sheet and graphic or defaults apply. Also it will discuss the verification process. These instructions will help novice users and at the same time intermediate users can ignore them and complete their task.
18	3	[H 6] Recognition rather than recall	pageID-do I have to re-type the name of the file I just created?	1
			*The system now adds this automatically so users don’t have to remember the title..*
19	4	[H 2] Match system to real world	Style sheets -don't know how to use style sheets in order to complete scenario 3.	2
20			Part of this was our fault. We didn't have a dummy style sheet in the system for you. We apologize. We plan to put instructions in for each step of working with a style sheet, which will provide some assistance.
21	2	[H 4] Consistency and Standards	Editor's console: does not go select category -then document like in scenario 1.	1
			*This is one way to help distinguish between the modes.*
22	3	[H 2] Match system to real world	System driven language-WAP, FLASH-Don't know what exactly to select	2
			We plan to provide instructions to clarify this part of the system. It should be noted that if the system were implemented in a company with a HTML site, WAP site, and Flash site for example, people working on the sites would know that the company has different types of sites. Also, suppose the company called these sites by different names (i.e. the wireless site) The instructions could be changed by the webmaster to reflect what people within the company actually call the site. However, for our user group we are using the generic terms: WAP, Flash, and HTML. We will put definitions of these terms in our vocabulary list, which we plan to make part of our help system.
23	4	H6 Recognition rather than recall	After selecting any of the primary options (Documents / Style sheets / Graphics) from "Main," it is unclear what you are looking at. Take "Style Sheets," for instance. It would be very helpful to know what pages would be affected if a style sheet is modified. This same violation is present in "Documents," and "Graphics." At the very least the CMS should report whether an object is in use on an active site. Without knowing what pages are using a particular style sheet it's possible that undesirable and unintended changes could be made to the site.	1
			This is a very good idea. We plan to put in a section that tells the users what documents, etc. are attached to the style sheet/graphic they are about to change.
24	4	H2 Match between system and real world	Not enough information is given about any of the objects accessible through the CMS. Who created the objects, when? Who's edited them, when? Convention generally dictates that a standard set of information about an object should be visible, such as indicated in the description of this violation. Without such information, users can have difficulty locating desired objects and may unwittingly modify or delete objects other people are using.	2
			This is a good point too. We plan to put in information about when a document, etc. was last modified and who modified them. In addition, the restricted access rights will help control who can make changes to different objects. Finally, the verification console checks changes/additions to documents (and the instructions will tell users about the process and the e-mail they can expect)
25	4	H1 Visibility of system status	It's unclear how much of what a user sees is based on his / her login. Nor is it clear who can edit what and what the approval process is for style sheets and graphics. Some kind of "Content Management Flow Chart" on "Main" would help. A user should be able to indicate where a document, graphic and style sheet should be used. For instance, a user should be able to say "take this text, format it with this style sheet, and use it on this web page."	1
			The webmaster installs the system and decides to assign rights to people based on their job responsibilities. Indicating where a style sheet, document and graphic should be used in the system is controlled by the category selection made by the user. We might consider having a flow chart in the help section. However, we don't think this would be useful on the main page because it would confuse the user. The system is designed to abstract away components from users based on rights.
26	4	H1 Visibility of system status	No preview options to show the user how the text, graphic, style sheet will look. Without a preview function users are unlikely to have confidence in the system. The interface doesn't explicitly instruct users on how to "view" content. Some users might find the listed options (add, modify, delete), intimidating. It appears that documents must be opened in order to authorize them for publication. This is cumbersome and would be tedious for an editor in cases where dozens of documents need authorization. At minimum, there should be an auto-preview feature (similar to e-mail) so that editors can see the content without "opening" the document.	3
			Good point. We plan to put in a preview section. We will make the section available to the people making changes as well as the person using the verification console.
27	3	H10 Help and documentation	It's clear that the CMS group intends to add more help and documentation features. A few specific suggestions: indicate how many characters of text can be used in specific places on specific pages and indicate the acceptable size range for graphics. Some explanation of style sheets (see below) would also be critical. It would be very frustrating for users (and damaging to Lobotech) to have documents truncated because they were too long.	3
			Currently there are no limits and the documents won't be truncated. Johnson in the Bloopers book discusses how users should not be constrained by artificial lengths (i.e. your name can only be 8 characters long). We feel it is better to not have any forced constraints.
28	3	H4 Consistency and standards	[This problem is referring to standards in the sense of adhering to external standards.] How does a user know if his or her style sheet conforms to Lobotech standards? Is there any validation of style sheets? Most users are likely to be intimidated by Style Sheets. Other users may try to use the 'add new style sheet' option to add text. Once you click delete (document, graphic, style sheet) there's no way to back out, no "Are You Sure?" message. Might result in accidental deletions.	1
			Only people who know about the company standards will have rights to the style sheet section. In our project, only Mekka has rights to style sheets he can make changes. We currently don’t have style sheet changes going to the verification console because in our user group only one person is making changes to the style sheets. Mekka would be annoyed with going to the verification console to accept the changes. If you click delete for document, it has to be verified before the file is actually deleted from the system. This does not apply to style sheets or graphics. . We have added instructions telling the user that the style sheet or graphic will be permanently deleted. We plan to add a section telling the user that before they delete a style sheet or graphic what documents will be affected.
29	2	H2 Match between system and real world	It's clear that Editor's Console is still in development. Here a few suggestions on making it conform to real world conventions: Editors should be able to make suggestions for changes and they should be able to reject documents. Both actions should result in a message being sent to the author. They should also be able to assign a text to a specific location on the site.	1
			*Good point. Ideally as mentioned above we would like to allow users to get this kind of feedback (#14)*
30	4	H1 Visibility of system status	A user should be able to indicate where a document, graphic and style sheet should be used. For instance, a user should be able to say "take this text, format it with this style sheet, and use it on this web page." The user isn't given enough information to know how the work that they are doing in the CMS will ultimately be used.	1
			*Same heuristic as #25. See response there.*
31	3	H3 User control and freedom	No undo; back; or edit. For instance, once you click delete (document, graphic, style sheet), no way to back out. There should be a means for users to "undo" or "cancel" "publish to the web" commands.	3
			We plan to add navigation within each page. A cancel button will take users back to the main page. Also a back button will return users to the previous page so they can make changes if they want to.
32	4	H10 Help & documentation	There is no help or documentation at this time.	1
			Yes, the instructions for the assignment specifically say not to point out things that obviously weren't implemented. Our help section was not implemented, as this was our first interactive prototype. We are not sure whether we will have time to put in a help section for the second prototype. We may wait until the final prototype to put it in.
33	3	H1 visibility of system status	It isn't clear that users are creating text marked up in HTML when they use Ezedit. This essential characteristic is completely obscured until the "edit source" icon is clicked. Users may not need to see the HTML that is embedded in their documents, but they should be informed that HTML is being generated.	1
			*As mentioned above, the icons from Ezedit are not ideal. See #10.*
34	2	H1 visibility of system status	The commands (submit, make it live) on the edit page aren't visible until I have scrolled down to the bottom.	1
			*We plan to take these out as mentioned above (#14)*
35	3	H1 visibility of system status	After selecting "add, modify, delete document" only two of the three options presented on the subsequent page were visible on my notebook computer's screen. The third option was "below the fold". Although the heading said "you have three options", I still missed it.	1
			According to Jared spool users don't mind scrolling if they feel they are on the right track. Having these options on different pages would make the system disjointed.
36	2	H1 visibility of system status	After selecting the "editors console" link, there is no confirmation that I am in the right place. The page doesn't have any sort of title or heading confirming that I successfully reached my destination.	1
			*We plan to make the color different and put a title at the top*
37	4	H3 User control & freedom	There is no search function in this interface. This is a standard information access tool that users expect, particularly in a database application. This violation could also be classified under "recognition not recall""	1
			*Search is something we have thought about and may want to implement at some later time. However, we think it is too much to deal with at this time.*
38	4	H5 Error prevention	On the "modify document" page where documented are created or edited, users are given the option of "make it live", which I thought was only accessible to editors. This may or may not be an error, but if it is I assume that selecting the checkbox would lead to an avoidable error message along the lines of "you don't have the authority to publish to the web".	1
			*We plan to take this out. See #14*
39	3	H6 Recognition over recall	During the modify/edit document process there is no indication of the "category" of my document (About us) in the case of scenario 1.	1
			*Good point. We plan to carry this information along as the user progresses through working with a document.*
40	3	H7 Flexibility & efficiency of use	The interface does not provide accelerators or short cuts for experienced users.	1
			We feel that we have designed it for perpetual intermediates. One accelerator we are adding is not forcing Mekka to go to the verification console to accept a change to the style sheet.

Responses to Summary Findings & Recommendations

·

Conceptually, this prototype might work in the context of a very small web site with just one or two contributors. However, a large organization with complex information publishing requirements would find it inadequate due to following key interaction problems.

The system is not designed for a company with a huge website. For example if you are creating a new website for Hewlett Packard or IBM this would not be the system to use. The system is designed for smaller websites. For example, the SIMS website or a website for a startup company.

1. Document management convention dictates that a standard set of metadata about an object should be visible. For example, who created the objects, when? Who edited them, when? Add histories and details about how items are used. Without such information, users can experience a number of difficulties such as not being able to locate desired items or unwittingly modifying or deleting objects other people are using because they share the same name.

We plan to put in information about who last modified a document and the date it was last modified. In addition, different people have different access rights which will address this. (See # 24)

2. A versioning or archiving feature is critical for a complex web site, especially one with many contributors. What if I accidentally write over the home page and it accidentally gets authorized by the editor? All versions of published content should be archived and accessible via the CMS system.

The webmaster would routinely take a snap shot of the system.

3. Another mandatory feature that is not present in this prototype is the ability to search for documents across a variety of attributes such as text, title, author, date, location, etc.

Search is something we have though about, but choose not to incorporate in the system at this time.

4. There is a significant Gulf of Evaluation problem with this design. There is no feedback regarding the impact of modifications made through the system. Users, whose actions could affect thousands of published web pages, need to know how their changes will affect the site. Will their new/modified content fit on a particular device's display? How many web pages will be effected? Users should be able to answer the following question: "were my intentions realized?" A preview feature would be ideal, but at minimum upon "save" users should be told which web pages would be affected upon authorization by the editor.

We plan to put in a preview page. (See #26)

5. There is no visible way to cancel or gracefully exit from the editing process. Users can log out, but not cancel, or clear modifications. Add functionality that allows users to undo previous actions.

Plan to add a cancel button and back button for additional navigation. (See #31)

6. The error messages (both javascript and cold fusion) are incomprehensible. Errors encountered on nearly every interactive screen. If possible, hide the details and display custom messages that offer recovery instructions.

Plan to put in nicer error messages. This was our first interactive prototype. Obviously this was something we choose not to implement. The instructions for the assignment clearly state not to comment on things that obviously weren't implemented.

Summary of Changes Made and To Do List

Changes we have made based on Heuristic

1. Put a short description of the system on the login page (see#1)

2. Put instructions on every page (see #1, #2, #7, #14, #15, #16, #17, #19, #22, #23, #24, #28) 3.Changed the name of the editors console to verification console (see #3)

4. Fixed the inconsistency complained about in #5 with delete document versus other options by making the delete option a button instead of a link.

5. Removed the 1,2,3, on the pages (see #7)

6. Removed some utilities we don't want users to use, fixed glitches, and allow addition of hyperlinks with the ezedit portion. (#10 and #11)

7. Removed the make it live/searchable check boxes (#13 and #14)

8. PageID field automatically filled with correct title (#18)

9. Provided a section telling user which documents will be effected by changing style sheet and which documents are linked to graphics (see #23)

10. Put information about who last modified a document and when it was last modified (see #24)

11. Put in a preview section so users can see how changes will look

12. Added a cancel button to take user back to main and a back button to go back to the previous page in each section (see #31)

13. Put in section to carry category information along while working with a document (see #39)

Additional Changes we made to the system (not from Heuristic evaluation)

1. Completed the visibility of options based on user rights on the main menu and the main page. You can see the difference if you want by logging in under cmarin and then bob (Password on both is test44).

2. You can now add hyperlinks through Ezedit.

3. The database schema was updated. New users rights like Delete were added, as well as an email field for user notification.

4. We got the system up and running on sandbox so it is more stable.

For the future (suggestions from Heuristic we don't have time to implement)

1. Add help and documentation (See#32)

2. Discuss adding search/browse features (see#37)

3. Further discuss how editors deal with having to go to the verification console to make changes live. For example, if Dave changes a document does he then have to go to the verification console to make it live. According to the heuristic evaluation the make it live button was confusing so we removed the button. Perhaps the usability test will provide further information.

4. We received e-mail from SF MOMA about #6 but didn’t have time to change it. #6 refers to the noun verb problem. We plan to change this by having the selection of a document/style sheet for deletion on the same page as adding and modifying a document/style sheet. We will have to rework the order of the buttons to do this.

5. We plan to add the list of documents that will be affected by a style sheet or graphic deletion before the user presses the delete button. (we provide this list in other areas, but we also want to include it right before deletion)

Demo

Instructions

In order to run the content management system, you need to use Internet Explorer.

Type in the following URL: http://www.rebelunit.com

You can see different access rights by logging in as different users. We have set up two separate log in names which provide you with access to different features in the system based on rights. One login name is cmarin and the other log in name is bob. Both logins use the following password: test44. After logging in you will see the main page which provides navigation to complete your tasks.

***