Our particpants:
P1
- Graduate student in law and foreign service from East coast
- Graduating this year and will be working as a prosecutor as a DA
- No experience with blogs (had only heard of them recently before the user study)
- Generally not tech or web savvy
P2
- Senior product marketing manager at major hardware/software firm
- Lives in San Francisco, from the Bay Area
- Familiar with blogs, though hasn't read much
- Both web and tech savvy (not a programmer, but can set up wireless networks, build websites, and so forth, and has a good understanding of many computer technologies)
P3
- Graduate student in Information Management
- Graduating this year and will be working as a developer for a non-profit
- Experienced with blogs both as a reader, has written in his blog but would not describe self as a "blogger"
- Technologically sophisticated
Results
General
- We need to rethink our metaphor. The idea of a "conversation" is confusing to people. What is a response? Is there a real difference between what PX comments on PY's blog versus what PX comments on her own blog while referring to PY's blog?
- All three users were confused about the way the site was structure. Or rather, they did similar things for similar tasks despite range of familiarity with "standard" web interfaces. Separating the three main areas of the site is not going to work. we already have ideas of ways to unify them.
- Perhaps we are thinking about it the wrong way entirely. Rather than show "search results" (like a search engine would), we try to show relevant entry points into a greater context. This would support browsing and wandering through the blogs.
- All there users were confused by the terminology. The terminology used was lifted directly from the blog world. We think we may have to invent or adapt a new set of terms. For example, no understood with confidence what an "inbound link" was. However, P3 did guess that a "blogger" would know (indicating that P3 was not a self-described blogger).
- The users came in with different mental models of what this thing was supposed to do or should do. They compared the interface to Goolge, to an email app, to a threaded discussion list, and to a "tree" (like a discussion thread or mail list archive).
- P1 and P2 were not clear on what should constitute a "relevant" search result (which of the visible data indicates relevance). P3 said that the number of inbound link made sense for this.
Specific:
- None of the usres were quite sure what clicking on "View entire conversation" would do, though P3 was closest.
- When asked to find stats on a particular blog, no one's first instinct was to click on "Find-a-blog" (the correct answer!). All three wanted to just put the URL in the address bar of the browser.
- When asked to search for a topic, all three clicked on the link to the topic that was next to each search result. No one first went to "Browse topics."
- We had some inconsistency between post title's that would open up the actual blog vs. titles that were a link to a Paparazzi profile page.
- They all had to do some work to determine how the posts were initially sorted.
- While not definitive, date maybe more signifcant than we made it out to be.
- Wasn't clear what "Rank" really meant.