IS 213 -- User Interface Design
& Development
ReadingTree: Lo-Fi Prototyping/ Usability Testing
|
lo-fi prototype
|
Appendices
Lo-Fi
Prototype Creation
Navigation Direction - first
try
Navigation Direction -final selection
Page Groupings
Determination of which pages
to do
Lo-Fi
Prototype Images
Tiltle
Bar
Title
Bar with Rollovers
Home Page
Whoops - Sign In or Sign Up Dialogue
Box
Sign In Dialogue Box
Sign Up
Find a Book
Searches
Search Results
Book Information
Review a Book
Book Suggestions
Thanks
Poll Dialogue Box
Rate
a Book Dialogue Box
Thanks Rating Dialogue Box
TreeHouse
Message Board Topics
Message Board Threads
What's New
Lo-Fi
Prototype Script
Consent Form
Pre-Test Questionnaire
Raw Test Data
Navigation
Direction - first try
Return
to Prototype page
Navigation
Direction - final selection
Return
to Prototype page
Page
Groupings
Return
to Prototype page
Determination
of what pages to do
Return
to Prototype page
Title
Bar
Return
to Prototype page
Title
Bar with Rollovers
Return
to Prototype page
Home
Page
Return
to Prototype page
Whoops
- Sign In or Sign Up Dialogue Box
Return
to Prototype page
Sign
In Dialogue Box
Return
to Prototype page
Sign
Up
Return
to Prototype page
Find
a Book
Return
to Prototype page
Searches
Return
to Prototype page
Search
Results
Return
to Prototype page
Book
Information Page
Return
to Prototype page
Review
a Book
Return
to Prototype page
Rate
a Book Dialogue Box
Return
to Prototype page
Thanks
Rating Dialogue Box
Return
to Prototype page
Thanks
Poll Dialogue Box
Return
to Prototype page
TreeHouse
Return
to Prototype page
Message
Board Topics
Return
to Prototype page
Message
Board Threads
Return
to Prototype page
What's
New
Return
to Prototype page
Introduction
[Facilitator introduces herself, the notetaker, and the "computer."]
I want to thank you for taking the time to help us with our project. This is the "first draft" of a computer system designed to help kids find and talk about books they like. Since it's for kids, we want to hear from kids about how we can make it better.
Test Description
Here's what will happen:
- We will ask you a few questions about your computer use and reading activities.
- We will ask you to do five small tasks.
- When you're done we're going to ask you what you liked and didn't like about the design While you're working, please try to "think aloud"-tell us what you're doing and why you're doing it. This part is especially important if something is confusing or difficult to do.
Please remember:
- We're testing the system, not you.
- Anything you do is absolutely right and will help us make the system better.
- Also, you can stop at any time-just let us know.
- You're welcome to ask questions--and we'll help you out if it seems like you're stuck.
Consent Form
Before we get started I need to ask you and your mom/dad/guardian to sign a permission slip, which repeats everything that I just told you.
[Give consent form to adult, then child to sign]
Questionnaire
Now I have few questions to find out what you think about books and computers. [Use attached questionnaire.]
Lo-Fi Test
As you can see, our "computer" is made out of paper. Instead of a screen, you'll be looking at sheets of paper. Instead of a mouse, you're going to use this pointer. If you want to click on something just tap it with the pointer. And instead of having the paper change, we actually have separate pieces of paper for each screen, which Amity will bring out at the right times. There are five tasks in all. We'll go through them one at a time. Again, remember that it's OK to ask questions at any time.
Danny Persona Tasks:
1. Find a book for school, subject: civil war
2. Find a book by title: "Harry Potter and the Goblet of Fire." What is its average rating?
3. "Let ReadingTree suggest a book." Get 1 book recommendation.Jenny Persona Tasks:
1. Sign up for Reading Tree
2. Find out what the number 1 and 2 books are on the Reading Tree "What's Hot" list
3. Go to the book club message board and find out if anyone is talking about Harry Potter. Find out what 2 kids think about it.
4. Rate (and review) that book.
Evaluation Questions
- What did you like most about Reading Tree?
- What did you like the least?
- Think about someone you know who is a year or two younger than you. Do you think that person would be able to use this website easily?
- Anything else you would like to tell us?
Consent Form for ReadingTree.org Research Participation
Amity Zeh amityz@sims.berkeley.edu,
510-271-0789
Kirsten Swearingen kirstens@sims.berkeley.edu, 510-658-2920
ReadingTree.org is a new website for kids who love to read. You can meet other kids, chat about books, and even get personalized recommendations for books that ReadingTree thinks you'll enjoy. But since it's for kids, we want to hear from kids about how we can make it even better -- more fun and easier to use.
ReadingTree is being developed by us -- Kirsten Swearingen and Amity Zeh. We're graduate students in the School of Information Management & Systems at the University of California at Berkeley. We'd like kids like you to help us improve Reading Tree. Conducting this research project is necessary for us to earn our Master's Degrees.
If you decide to participate in our study, here's what will happen.
- Sign this permission slip. Your parent or guardian will need to sign it too.
- One of us will ask you some questions about your computer use and your reading activities. Then we'll ask you to do some activities on ReadingTree. The other one of us will be taking notes on what you think about it, and how easy it is for you to use. We hope that this will be fun for you.
- When you've finished the activities, we'll ask a few questions about what you liked and didn't like
- We will redesign ReadingTree based in part on what you tell us.
The only risks of participating in the study is that you might feel embarrassed if we told anyone what you thought about ReadingTree, or if you find ReadingTree hard to use. We will keep anything you say to us private and confidential. We won't identify you by name in any of our reporting. We also won't report anything you say or do during the study to your parents.
You can benefit from participating in our study by helping to improve something that you may use later, and have fun with!
Participation in the research study is completely voluntary. You don't have to take part. If you do decide to take part and you want to stop partway through for any reason, that is OK too.
If you have any questions about ReadingTree, or this research study, you can call us or email us at the numbers above. I have read this consent form and agree to take part in the research.
Student's signature: _______________________________________ Date ________
I have read this consent for and agree to have my child take part in the research.
Parent's signature: ________________________________________ Date _______
(administered verbally)
Demographic Questions
Grade
- 1st
- 2nd
- 3rd
- 4th
- 5th
- 6th
Gender
- Male
- Female
Part 1. Computer Use/Chat
1. Do you use a computer?
- Yes
- No
If the answer to Question 1 is no, please go straight to part 2.
2. Where do you use a computer? (check as many as you like)
- home
- school
- library
- community center
- friend's house
- other place: _________________________________
3. What do you use the computer for?
- games
- chatting with friends or emailing
- research
- homework
- educational programs
- other uses: _________________________________
4. How much time do you spend on the Internet when you use the computer?
- Never more than a few minutes
- It depends-sometimes a little time, sometimes a lot
- I'm always on the Internet!
5. How do you stay in touch with friends online?
- bulletin boards
- chat
- games
- I don't talk to my friends on the computer
- Other ______
Reading Habits
1. Do you like to read books for fun?
- No, they're boring
- Sometimes, but I'd rather do something else
- Sure, I like books.
- I love to read!
2. How do you find books that you want to read? (check as many as you want)?
- bookstore staff
- browsing in a library or bookstore
- friends
- librarians
- parents
- reading groups
- teachers
- displays in stores or libraries
- magazines
- TV shows
- websites
- other ________________
3. Would you like a website that suggested books for you that you might enjoy?
- No, I wouldn't use it.
- Maybe.
- Yes, that sounds great!
4. Would you be interested in reviewing and recommending books for other kids?
- No.
- Maybe
- Definitely
Changes resulting from Pilot Tests
- Added mouse-overs to the navigational icons
- Made the sign-up button more prominent.
- Added instructions about how to make up a nickname
- Changed wording from "confirm" to "retype secret password" Include a drop-down list of the States
- Changed label on book reviews page to "Book Information"
- Added a confirmation dialog for "Thanks for rating a book!" as well as "Thanks for answering a poll!"
- Changed wording on Personalized Recommendations page to "Improve Your Recommendations."
- Changed wording on Improve Your Recs--could not refer to "earlier questions" if user hadn't answered any yet.
- Fixed wording on "Other kids like you who like books like these."
Other changes we considered but postponed:
- Change name of Today's Special icon.
- Changing name of bulletin board to "messages" or "book club."
- Adding a Help page - we decided that the "computer" (Amity) would act as the "magic help." (If the child clicked help, she would answer any questions.)
- From the book info page, user is looking for links to "what kids think"- should we connect book info to bulletin boards?
- Need to add a Bookshelf page.
Subject 1
Subject 1 was distracted during the test - by his brother, by the new environment, by the television in the other room. This lack of concentration was reflected in his asking for reminders about the task several times during the observation and, we believe, in his disinterest in exploring the system beyond asking for help or giving up. In addition, though he played with the pointer throughout the test, he interacted with the paper prototype primarily by pointing with his finger.
Task 1. Sign Up for ReadingTree. He first seemed to read everything on the page, then started to fill in the "sign in" boxes. Once he saw the dialogue box that he had to sign up first, he clicked that option and filled out the sign up form without any problems or questions. He knew to click submit when he finished the form. Task 1 took approximately 2-3 minutes.
Task 2: Find Harry Potter and the Goblet of Fire and rate it. He reread the home page but couldn't figure out how to find a book. We suggested the "Find a Book" link. Once there he performed an author search, using the hyperlink list to type a name in the fill in box. He found the book on the search results, and clicked the book link without prompting. Once one the book page he carefully read the summary, then asked where the rate a book part was. We pointed it out, and he rated the book. It took him a few seconds to know what to do with the "you're welcome" button in the resulting "thanks for submitting your review" dialogue box, but he clicked it. Task 2 took approximately 3-4 minutes.
Task 3: Find out what book is #2 on the "What's Hot" list. He didn't seem to understand the task, and first asked where the numbers were. We reread the task, he selected "Help" and asked where the "What's Hot" list was. We told him, he found the list and completed the task. Task 3 took approximately 2 minutes.
Task 4: Find out what 2 kids think about Hermione from Harry Potter. On the "Today's Special" page (the page where the last task left him) he kept typing on the icons for the poll, then pressed "Help". Help suggested looking in the TreeHouse. On the TreeHouse page he couldn't find the bulletin board, located n the center of the page. Help pointed it out. He first clicked on the first link on the list (not about Hermione), then on the the link about Hermione. Task 4 took approximately 4 minutes.
Task 5: Get Personalized Recommendations. He clicked on the message board messages, then on "Help" which suggested going to "Home" or "Find a Book". Did a mouseover on "Find a Book", said he couldn't get personalized recommendations there, then saw that he could. Didn't really read instructions, but hit the get recommendations button. Task 5 took approximately 5 minutes.Post-test follow-up summary
Said that he liked that Harry Potter was in it, but that he didn't like all the clicking around and having to move from place to place. When asked if he thought someone younger than him would enjoy using Reading Tree he said that he didn't know.
Subject 2 (and mother)
Test was done jointly by Subject 2 and his mother, because she said that she would probably use a site like this with him so she could help him with the reading. Subject 2 was distracted throughout the test and was most interested in participating when we paid attention to what his mom was doing rather than pay attention to him protesting.
Task 1. Sign Up for ReadingTree. Tried to sign in, and received the dialogue box directing them to sign up instead. Clicked on "Sign Up" and went to the Sign Up page. Filled out the sign up form without any problems. Subject 2 didn't know his city and state but his mom did. Task 1 took approximately 4 minutes.
Task 2: Find Harry Potter and the Goblet of Fire and rate it. Went directly to find a book, and tried title search by using the hyperlinks like a typewriter. From the returned search results they found the Harry Potter book, and on its book page they found the "rate this book" button. Subject 2 didn't know what "rate" meant, but his mother explained it to him and they completed the form. The subject's mother was amused by the "you're welcome" button. Task 2 took approximately 4 minutes.
Task 3: Find out what book is #2 on the "What's Hot" list. They used the mouseovers and clicked on Today's Special. From there they easily found the "What's Hot" list. Task 3 took approximately 1 minute.
Task 4: Find out what 2 kids think about Hermione from Harry Potter. They did a mouseover on the TreeHouse icon, and clicked onto that section. They easily found the bulletin board and its section on Hermione. Task 4 took approximately 2 minutes.
Task 5: Get Personalized Recommendations. They did a mouseover on the Find a Book icon and clicked onto that section. Subject's mother read the entire section on "Personal Recommendations", and clicked on the link to answer additional questions. They answered the sports poll, and asked if there were any more questions to answer. They returned to the "Personal Recommendations" section in Find a Book and clicked on the rating other books option. This returned them to the Find a Book page and didn't offer further explanation. At this point the computer recommended that they just get recommendations with the information they have. They got the recommendations, and wanted to see the bookshelves listed on the recommendations page.Post-test follow-up summary
The mother of Subject 2 said that it is hard to work with kids like hers that aren't familiar with web surfing, and made several suggestions and requests. She'd like to have reading level ratings because she feels that grade level is not specific enough. She wants to be able to see a book's pages, to get a feel for how large the words are, how many are on a page, and if it has pictures - this is part of how she and her son pick out books at the library. She suggested developing a introductory lesson on web surfing, as well as a tutorial on ReadingTree - what the icons are, why click on them, what to do, why they should do it. She's afraid that kids might get lost and suggested explaining to them what a home page is and teling them that if they get lost they should click on it.
Subject 3 raw data (p)
Test subject tried to put herself in the role of her 7-year-old brother. This turned out to be not so much a test of the system's usability as a reality check on what 7 year olds would want from a website.
Task 1: Sign up for Reading Tree. Clicked on the "Sign Up" button right away, but commented that her brother would want to see a picture. He would scan the screen and look for a pencil or pen icon. The sign-up form was OK. Brother would have trouble spelling the name of his city but could ask someone. Subject thought "Enter" would be better than "Submit."
Task 2: Find out "What's Hot" (Started from the home page.) Her brother would not read through each thing presented. Would not have enough patience. Maybe if an icon appeared to grab his attention and point him the right way…but if I showed him where to go the first time, he could remember where to click the next time.
Task 3: Find out what other kids think about Harry Potter. Subject stated that 7 year olds aren't really interested in what other kids say. He would want to see information from and about the book-pictures, plot, maybe an animated movie with voices or music. Would want interactivity. He would click on the book image first, rather than the words. Subject commented on the connection between TV and web use. If a kid is motivated, will seek out a site, or a particular feature of a site. Kids also like doing activities. Like at the Sesame Street site where he can paint using the computer.
Task 4: Review Harry Potter. Again, would like to see picture to represent Rate or Review. Brother would write only a sentence or two and skip the rest. Or maybe just go straight to the ratings. Subject suggested having the parts of a book review available so that kids could pick and choose to build the review, or do fill in the blanks.
Task 5: Search for Harry Potter. Found the Find a Book option easily from the Home Page, clicking on the books icon. Once at the search page, commented that it seemed like a standard search interface. Would rather not have to distinguish between a title or author search. Would type in part of the title without a problem. For the search results, showing the cover is important-so kids can recognize it if they've seen it before on TV or at their friends' houses. Would click on the book image to find out more.
Subject 4 raw data (d)
Questionnaire summary:
(4th grade male)
Computers: Uses a computer at home and school for games, homework, going online, computer lab stuff at school ("like programming?"). Doesn't use the Internet for more than a few minutes and doesn't talk to friends online (although his 13-year-old sister does).
Books: Loves to read. Finds books through browsing in a library or bookstore, from his friends, from reading book reviews (at school) or from his sister. Would like a website that suggested books and would be interested in reviewing and recommending books for other kids.Tasks:
Task 1: Sign up for ReadingTree. Subject clicked on 'sign up', filled out sign up information and clicked submit without any difficulty.
Task 2: Find the number 2 book on the 'what's hot." Subject read over the right hand side of the page, and clicked on "what's hot" link. Found second book on list and mentioned that he'd read it already.
Task 3: Find out what two kids think about Harry Potter on the book club message board. Facilitator asked subject to return to the home page before completing this task. Subject clicked on 'home' icon. Read the right hand side of the page and followed link to message board. Subject read each thread title aloud in order to complete task. Without bulletin board experience, subject needed to be prompted to click on title of thread to read links below it.
Task 4: Find Harry Potter and the Goblet of Fire. When informed of next task, subject repliesd, "I'm already there," since he was at the bulletin board topic pertaining to HP&GOF. Facilitator informed subject that there was a general information page about Harry Potter and could he find that. Subejct clicked on Find Book, then "typed" the full title of the book into the search field and clicked Find Title. This returned a sheet of all Harry Potter books (limitation of the prototype) so he then clicked on the title to get to the information page without a problem.
Task 5: Rate and/or review this book. Subject clicked on Rate/Review this book without any problem and completed a review and rating without any problem. Facilitator volunteered that it was not necessary to fill in every field, so he left "what's wrong with this book?" blank; this wasn't clear to him before.Post-test follow-up summary
Liked the find-a-book aspect of it. Didn't find any of it confusing and was reluctant to volunteer any critical information. Thought reviewoing and rating a book might be hard for kids younger than 1st grade. Interested in testing "live" site when available.
Subject 5 raw data (r)
Questionnaire summary:
(4th grade male. Wanted to fill out questionnaire himself rather than replying verbally.)
Computers: Uses a computer at home for games. Sometimes uses the Internet for a little bit and sometimes a lot. In reply to "how do you stay in touch with friends online" subject wrote in, "I usually ask if I can e-mail them but they don't have email addresses yet." Then checked that he doesn't talk to friends online.
Books: Loves to read. Finds books through browsing in a library or bookstore, from his friends, and from displays in stores or libraries. He might like a website that suggested books and would be definitely interested in reviewing and recommending books for other kids.Tasks:
Task 1: Sign up for ReadingTree -- Subject clicked on 'sign in' rather than 'sign up' and got an unsuccessful error message (this was the fault of the prototype). The computer told the subject what the error was and to click Sign Up on the dialog box in order to sign up. Subject then filled in sign up sheet verbally, but said, "I click on Berkeley" and "I click on California" for fields where actually typed input was required. The computer took the input and did not correct the subject.
Task 2: Find the number 2 book on the 'what's hot' -- Subject read over the right hand side of the page, and clicked on "what's hot" link. Had to be re-prompted as to the task at hand. Then found second book on list and mentioned that he'd read it already.
Task 3: Find out what two kids think about Harry Potter on the book club message board. -- Facilitator asked subject to return to the home page before completing this task. Subject clicked on 'home' icon. Read the right hand side of the page and followed link to message board. Read the threads and then clicked on one to get more data. Computer requested that he click on a different thread in order to display conversation topics.
Task 4: Find Harry Potter and the Goblet of Fire. Subject clicked on Find Book, then "typed" the full title of the book into the search field and clicked Find Title. This returned a sheet of all Harry Potter books (limitation of the prototype) so he then clicked on the title to get to the information page without a problem.
Task 5: Rate and/or review this book. Subject clicked on Rate/Review this book without any problem. He skipped over the review process entirely and only clicked on "one of my favorites" as a rating option. Then clicked submit.Post-test follow-up summary
Liked the find-a-book aspect of it and the variety of chat and bulletin boards. Sign in part was confusing to him. When prompted for comments, he thought that it would be cool if there were reviews from other kids and if kids could respond to them. (Note: I wasn't sure whether he had noticed that there were in fact reviews from other kids on the book information page.) He thought kids younger than him might be able to use it without a problem. Interested in testing "live" site when available.