PROTOTYPE Experiment KIT
Sfnight project

 

 

Procedures:

  1. “Facilitator”
    1. Ask user to Fill up User Questionnaire and the consent agreement    : The subject filled up a questionnaire about their user, Internet and web site experiences. 

2. Will remind the user that we are not testing the user ability, but the system it self.

2. Also will explain the tester about the technique “thinking loud”

3. Read the Task. The facilitator would ask the subject to read each Task before start each site evaluation. The time started to be counted only after reading the task 2. The task description sheet should be available for subject’s consultation anytime.

 

  1. “Computer”. Has to be aware of each movement of the tester, and be able to change the interface. Do not do any comments, observation etc, to be very concentrated.

 

  1. “Observer”. Will do the following

·                    Follow each Task Observation Requirement. 

·                    Take notes of the user’s expressions and words, when the user is “thinking out loud”.

·                    Make note when the user clicks in the wrong place. This requires high concentration; otherwise we loose data.

·                    Be responsible for counting the time.

 

  1. Start the Task #: The tester begins to perform the Task 1 in the prototype.

 

  1. Finish the Task #: Once the subject has specified that the task is done, give him/her a small evaluation form, see item 6.

 

  1. Task Questionnaire # of Site: After each task, ask the tester to complete a questionnaire about the task related.

 

  1. Site Questionnaire: After all tasks are finished, ask the test to complete an evaluation questionnaire for the site. If the subject is tired, skip this part.

 

 

 

1 About the Tester

 

 

1.      How old are you?

 Over 45 years old

 31-45 years old

 21-30 years old

 18-20 years old

 

 

 

2.      How long have you been using the Internet

 

 More than 8 years

 5-8 years

 2-5 years

 1-2 years

 Less than 1 year

 

 

3.      Have you used one of the following web sites about entertainment?

 

 Sfgate.com

 Citysearch

 Other, please specify     

 

 

 

4.      Have you created any personal page [such as MyYahoo, etc] in a web site?

 

 Yes, Please specify: …………………………..

 No

 

 

 

 

 

 

 

Task 1 - PLANNING AN EVENT

 

Background:  It is Wednesday evening. A friend is visiting from out of town and wants you to help organize an outing with some local friends Saturday night.  At SFnight, you find some events and venues where you think you’ll all have fun. 

 

You email all your friends some options.


 

 

Task 1 Task Questionnaire

 

 

SFnight Questions

Please indicate the degree of which you  DISAGREE (1) and AGREE (7) with the following statements about this SFnight when you performed this task. 

                                                                                             

 

Disagree <

1

2

3

4

5

> Agree

1

Selection process is easy and straightforward. 

 

 

 

 

 

 

2

You are sure that you send all information you need from MySFnight to your friends

 

 

 

 

 

 

                                             

 

General Questions

1.      Was any part of the steps confusing, hard to understand?

 

 

 

2.      Did the site provide adequate information to help you complete the task? Or too much information?

 

 

 

3.      Was it easy to complete the steps?

 

 

 

 

 

4.      Would you use this feature to email selections to your friends again?

 

 

 

 

 

5.      Any other thoughts and comments?  (use Back)

 

 

Task 2 – SIGN UP FOR A NEWSLETTER

 

Background:You just heard from a friend that you could receive information about free nightlife events and promotions by e-mail. All you would need to do is to sign up for a newsletter at the SFnight website.

 

 

Task 2 Task Questionnaire

 

 

 

1.      Was any part of the steps confusing, hard to understand?

 

 

 

 

 

2.      Did the site provide adequate information to help you complete the task? Or too much information?

 

 

 

 

3.      Was it easy to complete the steps?

 

 

 

 

 

4.      Would you use this feature to in real life? 

 

 

 

 

 

5.      Any other thoughts and comments?

Task 3 – CREATE & CUSTOMIZE MYSFNIGHT

 

 

Task 3 Revised wording:

Background: You have used SFnight a number of times and have not signed up for an account yet. You are aware that you can create a customized MySFnight page that could help you find out more easily what is happening at your favorite venues, when events you like may take place… You decide to sign up for MySFnight and explore how MySFnight can be useful for you.

 

Task 3 Original wording used with participants

Background: You have used SFnight a number of times.  You want to be able to easily find out what is happening at your favorite venues and perhaps track the schedules of some of your favorite  groups when they perform in SF.  You decide to check your MySFnight page.

        

Task 3 Task Questionnaire

 

 

General Questions

1.      Was any part of the steps confusing, hard to understand?

 

 

 

 

 

2.      Did the site provide adequate information to help you complete the task? Or too much information?

 

 

 

 

3.      Was it easy to complete the steps?

 

 

 

 

 

4.      Would you use this feature in real life?

 

 

 

 

5.      Any other thoughts and comments? (use the back if you need room)

 

Instructions for conducting TASK 1

 

      

Procedure: Give only the Task description related with the site, if the tester decide to complete registration, and customize, simply follow the notes for Task 2 and Task 3. 

 

Time & Help:  We will not count time, since the user might want to see information related to venues or events. More important to observe.  If tester cannot figure out how to do after 30 seconds, the facilitator should help, and observer register.

 

Expected Procedure of the User:

 

1.      Find 2-3 venues/events that would recommend to your friends.

2.      Add each event to MyList

3.      View the profile of the venues he/she selected.

4.      Confirm the selection and send it to to friends.

 

 

 

What to observe: If the user would use the personal area, or would tend to go to the other areas; how ease is to user to pick their choices, and send to friends; what other needs would come up when performing the option.

After finishing the task: Ask the user what he would like his friends to receive.

 

1. Did the subject understand the task?

3. Did the subject need help to find information after 2 minutes?

4. Did the subject follow correct procedure? Did he get disoriented?

5. Did the subject come up with other needs during the process?

 

 

 

Measurement: Questionnaire satisfaction.

 

 

Instructions for conducting TASK 2

 

 

 

Procedure: Give only the Task description.  If tester has registered, skip and go to Task 3

 

 

 

                        Expected Procedure of the User:

1.      Find sign up newsletter button, click and follow the instructions

2.      Select favorite venues and music types would like to include in the newsletter.

3.      View welcome and confirmation page and go back to homepage

 

Measurement: time to perform the task; number of clicks in the wrong place.

 

 

What to observe: if the user see the two kinds of newsletter/registration; how confusing this might be; and the process; if the user has doubts, feeling expressions [bored, smiling etc]

 

      1. Did the subject understand the task?

 

 

      2. Did the subject spot MySFnight favorite list in the first 30 seconds?

 

 

      3. Did the subject need help to find information after 2 minutes?

 

 

      4. Did the subject follow correct procedure? Did he get disoriented?

 

 

     5. Did the subject come up with other needs during the process?

Instructions for conducting TASK 3

 

Procedure: Give only the Task description if tester has completed customization in Task 2, then the tester is finished with the entire task.  Give the tester overall site evaluation form.

 

 

Expected Procedure of the User: 

1.      Find and register for MySFnight by following site instructions

2.      Select favorite venues and music types he/she would like to include in the personal area.

3.      View welcome and confirmation page and go back to homepage

 

What to observe: If the user would use the personal area, or would tend to go to the other areas; how ease is to user to pick their choices, and send to friends; what other needs would come up when performing the option.

 

Measurement: How many time they clicked at the wrong place. We will not measure time here, since users might take their time to make decisions of music etc. The most important would be to measure the wrong clicks.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Post-test questionnaire

 

 

1. Which aspects did you find particularly easy in this site?

 

 

 

2. Which aspects did you find particularly difficult in this site?

 

 

 

 

3. Please give additional comments about this site. We would especially appreciate your feedback on the following topics:

 

 

n       Which functions did you like most, which ones you think you would not use

 

 

n       Would you in reality create your personal page to have nightlife information at this site?

 

 

n       Anything that you think that is missing in this site