FAST Project |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
FAST Project Main Page |
Assignment 3Revised Problem Statement Revised Problem StatementCurrently at UC Berkeley, all ladder-rank faculty are required to submit an annual paper report entitled the Annual Supplement to the Bio-Bibliography (Bio-Bib) which documents a professor’s efforts and achievements across an academic year. The Bio-Bib is an aggregation, distillation and reformatting of a large corpora of collected documents and data sets. Although this data is necessary for review purposes, completion of the report it is often viewed by faculty as tedious and time-consuming. Building an interface which could reduce the effort and cost of producing this report would be a major service to the faculty. Because the Bio-Bib is a paper report, its data is not reusable; data from a previous Bio-Bib can not be inherited to the present Bio-Bib and must be reentered. Consequently, input effort by a professor could be significantly reduced by simply making the form electronic and recalling data from a previous year. There is also a design challenge to build a Bio-Bib interface that is clearly understood and easy-to-use for any professor in any discipline. Although there is a standard form for all professors, many of Bio-Bib’s input fields use categories and terms relevant to traditional academic research; these categories request data for classic academic accomplishments such as journal articles and invited lectures. However, these same categories may appear much more ambiguous to professors who choreograph a ballet, direct a play, or write a musical composition. Finally, the Bio-Bib currently provides no means to clarify any ambiguity which may arise from a given question. Providing a professor with documentation about a question’s purpose or displaying a concrete example of how to input specific data could also increase faculty satisfaction and reduce their effort. Revised Project GoalThe FAST team will design a prototype interface for the Research and Professional Services subsections of the Bio-Bib. We will develop a user-tested conceptual model which removes ambiguity for professors who develop creative works without sacrificing clarity for professors who are engaged in more traditional research. Revised Personae and TasksAfter revisiting our personas this week, our team decided to make some important changes to our personae descriptions, user tasks, and project scope. We began by discussing the goals of each of our personas, and how these goals impacted the way that they spent their time. We felt that it was critical to distinguish Allison from Jennifer in terms of the breadth of her academic commitments and her status on the tenure track. Jennifer is energetic and ambitious, and willing to invest significant time into collaborating with her graduate students on publications, setting up her lab, and serving the educational community. Her goal is to build public credibility and respect from her colleagues. Allison is more focused with her time; she splits it between finishing the last chapters of her book and directing a large play. Her immediate goal is to attain tenure so that she can start a family.. As we were teasing apart the unique qualities of each personae, we also felt that it was important to further differentiate Mark from Jennifer, particularly along creative and academic lines. Mark produces more traditional academic works, such as journal publications and reviews of scholarly works, and he plays the role of an academic advisor to his Ph.D. graduate students. Allison, on the other hand, is both creative and academic in that she directs plays and writes books. Because her area of expertise is not as traditional as Mark’s, she faces unique challenges in understanding where to place her accomplishments on the Bio-Bib report. We selected Allison as our primary persona because we felt that if we could build a system to support the unique needs of a professor in the field of creative arts (someone who might be frustrated by blank white space on the form or might be confused about which areas of the form to record her creative accomplishments), we would also address the needs of academics in more traditional fields as well. We chose Jennifer as our secondary persona because we wanted to make the form easy to use for professors who were new to the process, and who were involved in a variety of academic and community activities. We decided not to design for Mark because of his limited motivation to complete the Bio-Bib; however, we kept him as a personae to remind ourselves that our final design should be non-obtrusive for someone like him. A particularly important part of our group process was to narrow the focus of our project. We decided to concentrate on the “Research—either publications or creative activity” and “Professional activities” portions of the Bio-bib because we felt that these were areas in which creative and academic works would differ most. As we assessed where certain achievements would belong, we noted that “Research” usually referred to University-related activities, while the “Professional Services” referred more to non-University activities and accomplishments. The process of refining our personae and limiting our scope helped us to focus our list of tasks. Instead of delineating tasks by the amount of effort exerted, we felt it was more helpful to define each task from the vantage point of a specific personae. This helped us to prioritize our tasks and to understand the complexity of entering information into the two areas of the form on which we chose to focus. Primary Personae – Allison Randall
Allison Randall is a 37-year-old Associate Professor of Dramatic Arts at UC Berkeley. Her husband is 39 and serves as an editor for Sunset Magazine (a West-coast lifestyle publication). He travels frequently. As they both approach 40, they have decided that now is the time to finally start a family.
Justification: Secondary Personae – Jenny Williams-Ortiz
Jennifer Williams-Ortiz is a 29 year-old Assistant Professor in the Department of Cognitive Psychology, but also has a co-appointment in Linguistics. She has been at UC Berkeley for three years, and was hired at the same time as her husband, who is a 34 year-old Associate Professor in Linguistics. Jennifer did a post-doc at the University of Michigan, and studies English language acquisition among Mexican-American immigrants. She currently serves as primary advisor for four Ph.D. students and secondary advisor for another four Ph.D. students who comprise her research group that meets informally every week. Last year, she received a grant from the National Institutes of Health to set up a lab in the department. She is eager to attract the best graduate students to her lab and start establishing her work. Jennifer is therefore very active within the department and frequently attends all of the department mixers. She also sees students regularly during her open office hours. Jennifer is up for her 2nd review in the Psychology Department, and is eager to impress her colleagues. Outside of work, Jennifer enjoys playing volleyball with friends, camping and knitting the occasional scarf. Goals:
Justification: Tertiary Personae – Mark Lund
Mark Lund is a 62-year old fully-tenured professor in the Chemistry Department at UC Berkeley. He has been married for 30 years to a counselor who works at Albany High School, and has two adult sons named John and Kyle. He also has two grandchildren through John: Lisa, who is 4 and Joel, who is 7. Last year, after taking a long overdue family vacation with his wife and grandchildren to DisneyWorld, he promised his wife that he would work on maintaining a healthier work/life balance, and reserve more time for his family. Although Mark and his wife have been thinking about traveling more lately, he does not plan to retire immediately. Because of his busy schedule, Mark is extremely selective about how he spends his time. He works closely with one Post-Doc, who assumes the responsibility of overseeing his lab. He also mentors 4 handpicked ambitious Ph.D. students, and meets with them by appointment to help supervise their independent research projects. Although he is less “hands on” than some other professors in his department, he provides his graduate students with focused guidance and feedback on their research, so that they can extend his research legacy. He also serves the university as Chair of the Committee on Committees. Mark specializes in environmental and physical chemistry, and is well regarded in the field. On average, he publishes 2 articles in peer-reviewed journals per year, typically allowing one of his graduate students to take first authorship if they work closely with him on the publication. Three years ago, he obtained a grant to work on a project related to carbon sequestration, but this grant ends in August 2007. He is currently seeking funding to continue this line of research for a few more years in order to wrap up his research and make his last publication a major one – perhaps even in the prestigious journal Science. Goals:
Justification: Mark is different from the other personae in that he has reached the highest level of tenure, and is no longer eligible for salary increases. He doesn’t understand the purpose of the report because as far as he is concerned, no one in the department reviews it. Mark’s desire to complete the form with minimal time and effort, along with his aversion for paperwork, make him representative of a significant portion of UC Berkeley professors. TasksThe following task list was derived from required tasks by the aforementioned personae. We understand that there is a larger set of tasks of which these tasks will be viewed as archetypical but we feel that by addressing these we will also be addressing the more abstract set. For example, where Allison might direct a play, another professor might write a musical composition—both require a mental model and physical means to express this creative work.
Y = Yes; N = No ScenariosAnd Cut? – Allison RandallGiven her expertise, the highly recognized American Conservatory Theater has just asked Allison to direct a Henrik Ibsen play. As high an honor as this is, Allison needs to ensure that directing the play will not detract from her attaining tenure. Allison is reviewing her Bio-Bib to see whether this will strengthen her effort; she needs to clearly see that directing a play at a distinguished outside institution would display itself prominently in her Bio-Bib. Just the Facts Ma’am – Allison RandallAllison has been working on her book of 19th century playwrights for three years. She has a signed contract with a publisher and it is due for release on September 15. She has twelve complete chapters that have already been copy-edited by the publisher; all she has left is the conclusion and introduction—these will not be complete before the end of the academic year. Completion of this book will validate her credibility and status in the dramatic art community. It is much more prestigious than journal articles in this field. In her Bio-Bib, Allison must reflect her extensive effort and quasi-completion of a book that will be neither formally complete nor formally published until after her Bio-Bib report is due. The book has a promised publication date and its completion is—for the most part—a formality. The Lab – Jennifer Williams-OrtizIn the past two years since her first review, Jennifer has received a grant from the National Institutes of Health to establish her own lab in the Psychology department. Since then she has invested significant amounts of time developing and shaping her lab. Not only was the process of applying for the grant labor-intensive, the process of purchasing equipment, supplies and creating a research plan for the lab has taken large amounts of time. She is currently in the final set-up phase of attracting a few more students and getting her first projects up and running. For Jennifer, the Bio-Bib must have a language and space to acknowledge the time and effort she has put into funding and developing her new lab. Type, Cut, Paste – Jennifer Williams-OrtizIt is Saturday, August 12, the day Jennifer has set aside to complete the Bio-Bib. On her laptop in her small study room, she opens the Bio-Bib document she received via email from her department secretary. Jennifer regularly keeps an electronic record of her professional activities and achievements. Most of these, like her co-publications with students and work with the Berkeley Unified School District as an ESL expert, are on her CV, which she keeps up-to-date. Those that are not on her CV, she stores in an electronic folder. She keeps a small stack of potentially useful paper documents in her special red folder in her office desk drawer. She has brought her red folder home to complete her report. Jennifer wants easy transfer of data, as close to automatic as possible. For her, the publications/professional activities section of the Bio-Bib must be able to accommodate the cutting and pasting of information from her CV and other on-line documents to minimize typing. Comparative AnalysisComparative Analysis #1 Positive:
Negative:
Positive:
Negative:
![]() Positive:
Negative:
Comparative Analysis #2
Comparative Analysis #3
Comparative Analysis #4 Positive:
Comparative Analysis #5 Internship application forms call for modular entry of data. These screenshots come from two online forms.
Negative:
Initial Design IdeasInitial Design #1
Initial Design #2
Work Breakdown
AppendixInterviewsInterview Analysis – Executive Summary
Findings – Interview AInterviewee A is a fully tenured professor teaching in an artistic discipline. She has a physical disability which impacts her use of mouse and keyboard. AdaptationInterviewee A walked me through multiple places in a past Bio-Bib where she described and item she wanted to input and asked me the question: ‘Where do I put this.’ This raised my awareness of the plethora of terms and roles in the Arts that were not reflected in the Bio-Bib. She stated that she chose to place a particular achievement in the ‘refereed publications’ subsection of the Bio-Bib even though she did not know what a refereed publication was. There was no obvious place for various roles she had performed or achievements she had earned. When asked the question: ‘Do you feel that the Bio-Bib adequately reflects your efforts and achievements?’ she astonishingly said ‘yes.’ She never retracted the statement but consistently commented how she made it ‘adapt.’ She did not seem frustrated, but merely accustomed to not fitting in. I would not say that she liked the Bio-Bib, but viewed it as an inescapable reality. This expectation to adapt could be reflective of her wok as an artist where non-traditional work is common and/or because of her disability which could have built an expectation of being different and requiring extra effort to fit in. It seemed very clear that the Bio-Bib did not offer any obvious place for expressing efforts in her discipline and would always force her to be in the other category. It did not seem to validate her work. In contrast, when I began to question her about the different roles and functions that she performed, her energy level and vivacity increased dramatically. There was a strong sense of self-efficacy as she described her work in the terms of her own field and discipline. Whether she was simply more open-minded or because she had to adapt everything in her Bio-Bib, she would include things that weren’t personal work or effort but things about her, e.g. a neurological study written about her condition and art work. Promotion WorryI was also aware that she seemed very confident that what she input—regardless of whether it was properly categorized— would be properly interpreted. There appeared a strong sense that her report would be understand by future readers and that she would get a fair ruling from those evaluating her. Notifications and SubmissionsInterviewee A is notified to begin her Bio-Bib on July 1 and is asked to submit it by September 15. The timing of the requests inclines her to put less effort into the report because she is ‘winding down her semester’ and is desirous to take a break. She was never requested to complete the Bio-Bib annually, nor did she. She only completed the Bio-Bib during years where she was a candidate for promotion. Inconsistency and IndependenceShe commented that because she was uncertain where to place things, she often input them in different sections for different years. Clearly delineating where to place things could greatly enable longitudinal consistency over all her reports. Further, she never asked for input or help regarding what to input; she made her own assessment and input her data as she saw fit. Whether academic pride or self-confidence, help would not be sought outside of the form. Motor SkillsInterviewee A typed in everything using only one finger, and her mouse movements were very limited. Drag and drop were completely out of the question and cut and paste functions seemed questionable unless she could more manually select large sections by double or triple clicking. She compensated by inputting less data and/or very little description/narrative.
Findings – Interview BOur second interview was with a tenured professor of Cognitive Psychology. He had been in the department for many years, but filled out the form only when we was up for review, which was every three years or so. He had somewhat conflicting opinions on the usefulness of the form, since he saw it as “superfluous”, but also necessary for promotion. As far as he knew, no one checked to see that he filled it out accurately, and he doubted that anyone looked at it very carefully. On the other hand, he had been on the review committee for professors in his department, and he appreciated the fact that the Bio-Bib listed just the facts, unlike the personal statement, where you could “spin” your accomplishments. For its objectivity, he thought that it might be helpful for the review and promotion of younger professors. This professor expressed a great deal of frustration about the time and effort it took to type and recreate a stack of paper information that was provided to him by the departmental secretary. He could cut and paste the publication information from his CV, but filling out the majority of the report (particularly the teaching and mentoring information) was extremely tedious because he needed to type it in himself. Even with the assistance of the departmental secretary (she would send him the report, along with all of the raw materials to fill it out), he felt that the process was extremely inefficient. Findings – Interview COur third interview was also with a professor from the department of Cognitive Psychology. He had reached the highest level of tenure, and found no reason to complete the Bio-Bib report since he was no longer eligible for salary increases or promotions. He had a great deal to say about the report; however, because he felt the Bio-Bib could be much improved if professors were required to outline their professional goals, and then complete the form with these goals in mind. He emphasized the importance of setting goals, such as his goal: “better understanding the cognitive development of infants,” so that professors could use the Bio-Bib report as an opportunity to assess their progress towards these goals. This professor also talked extensively about the importance of outlining the level of involvement that professors have in graduate training. Some professors will meet with their students once every few weeks, and provide only minimal guidance, while other professors will act as collaborators on research projects and spend hours each week working with students. He found it problematic that the Bio-Bib report, in its current form, does not distinguish between these levels of involvement. Lastly, this professor brought an important consideration to light – he has a visual impairment that makes it difficult for him to scan large paragraphs of text. He said that although it’s difficult to accommodate all disabilities, it is important to create a product that limits the amount of reading and typing that professors need to do in order to complete the form. Findings – Interview D
She received an e-mail from her department administrator when it was time to complete the Bio-Bib. Her mother-in-law and her husband are both professors. They advised her, before she began working at UC Berkeley, to keep track of "everything" that might help her in her tenure review case. She keeps a folder in her e-mail account for professional activity related messages. She also prints out copies of documents that may be relevant. She estimates that 90% or her records are in electronic form and 50% are in hard copy form (so there is duplication in her record keeping.). For example, she changed an Excel file on her laptop, where she lists her interactions and capacities with graduate students (e.g. committee work). Most of her hard copy papers are kept in individual folders by topic (e.g. grants, classes taught etc.). Some of the smaller topics (e.g., short-term student committee work, etc.) are kept in a small open stack pile in her cabinet. In her department, the administrative Secretary also e-mailed professors a copy of the Dean's form that shows a list of their official professional activities as recognized by the Dean's office. She submits the document to the departments administrative Secretary, who sends it to the review committee. She prints out and turns in a hard copy form to the administrative Secretary, as her primary means of submitting the form. She also sends the administrative Secretary an e-mail version as backup, and keeps an electronic copy of the Bio-Bib on her computer. She estimates that she spends approximately three to six hours total to complete the Bio-Bib, including time spent filing and searching for documents. However, when asked for specifics of where she would acquire certain data, unaccounted for preparation time kept increasing. She did not feel that the Bio-Bib of adequately reflected her accomplishments and achievements. She felt that the form was highly weighted towards the prestige of publications. "The whole experience makes me feel like I haven't been doing anything here." "It's really a difficult thing to do on your ego." Because of hiring timing, she filled out her first Bio-Bib during the first semester of her time at UC Berkeley. This was after only two months of her arrival, when she "had done nothing" yet. She remembers having a big empty form, but remarks the next time she filled it out, she had more to enter. "It's good because it reminds you. In some departments, it's not clear what you need to do to get tenure. So this form helps you organize." She suggests that there should possibly be two different forms for junior and senior faculty. She states that setting up a lab takes a lot of time. Senior faculty do not have such draws on their time. She felt that while junior faculty shouldn't explicitly try to take credit for activities such as setting up a lab, she did feel that the effort that goes into it and time spent should be recognized by the administration. There is space on the Bio-Bib for noting the development of a new class, she points out, but there is no space for noting the development of a new lab. Hands on mentoring of students and other similar activities, builds community in the department and deserves recognition, because it creates a better research environment for everyone. The first time she completed the Bio-Bib she was unsure at times about what to put where. It was not always clear whether or not something should or could be included in the form. For example, she wrote a solicited book review for a book review journal. "It's not quite peer review, but it's not a regular book review either. So how do I classify that?” She suggests that a tutorial be created to help new users understand the process. She discusses the process of sneaking in publications and other activities that are in progress on the form. She appears unaware that the form explicitly prohibits works in progress. The most time-consuming aspect of completing the Bio-Bib, she felt, was finding all of the records keeping track of them and remembering where they are located. She suggests that since, some new professors are never told how to prepare for this that it might be beneficial to include in hiring packages for new faculty, a notice that they will need to keep track of all of their professional activities in order to complete the Bio-Bib and other documents that comprise the review package. She was interested in trying to figure out a way to quantify time invested for students and mentoring. She relates the story of occasionally, having served on a committee, but not in an official capacity. She would read the dissertation and attend the defense, but because she was not formally listed as a committee member, she "didn't get credit" for her work, because it was not on the Dean's list of her activities. She attempted to explain the work she did, that had no explicit place on the Bio-Bib in her personal statement. She suggested different ways to measure time spent with students. For example, how well do a professor students do on the job market? What about a professor’s student finishing rate? Attrition/loss rate? She would consider putting some of the information in the Bio-Bib online, mainly as a way to attract perspective graduate students to work with her. She did not think it would be useful for her colleagues.
|