Technology and Delegation, Fall 2019

From Techdel
Jump to: navigation, search

This is the syllabus for the Fall 2019 INFO 239 Technology and Delegation Lab 2019, taught by Deirdre K. Mulligan (dmulligan@berkeley[dot]edu) and Daniel Griffin (daniel.griffin@berkeley[dot]edu).

(See also the 2018, 2017, 2016, 2015, 2013, 2011, 2010 and 2009 versions.)

Class meeting
Tuesday and Thursday 3:30-5:00 PM; 202 South Hall
Office hours
Deirdre: Wednesdays 2:15pm-3:15pm, 303b South Hall or by appointment
Daniel (to be confirmed): Thursdays 11am-noon, Rm 2 South Hall (PhD Office at the south end of the lower floor. Knock if the door is locked), or by appointment.
mailing list
#techdel_class on Slack
(email Daniel if you have problems joining either - non-I School students will need to email Daniel to get access to the Slack channel)

Course description

Themistocles said his infant son ruled all Greece -- "Athens rules all Greece; I control Athens; my wife controls me; and my infant son controls her." Thus, nowadays the world is controlled by whoever buys advertising time on Dora the Explorer.
XKCD highlights technology and delegation.

Technology is often put forth as inevitable progress toward modernization and as a value-neutral means for implementing the policies of law, agency rules, or corporate planning. The introduction of technology increasingly delegates responsibility for some function — sensing, sensemaking, deciding, acting — to technical actors. These handoffs of function between people, processes and technologies are rarely as inconsequential as they are initially made out to be. Often these shifts reduce traditional forms of transparency—as black boxes embed rules and make decisions less visible—and challenge traditional methods for accountability.

Meanwhile, policymakers are asking those who design technology to build-in values such as privacy and fairness—handing off some responsibility for their expression and protection. It is rarely a simple case of a function being handed wholesale from people to technology, or one technology to another; but rather a complicated interleaving in which a value is shaped and constrained by multiple modes of regulation.

We will draw on a wide range of literature, including: design, science and technology studies, computer science, law, and ethics, as well as primary sources in policy, standards and source code. We will explore the interaction between technical design and values including: privacy, accessibility, fairness, and freedom of expression. We will investigate approaches to identifying the value implications of technical designs and use methods and tools for intentionally building in values at the outset. And we will be not only critical, but also constructive: this lab will give us a hands-on opportunity to try out technologies for ourselves and experiment with building alternatives that address rights and values.


First, we will look at the theory and examples of social values and the effects of technological delegation on those values; next, we will review and test the practices, tools and methodologies to design for values we care about; and finally, we will consider some ongoing case studies to discover what affirmative agenda or common principles we can elicit for embedding values in design.

Typically, Tuesday meetings will be discussions based on the readings and Thursday meetings more lab exercises, but this will vary.

Week 0: Introduction

Thursday, August 29

We will introduce the content and methods of the course, and introduce ourselves to one another.


  • Winner, Langdon. "Do Artifacts Have Politics?" Daedalus, Vol. 109, No. 1, Modern Technology: Problem or Opportunity? (Winter, 1980), pp. 121-136 The MIT Press on behalf of American Academy of Arts & Sciences
  • Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, "[1]" Machine Bias There’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica May 23, 2016
  • Technical Flaws of Pretrial Risk Assessments Raise Grave Concerns "[2]"

Week 1: Values

Tuesday, September 3; Thursday, September 5

What are values? How do they relate to rights, preferences, valuation? How do different disciplines view and situate values-related work?




  • Before class: work with a partner to find a value that interests you, and see how that value is discussed in different ways, or by different fields. (sign up on the class sign up sheet)
  • Class: Provide a short oral presentation on your findings. (3 minutes maximum, followed by 4-5 minutes of feedback and discussion by the class)

Assignment 1

Due Wednesday September 11 at 11:59pm

  • With your lab partner, provide a write up on your value from this week's lab (which will be posted to the TechDel repository), the multiple ways it gets discussed. 2-4 pages (single spaced), and your preferred way of positioning it (value, right, preference etc.). Make sure references are cited (use any citation style you’d like, but be consistent). Submit as a pair via email to Deirdre and Daniel.
  • If you need help thinking about what values to look at, consider the examples listed on the TechDel repository, Koops paper, or the following paper (see Table 2 in particular): Cheng, An-Shou, and Kenneth R. Fleischmann. 2010. "Developing a meta-inventory of human values." Proceedings of the 73rd ASIS&T Annual Meeting on Navigating Streams in an Information Ecosystem-Volume 47.

Week 2: Why Does Technology Matter?

Tuesday, September 10; Thursday, September 12

We'll continue discussing artifacts and politics, as well as what things regulate.


  • Latour, Bruno, "The Moral Dilemmas of a Safety-belt." 1989
  • Akrich, M. (1992). The description of technical objects. In Shaping technology/building society, studies in socio technical change, edited by WE Bijker and J. Law, 205-224. Cambridge, MA: MIT Press.
  • Kleinberg, Jon, Sendhil Mullainathan, and Manish Raghavan. "[3]" Inherent trade-offs in the fair determination of risk scores . arXiv preprint arXiv:1609.05807 (2016). (Read the Abstract, Section 1, and Section 5 -- if you're really digging the math, feel free to read the other sections as well, but they are by no means necessary)
  • Christin, A. (2017). "[4]"Algorithms in practice: Comparing web journalism and criminal justice. Big Data & Society, 4(2), 2053951717718855.


  • Before Thursday's class: work with a partner to interrogate a consumer device or app or system or service using the readings as guidance and inspiration. What is the script? How has the rearrangement of the activity by the technical system shifted responsibility for morality? What, if any, values have to be computationally addressed in this new configuration?(sign up on the class sign up sheet)
  • Class: Provide a short oral presentation with appropriate visual aids (slides or something else) (3-4 minutes max, with 5-6 minutes of discussion) on your findings
    • Document your methods: how you explored the artifact, why you chose them
    • Address the questions
    • What did you find

Assignment 2

Due Friday September 20 by 10:00 pm

  • With your lab partner, provide a write up of your analysis from lab, with improvements based on feedback during class (which will be posted to the TechDel repository). 2-4 pages (single spaced). Make sure references are cited (use any citation style you’d like, but be consistent). Submit as a pair (slides or whatever from presentation and write up) via email to Deirdre and Daniel.

Week 3: Protecting Values

Tuesday, September 17; Thursday, September 19

9/17 We'll explore the implications of using different modalities—technical designs, legal rules, norms, markets—to regulate.


9/19 Legitimacy and Governance


Week 4: Legitimacy and Governance con’t

Tuesday, September 24; Thursday, September 26 (Deirdre away Tuesday)

9/24 Lab:

  • We'll do some brainstorming activities to think about potential Final Project ideas

And file a Public Record Act Request for details on an algorithmic tool in government use [5]

9/26 Discussion of search and content moderation issues.


Assignment 3

Due Friday October 25 10 pm

  • Individually, write a short analysis looking at governance issues in relation to a specific case (could be something related to your final project ideas, or could be another case you’re interested in). Your analysis should deeply engage with (not just summarize) at least one of the readings from the course (we suggest using the readings from the second half of week 3 or from week 7). You’re also welcome to engage with additional readings from other weeks.
  • 2-4 pages (single spaced). Make sure references are cited (use any citation style you’d like, but be consistent). Submit individually via email to Deirdre and Daniel.

Week 5: Studying Values

Tuesday, October 1; Thursday, October 3



  • We'll do some activities with the Moral Machine and Moderation Machine websites. Bring your laptop/tablet to class.

Week 6: Values in Design

Tuesday, October 8; Thursday, October 10

Working with values. What values might we want to support through design? What do we mean by “by design”? Given their contested and contextual nature, how can we model values?



  • Working session thinking through values and design choices for a particular technical system. Please come with something you and 3-4 classmates want to interrogate using Tuesday’s readings.

Week 7: Designing for Law & Regulation

Tuesday, October 15; Thursday, October 17 (Deirdre away Thursday)

What are different ways to think about the relationship between law and technological design?


Lab (First half):

  • By 2pm Thursday 3:30pm Tuesday 10/22, add 1 slide to the shared Google Slide deck with your idea for a final project:
  • Everybody will have 60 seconds to pitch an idea for their final project (we'll use a timer!)
  • If you aren't going to be in class physically, you can still put in a slide and we'll share it with the class
  • We'll give some time for people to ask questions

Lab (Second half)

Week 8: User Centered Design & Value Sensitive Design

Tuesday, October 22; Thursday, October 24



  • Norman, Don. (1988). The Psychopathology of Everyday Things, Chapter 1 in The Design of Everyday Things (1-33)
  • Maguire, Martin. (2001). Methods to support human-centred design. International journal of human-computer studies, 55(4), Read Sections 1-3 (pg 587-593 and skim the rest).
  • Friedman, Batya, et al. "Value sensitive design and information systems." in Early engagement and new technologies: Opening up the laboratory. Springer Netherlands, 2013. 55-95. Read 4.1 - 4.3 (pg 55-61), Read 1 case study in section 4.4 & skim the others, read 4.6 (pg 74-79), and read "4.8.1:Practical Value Sensitive Design Challenges" (pg 85-88) after the citations.


  • Let’s try out some of the Value Sensitive Design tools (envisioning cards, etc)

Week 9: Critically Oriented Design Approaches

Tuesday, October 29; Thursday, October 31 (Deirdre away Tuesday)




Week 10: Engaging Stakeholders

Tuesday, November 5; Thursday, November 7



  • Before class, meet with your team and identify stakeholders, models and tools for engagement for your final project
  • Prepare a brief presentation for class to get feedback

Week 11: Impact Assessments & Frameworks

Tuesday, November 12; Thursday, November 14 (Deirdre away Thursday)


  • Cooper, A., Tschofenig, H., Aboba, B., Peterson, J., Morris, J., Hansen, M., Smith Janet, R. RFC6973 Privacy Considerations for Internet Protocols. 2013.
  • Department of Homeland Security. Privacy Impact Assessment template. Introduction and the Template (just skim, no need to print these out).
  • Reisman, Dillon, Jason Schultz, Kate Crawford, and Meredith Whittaker [14] Algorithmic Impact Assessments: A Practical Framework For Public Agency Accountability AI Now Institute(2018).
  • Andrew D Selbst, Danah Boyd, Sorelle A Friedler, Suresh Venkatasubramanian, and Janet Vertesi. "[15]" Fairness and abstraction in sociotechnical systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 59–68, 2019.

Choose 1 of these to look at (will be updated with more):

  • Microsoft, Custom Neural Voice gating overview [16]
  • Josh Lovejoy Human-centered AI cheat-sheet [17]
  • Aequitas Bias and Fairness Audit Toolkit, University of Chicago Center for Data Science and Public Policy[18]
  • Natalie Cadranel, Anqi Li, An Xiao Mina, Caroline Sinders. 2018. Digital Security and Privacy Protection UX Checklist
  • Ethical OS Toolkit
  • EFF Know your customer standards applied to ICE [19]
  • Markkula Center for Applied Ethics, An Ethical Toolkit for Engineering/Design Practice [20]
  • Ethics and Algorithms Tool kit [21]
  • IEEE Ethically Aligned Design [22] (feel free to look at general principles and a section or two if you don't want to look at the entire doc, its long)
  • CDT's Digital Decisions Tool [23] and backgrounder [24]
  • Model Cards for Model Reporting [25]

Tuesday we'll be discussing the variety of toolkits and frameworks that folks have looked at. Lab: Thursday we'll try implementing them to compare and contrast their utility.


Assignment 4

Due Tuesday, November 26 at 10pm

  • During weeks 5-12, we’ve discussed different methods and tools for thinking about values in design. Individually, apply one of the methods or tools to a problem or artifact. Write up an analysis, including:
    • Your problem/artifact that you are apply the method/tool to.
    • Document your method/tool used and why you chose it
    • What did you find?
    • What was useful about this method/tool? What was less useful or difficult?
    • What outstanding issues have not been addressed by this method/tool
  • This does not have to be focused on your final project topic. However, you might find it helpful to use this as an opportunity to apply some of the methods/or tools to your final project topic
  • 2-4 pages (single spaced). Make sure references are cited (use any citation style you’d like, but be consistent). Submit individually via email to Deirdre and Daniel.

Note: You may optionally complete the assignment in pairs, but if you do so, we expect the analysis to be a little more deeper and submit a single paper in the 4-8 page range)

Week 12: Analyzing Systems

Tuesday, November 19; Thursday, November 21

We've talked about a lot of tools and approaches to use in the design of systems, but how might we evaluate the tradeoffs of the choices we have? And how might we critique and analyze existing systems?




  • Let's apply the handoffs model to a system

Week 13: Organizational Change

Tuesday, November 26; [no class 11/28, Thanksgiving]

How might we consider the organizational, social, and cultural contexts in which we try to apply the methods and tools we've discussed? What challenges might occur?




  • No Lab, Happy Thanksgiving!

Week 14: Final Project Presentations

Tuesday, December 3; Thursday, December 5

Each group will give a short presentation on their final project for class feedback.

Week 15 (RRR): No Class, Work on Final Projects

Thursday, December 6

Final projects are due December 20 (at or before 11:59pm)

Course policies


This is a 3-unit class. Meetings each week will include seminar-style discussion, lab-style exercises, and collaborative design and technical work. Readings should be read carefully. Throughout the semester students will complete short writing assignments, lead class discussions, and critique artifacts during the semester. During the second half of the semester students will work collaboratively in teams of 2-4 on a larger design-oriented final project.


Your grade is based on class participation 15% and assignments 85%. This class is designed to hone your critical inquiry skills. You are expected to fully participate—present, actively listen, engage with your classmates and the materials, bring your own insights to the discussion, share your experience and knowledge. Please come prepared to argue, explain, revise, borrow, refine, and of course junk your ideas. Thinking out loud is encouraged. This is how one learns. The success of this class depends upon student’s diligent preparation and active participation—listening, speaking, designing, building—in class. Readings will be assigned throughout the semester. Everyone is expected to read and reflect on the readings.

Assignments account for the remainder of your grade. They will be graded primarily on substance, however a minimal portion of each grade will reflect organizational clarity, grammar, and presentation style as appropriate. The assignments are staggered throughout the semester.


1. Written Assignments 50% (4 assignments, 12.5% each) The written assignments are an invitation to apply both the theoretical and practical learning from the course to new problems. They are designed to develop your skills as readers--critiquing, building upon, relating various pieces we read. Reflection pieces should synthesize readings and ideas from class discussion, and use the resulting insights to analyze an issue or object of interest to you, critique readings, or anything else you would like. Specific requirements are described in each assignment, but for all assignments, you must 1) seriously engage with the readings (could be 1 could be 2 could be more); and 2) write about something that interests you.

Do not, under any circumstances, provide a summary of the articles. You've read them. We've read them. We know you've read them. The form is in between short essay and journal entry. This is a playful style. We want to get a sense of what you are taking away from the class and what sorts of thoughts, ideas, questions it is raising for you. 2-4 normal single spaced pages typed. More details for each assignment can be found here:

2. Critiques 10% The second half of the semester we will begin each class with a student led critique of an information or physical artifact. The students assigned (teams of 2) will select an interface, object, algorithm, design, instructable, kickstarter, toy, etc. and offer a brief—4 minute maximum—critique that introduces the item and reflects on its values implications drawing on class readings and assignments up to that point. The class will then collectively critique the artifact.

3. Final Project 25% DUE December 20 (hard deadline) During the second half of the semester students will work collaboratively in teams of 2-4 on a larger final project. While we are flexible on the type of project, projects might include: a design-oriented project where something is analyzed or prototyped, writing out a research protocol for further exploration, or writing a part of a paper aimed toward publication.

Late assignments will be penalized: each day an assignment is late will result in a half a grade deduction. Recognizing that emergencies arise, exceptions will be made on a case-by-case basis.

Scheduling: Use [TBD] to choose which week you'll do a design critique, and to list who you're working with for Assignments 1 and 2.


The high academic standard at the University of California, Berkeley, is reflected in each degree that is awarded. As a result, every student is expected to maintain this high standard by ensuring that all academic work reflects unique ideas or properly attributes the ideas to the original sources. Academic integrity for this course depends on clear citation of ideas, text and code. At this level, we harbor no romantic illusions that work is born whole from your mind alone; instead, we want you to engage with the assigned readings and ideas you connect from other classes or the world around you, and to make those connections clear. Programmers often copy-paste code or re-use libraries in order to build upon the shoulders of giants without reinventing the wheel. This is encouraged, but we expect students to be careful in their assignments to note what they wrote themselves and to attribute code snippets and libraries to their original authors. These are some basic expectations of students with regards to academic integrity: Any work submitted should be your own individual thoughts, and should not have been submitted for credit in another course unless you have prior written permission to re-use it in this course from this instructor. All assignments must use "proper attribution," meaning that you have identified the original source and extent or words or ideas that you reproduce or use in your assignment. This includes drafts and homework assignments! If you are unclear about expectations, ask. Do not collaborate or work with other students on assignments or projects unless you have been directed to do so. For more information visit:

Research Ethics

Class projects for the purpose of learning are typically exempt from Berkeley’s Committee for the Protection of Human Subjects (our Institutional Review Board) review. However, if you are conducting a research project that might result in publishable work (and many of you may!), keep in mind that UCB policy and publication venues will typically require you to have gone through the IRB process even if it is to receive an exemption. In addition, being exempt from IRB review does not mean that you are somehow exempt from ethical norms or that you wouldn’t benefit from advice or review. Feel free to come to the instructors for advice on a project which involves intervention with, data collection, or use of data about, human subjects.

Diversity and Inclusion

We value diversity in this course. Diversity can stem from disciplinary training, political or religious beliefs, identity (including race, gender, nationality, class, sexuality, religion, ability, caregiving roles, etc.), professional and personal experiences. We welcome it all. We aim to create an inclusive classroom. The syllabus includes a wide range of scholars and practitioners representing diverse perspectives. Due to the current composition of fields it is not optimal. If there are articles, papers, or other materials relevant to this course that you think would enhance the goals of diversity please share them with us, and feel free to share them (without asking me first) on the course slack channel.

If you go by a name that differs from what appears in your official UC-Berkeley records, or if you have preferred pronouns, please let me know. I struggle with names, please do not hesitate to remind me publicly or privately if I mispronounce your name.

I welcome accommodation letters from the Disabled Students’ Program. Know that I read them and adjust my course design and/or course policies accordingly.

I acknowledge that having a personal computer available to you makes keeping up with course readings and completing class assignments much easier. If you are struggling with the expense of technology or with unreliable equipment or if you are having any trouble with accessing materials (books, websites) for this course I want to know. The UC-Berkeley library has a laptop loan program.

Life doesn’t stop because you are a student. If experiences outside of school are interfering with your studies, please don’t hesitate to come and talk with me. If you prefer to speak with someone outside of the course or the program, our campus-side diversity office is available (email:, phone: 510-642-7294).

I (like many people) am still in the process of learning about diverse perspectives and identities. If something was said in class (by anyone, including me) that you feel was unfair, ill-informed, or personally hurtful, please talk to me about it. To report any incident of intolerance, hate, harassment or exclusion on campus or by members of the campus community, you can start here.

Finally, as a participant in course discussions, you should also strive to honor the diversity of your classmates.

UC Berkeley Statement on Diversity These principles of community for the University of California, Berkeley are rooted in a mission of teaching, research and public service and will be enforced in our classroom this term. We place honesty and integrity in our teaching, learning, research and administration at the highest level. We recognize the intrinsic relationship between diversity and excellence in all our endeavors. We affirm the dignity of all individuals and strive to uphold a just community in which discrimination and hate are not tolerated. We are committed to ensuring freedom of expression and dialogue that elicits the full spectrum of views held by our varied communities. We respect the differences as well as the commonalities that bring us together and call for civility and respect in our personal interactions. We believe that active participation and leadership in addressing the most pressing issues facing our local and global communities are central to our educational mission. We embrace open and equitable access to opportunities for learning and development as our obligation and goal. For more information, visit UC Berkeley's Division of Equity, Inclusion & Diversity page:

Learning Accommodations & Access

If you need accommodations for any physical, psychological, or learning disability, please speak to us after class or during office hours. If appropriate, please obtain an accommodation letter from the Disabled Students’ Program.

Additional Campus Resources

These additional campus units may, at times, prove helpful during the course of the semester: