Software Decisionmaking Archives - Columbia Law Review

The full text of this essay may be found by clicking the PDF link to the left.

Introduction Many of the developments that go under the banner of artificial intelligence that matter to the legal system are not so much new means of breaking the law but of bypassing it as a means of enforcing rules and resolving disputes.[footnote]In this Essay, the broader meaning of “artificial intelligence” is used—namely a computer system that is “able to perform tasks normally requiring human intelligence” such as decisionmaking. Artificial Intelligence, Lexico, https://www.lexico.com/en/definition/ artificial_intelligence [https://perma.cc/86XR-2JZ8] (last visited July 31, 2019).[/footnote] Hence a major challenge that courts and the legal system will face over the next few decades is not only the normal challenge posed by hard cases but also the more existential challenge of supersession.[footnote]A small legal literature on these problems is emerging. See, e.g., Michael A. Livermore, Rule by Rules, in Computational Legal Studies: The Promise and Challenge of Data-Driven Legal Research (Ryan Whalen, ed.) (forthcoming 2019) (manuscript at 1–2), https://papers.ssrn.com/abstract=3387701 (on file with the Columbia Law Review ); Richard M. Re & Alicia Solow-Niederman, Developing Artificially Intelligent Justice, 22 Stan. Tech. L. Rev. 242, 247–62 (2019); Eugene Volokh, Chief Justice Robots, 68 Duke L.J. 1135, 1147–48 (2019).[/footnote] Here are a few examples. The control of forbidden speech in major fora, if once the domain of law and courts, has been moving to algorithmic judgment in the first instance.[footnote]See infra Part II.[/footnote] Speeding is widely detected and punished by software.[footnote]See Jeffrey A. Parness, Beyond Red Light Enforcement Against the Guilty but Innocent: Local Regulations of Secondary Culprits, 47 Willamette L. Rev. 259, 259 (2011) (“Automated traffic enforcement schemes, employing speed, and red light cameras, are increasingly used by local governments in the United States.” (footnote omitted)).[/footnote] Much enforcement of the intellectual property laws is already automated through encryption, copy protection, and automated takedowns.[footnote]See infra Part I.[/footnote] Public prices once set by agencies (like taxi prices) are often effectively supplanted by prices set by algorithm.[footnote]See Jessica Leber, The Secrets of Uber’s Mysterious Surge Pricing Algorithm, Revealed, Fast Company (Oct. 29, 2015), https://www.fastcompany.com/3052703/the-secrets-of-ubers-mysterious-surge-pricing-algorithm-revealed [https://perma.cc/H7MB-SC8T].[/footnote] Blockchain agreements are beginning to offer an alternative mechanism to contract law for the forging of enforceable agreements.[footnote]See Eric Talley & Tim Wu, What Is Blockchain Good for? 5–9 (Feb. 28, 2018) (unpublished manuscript) (on file with the Columbia Law Review ).[/footnote] Software already plays a role in bail determination and sentencing,[footnote]See Algorithms in the Criminal Justice System, Elec. Privacy Info. Ctr., https://epic.org/algorithmic-transparency/crim-justice/ [https://perma.cc/KY2D-NQ2J] (last visited Apr. 23, 2019).[/footnote] and some are asking whether software will replace lawyers for writing briefs, and perhaps even replace judges.[footnote]See Volokh, supra note 2, at 1144–48, 1156–61.[/footnote] Are human courts just hanging on for a few decades until the software gets better? Some might think so, yet at many points in Anglo American legal history, courts have been thought obsolete, only to maintain their central role. There are, it turns out, advantages to adjudication as a form of social ordering that are difficult to replicate by any known means.[footnote]See Lon L. Fuller, The Forms and Limits of Adjudication, 92 Harv. L. Rev. 353, 357 (1978).[/footnote] This Essay predicts, even in areas in which software has begun to govern, that human courts[footnote]This Essay uses “courts” to refer to any adjudicative body, public or private, that resolves a dispute after hearing reasoned argument and explains the basis of its decision.[/footnote] will persist or be necessarily reinvented. It predicts, however, that human–machine hybrids will be the first replacement for human-only legal systems, and suggests, if done right, that there lies real promise in that approach. The case study of content control on online platforms and Facebook’s review board is used to support these descriptive and normative claims. The prediction that courts won’t wholly disappear may seem an easy one, but what’s more interesting is to ask why, when software is “eating” so many other areas of human endeavor. Compared with the legal system, software has enormous advantages of scale and efficacy of enforcement. It might tirelessly handle billions if not trillions of decisions in the time it takes a human court to decide a single case. And even more importantly, the design of software can function as an ex ante means of ordering that does not suffer the imperfections of law enforcement.[footnote]Lawrence Lessig, Code: Version 2.0, at 124–25 (2006).[/footnote] But human courts have their own advantages. One set of advantages, more obvious if perhaps more fragile, is related to procedural fairness. As between a decision made via software and court adjudication, the latter, even if delivering the same results, may yield deeper acceptance and greater public satisfaction.[footnote]See generally E. Allan Lind & Tom R. Tyler, The Social Psychology of Procedural Justice (Melvin J. Lerner ed., 1988) (providing a classic overview of the advantages human courts have over software because of procedural fairness considerations in human courts, potentially ignored by software, that lead to acceptance of their decisions by the public).[/footnote] In the future, the very fact of human decision—especially when the stakes are high—may become a mark of fairness.[footnote]See Aziz Z. Huq, A Right to a Human Decision, 105 Va. L. Rev. (forthcoming 2020) (manuscript at 21–22), https://ssrn.com/abstract=3382521 (on file with the Columbia Law Review ) (explaining that concerns about transparency have led some to demand human decisions).[/footnote] That said, society has gotten used to software’s replacement of humans in other areas, such as the booking of travel or the buying and selling of stocks, so this advantage may be fragile. A second, arguably more lasting advantage lies in human adjudication itself and its facility for “hard cases” that arise even in rule-based systems. Most systems of social ordering consist of rules, and a decisional system that was merely about obeying rules might be replaced by software quite easily. But real systems of human ordering, even those based on rules, aren’t like that.[footnote]This refers to a caricatured version of H.L.A. Hart’s view of what law is. See H.L.A. Hart, The Concept of Law 2–6 (Peter Cane, Tony Honoré & Jane Stapleton eds., 2d ed. 1961).[/footnote] Instead, disputes tend to be comprised of both “easy cases”—those covered by settled rules—and the aforementioned “hard cases”—disputes in which the boundaries of the rules become unclear, or where the rules contradict each other, or where enforcement of the rules implicates other principles.[footnote]Cf. Ronald Dworkin, Hard Cases, in Taking Rights Seriously 81, 81 (1978) (stating that when “hard cases” fall on the edge of clear rules, judges have the discretion to decide the case either way).[/footnote] There is often a subtle difference between the written rules and “real rules,” as Karl N. Llewellyn put it.[footnote]Karl N. Llewellyn, The Theory of Rules 72–74 (Frederick Schauer ed., 2011).[/footnote] Hence, a software system that is instructed to “follow the rules” will produce dangerous or absurd results. Justice Cardozo argued that the judicial process “in its highest reaches is not discovery, but creation . . . [by] forces . . . seldom fully in consciousness.”[footnote]Benjamin N. Cardozo, The Nature of the Judicial Process 166–67 (1921).[/footnote] Better results in hard cases may for a long time still depend instead on accessing something that remains, for now, human—that something variously known as moral reasoning, a sensitivity to evolving norms, or a pragmatic assessment of what works. It is, in any case, best expressed by the idea of exercising “judgment.” And if the courts do indeed have a special sauce, that is it. It is possible that even this special sauce will, in time, be replicated by software, yielding different questions.[footnote]See Huq, supra note 14, at 18; Volokh, supra note 2, at 1166–67, 1183–84.[/footnote] But as it stands, artificial intelligence (AI) systems have mainly succeeded in replicating human decisionmaking that involves following rules or pattern matching—chess and Jeopardy! are two examples.[footnote]See John Markoff, Computer Wins on ‘Jeopardy!’: Trivial, It’s Not, N.Y. Times (Feb. 16, 2011), https://www.nytimes.com/2011/02/17/science/17jeopardy-watson.html (on file with the Columbia Law Review ).[/footnote] It would risk embarrassment to argue that machines will never be able to make or explain reasoned decisions in a legal context, but the challenges faced are not trivial or easily overcome.[footnote]See Drew McDermott, Why Ethics Is a High Hurdle for AI 2 (2008), http://www.cs.yale.edu/homes/dvm/papers/ethical-machine.pdf [https://perma.cc/PZ7Z-5QFN]; Adam Elkus, How to Be Good: Why You Can’t Teach Human Values to Artificial Intelligence, Slate (Apr. 20, 2016), https://slate.com/technology/2016/04/why-you-cant-teach-human-values-to-artificial-intelligence.html [https://perma.cc/4LR9-Q2GH]. But see IBM’s project debater, a program that presents arguments on one side of an issue, and thereby might be thought to replicate aspects of lawyering. Project Debater, IBM, https://www.research.ibm.com/artificial-intelligence/project-debater/ [https://perma.cc/4A2W-XSB9] (last visited July 31, 2019).[/footnote] And even if software gets better at understanding the nuances of language, it may still face the deeper, jurisprudential challenges described here. That suggests that, for the foreseeable future, software systems that aim to replace systems of social ordering will succeed best as human–machine hybrids, mixing scale and efficacy with human adjudication for hard cases. They will be, in an older argot, “cyborg” systems of social ordering.[footnote]The Merriam-Webster dictionary defines “cyborg” as “a bionic human,” meaning a being comprised of mixed human and machine elements, like the fictional character “Darth Vader” from the twentieth-century film series Star Wars . See Cyborg, Merriam-Webster Dictionary, https://www.merriam-webster.com/dictionary/cyborg [https://perma.cc/FD6A-UWZ2] (last visited Aug. 31, 2019).[/footnote] When we look around, it turns out that such hybrid systems are already common. Machines make the routine decisions while leaving the hard cases for humans. A good example is the flying of an airplane, which, measured by time at the controls, is now mostly done by computers, but sensitive, difficult, and emergency situations are left to a human pilot.[footnote]Reem Nasr, Autopilot: What the System Can and Can’t Do, CNBC (Mar. 27, 2015), https://www.cnbc.com/2015/03/26/autopilot-what-the-system-can-and-cant-do.html [https://perma.cc/Q7CT-GBLN].[/footnote] The descriptive thesis of this Essay is supported by a case study of content control (the control of hate speech, obscenity, and other speech) on online platforms like Facebook and Twitter. Despite increasing automation, the generation of hard questions has yielded the development, by the major platforms, of deliberative bodies and systems of appeal, such as Facebook’s prototype content review board, designed to rule on the hardest of speech-related questions. In the control of online speech, and in the autopilot, we may be glimpsing, for better or worse, the future of social ordering in advanced societies. While automated justice may not sound appealing on its face, there is some real promise in the machine–human hybrid systems of social ordering described here. At their best, they would combine the scale and effectiveness of software with the capacity of human courts to detect errors and humanize the operation of the legal system. As Lon Fuller put it, human courts are “a device which gives formal and institutional expression to the influence of reasoned argument in human affairs.”[footnote]Fuller, supra note 10, at 366.[/footnote] What might the court system look like without scaling problems—if routine decisions went to machines, reducing the court’s own workload as a major factor in decisionmaking? To be sure, what could arise are inhumane, excessively rule-driven systems that include humans as mere tokens of legitimization,[footnote]This is a concern expressed in Re & Solow-Niederman, supra note 2, at 246–47.[/footnote] but hopefully we can do better than that. This Essay provides advice both for designers of important AI-decision systems and for government courts. As for the former, many AI systems outside of the law have aspired to a complete replacement of the underlying humans (the self-driving car,[footnote]Alex Davies, The WIRED Guide to Self-Driving Cars, WIRED (Dec. 13, 2018), https://www.wired.com/story/guide-self-driving-cars/ [https://perma.cc/7P3A-5NLE].[/footnote] the chess-playing AI[footnote]Natasha Regan & Matthew Sadler, DeepMinds’s Superhuman AI Is Rewriting How We Play Chess, WIRED (Feb. 3, 2019), https://www.wired.co.uk/article/deepmind-ai-chess [https://perma.cc/WF5Z-VEV7].[/footnote]). But when it comes to systems that replace the law, designers should be thinking harder about how best to combine the strengths of humans and machines, by understanding the human advantages of providing a sense of procedural fairness, explainability, and the deciding of hard cases. That suggests the deliberate preservation of mechanisms for resort to human adjudication (either public or private) as part of a long-term, sustainable system. Human courts, meanwhile, should embark on a greater effort to automate the handling of routine cases and routine procedural matters, like the filing of motions. The use of intelligent software for matters like sentencing and bail—decisions with enormous impact on people’s lives—seems exactly backward. The automation of routine procedure might help produce both a much faster legal system and also free up the scarce resource of highly trained human judgment to adjudicate the hard cases, or to determine which are the hard cases. Anyone who has worked in the courts knows that the judiciary’s mental resources are squandered on thousands of routine matters; there is promise in a system that leaves judges to do what they do best: exercising judgment in the individual case, and humanizing and improving the written rules. This also implies that judges should seek to cultivate their comparative advantage, the exercise of human judgment, instead of trying to mimic machines that follow rules.[footnote]Cf. Kathryn Judge, Judges and Judgment: In Praise of Instigators, 86 U. Chi. L. Rev. (forthcoming 2019) (manuscript at 1–2), https://ssrn.com/abstract=3333218 (on file with the Columbia Law Review ) (describing Judge Richard Posner’s rejection of such machine-like jurisprudence).[/footnote] Part I frames software decisionmaking among its competitors as a system of social ordering. Part II introduces the case study of Twitter and Facebook’s handling of hate speech, focusing on the evolution of online norms, and the subsequent adoption of a hybrid human–software system to control speech. Part III assesses, from a normative perspective, the hybrid systems described in Part II.