We can generalize this idea across our entire model space. Suppose we enumerated all the possible numerical pairs <error, unfairness> achieved by the models we are considering (e.g., SAT cutoffs)....
The Pareto frontier of accuracy and fairness is necessarily silent about which point we should choose along the frontier, because that is a matter of judgment about the relative importance of accuracy and fairness. The Pareto frontier makes our problem as quantitative as possible, but no more so.
The good news is that generally speaking, whenever we have practical algorithms for “standard,” accuracy-only machine learning for a class of models, we also have practical algorithms for tracing out this Pareto frontier. These algorithms will be a little bit more complicated—after all, they must identify a collection of models rather than just a single one—but not by much...
While the idea of considering cold, quantitative trade-offs between accuracy and fairness might make you uncomfortable, the point is that there is simply no escaping the Pareto frontier. Machine learning engineers and policymakers alike can be ignorant of it or refuse to look at it. But once we pick a decision-making model (which might in fact be a human decision-maker), there are only two possibilities. Either that model is not on the Pareto frontier, in which case it’s a “bad” model (since it could be improved in at least one measure without harm in the other), or it is on the frontier, in which case it implicitly commits to a numerical weighting of the relative importance of error and unfairness. Thinking about fairness in less quantitative ways does nothing to change these realities—it only obscures them.
Making the trade-off between accuracy and fairness quantitative does not remove the importance of human judgment, policy, and ethics—it simply focuses them where they are most crucial and useful, which is in deciding exactly which model on the Pareto frontier is best (in addition to choosing the notion of fairness in the first place, and which group or groups merit protection under it, both of which we discuss shortly). Such decisions should be informed by many factors that cannot be made quantitative, including what the societal goal of protecting a particular group is and what is at stake. Most of us would agree that while both racial bias in the ads users are shown online and racial bias in lending decisions are undesirable, the potential harms to individuals in the latter far exceed those in the former. So in choosing a point on the Pareto frontier for a lending algorithm, we might prefer to err strongly on the side of fairness—for example, insisting that the false rejection rate across different racial groups be very nearly equal, even at the cost of reducing bank profits. We’ll make more mistakes this way—both false rejections of creditworthy applicants and loans granted to parties who will default—but those mistakes will not be disproportionately concentrated in any one racial group. This is the bargain we must accept for strong fairness guarantees. Michael Kearns & Aaron Roth (2019) The Ethical Algorithm: The Science of socially Aware Algorithmic Design, Oxford University Press, 80-84 [the linked page should have their diagram]
A few weeks ago, after I started blogging (here and here) about the ethics of algorithms,* I received a Facebook advertisement-notification from Amazon that Michael Kearns and Aaron Roth (the authors of The Ethical Algorithm) had become Amazon scholars. "What an interesting way," I thought, "to advertise books to me." I clicked on the link and read the corporately sponsored interview with them. Perhaps because my expectations were low , but I was surprised how thoughtful -- "the main thesis of our book, which is that in any particular problem you have to start by thinking carefully about what you want in terms of fairness or privacy or some other social desideratum, and then how you relatively value things like that compared to other things you might care about, such as accuracy" -- they sounded and how relevant to my own project. It was clear theirs was a project about thinking about trade-offs (a favored phrase of theirs) about social values in algorithm design. More important to me, since I had just argued that thinking about the ethics of algorithms seems to replicate the very social problems familiar from thinking about ethics in economics, I wanted to see if my hunch was correct. Two clicks later I ordered the book, and a day later it was delivered.
A few years ago (well back in 2012), I used to joke that as even theoretical economists become more applied, becoming (thanks to increasingly cheap computer power) experts at massaging results out of giant "administrative data-sets,"+ economics ran the risk of being displaced by statisticians and, especially, computer scientists. While then I was largely ignorant of deep learning, it was clear to me that fruitful searches for robust and surprising correlations in data didn't require (as Justin Wolfers then suggested) the restrictive assumptions of economic theory. (Think about it: econometrics and economic theory artificially restrict the search space that can be more fully explored with deep learning machinery.) And indeed deep learning can dispense with economic theory.**
But in reading their interview and their splendid The Ethical Algorithm I realized that back in 2012, I had missed a crucial issue: that algorithmic design is conceived in terms of optimization problems under constraints. Since Lionel Robbins (back in the 1930s) this just is the definition of economics (and it enabled a split between ethics and economics). And once in algorithmic design you are interested in more than predictive accuracy, and so have to deal "with multiple competing criteria," a whole set of mathematically precise diagnostic tools familiar from economics can be imported into algorithmic design (as the passage quoted above suggests). So, I now realize that increasingly computer science and economics will merge (as presumably is happening already in finance-notice the passing mention of "portfolio management" in the block quote above).
The previous is sufficient for a digression. But I also want to call attention to how moral issues are conceived, and not, in the (ahh) paradigm articulated by Kearns and Roth. And, in particular, I want to call attention to four levels, or sites, where they enter in. So, formally moral issues enter into two sites of algorithmic design: (i) first a choice in setting a goal (such as privacy or fairness) or success criterion for the algorithm. In general, Kearns and Roth view this as a constraint on accuracy that generates a trade-off among accuracy and other possible criteria. Kearns & Roth are very good at explaining that the way one makes precise these moral goals need not be univocal or can be very context sensitive. (So, there are different ways to think about fairness or privacy in algorithmic settings.)
In turn, these trade-offs can be modeled in terms of pareto frontiers. And this generates the second site: (ii) at the level of a decision about ''which model on the Pareto frontier" to use in practice. Conceptually that's a distinct choice from how to encode 'fairness' into an algorithm, but obviously one can imagine that in practice, in the spirit of experimentation, there are going to be interactions between (i) and (ii) within a general goal-oriented design process. (Recall that an algorithm can be understood, even identified, in terms of the functions and goals it serves--something I have promised to return to.)
Some time soon, I hope to write some more critical posts about their hope of turning algorithm design into a kind of dream of turning ethics into an optimization problem. (That way of putting it is indebted to Kathleen Creel, a young scholar who has written a gem of a paper on opacity in computational systems.) And one can see how the combination (i) and (ii) lend themselves, despite the situationism in (i), to a form of moral reasoning familiar from (and analogous, if not identical to) utilitarianism. Of course, one can let other values enter into the decision of (ii).
Now, in their argument (ii) tends to happen in algorithm design/development process within, say, a company. But the choice one makes about which model on the Pareto frontier to inhabit generates (ahh) consequences to wider society. Some of these wider consequences -- laws, regulations, social norms, etc. -- are already internalized as constraints, but some are simply outside the company's mission/attention and not anticipated by the legislature.
The point in the previous paragraph fits their general argument. Because the "core concern" of their book is that "optimizing" on some explicit social goals, predictably and foreseeable, has (even in the relatively short term) unintended and perhaps ex ante unknowable side effects. (188) There is, thus, not just (as Kearns & Roth note) a (iii) a prior moral and political decision to be made whether or the extent to which algorithms are permitted or primarily responsible for decisions in a domain or (as they often describe it) norm enforcement (177). In the book they offer as an example, of the former "automated warfare," (175; 178). But there is also (IV) a question how society should think about, and perhaps compensate for or mitigate, these entirely unintended and perhaps unknowable side effects that are a foreseeable outcome pattern from embracing algorithmic practices. This matters especially if such outcome patterns (recall) create asymmetric harm patterns to vulnerable populations.
And, not unlike the economists, the way Kearns & Roth have set up their conceptual scheme (IV) turns out to be delegated to policy and so is not part of the ethical algorithm at all (or the responsibility of the firms that profit from them). To put this in their terms, we (now) know that a system with the very best portfolio management can create unexpected, general externalities. Something similar is now foreseeable in their field. (I am not sure what the equivalent term of true Knightian uncertainty in AI is, but they need it!) So given (IV), it would be good to prevent regressive forms of socialization of risk while privatizing profit.
*Disclosure: together with Federica Russo and Jean Wagemans, I work on a project (see here)" Towards an Epistemological and Ethical ‘Explainable AI’, funded by Human(e) AI.
**Actually that's false about the state of play today. Many of Kearns and Roth's chapters are about the application of ideas from game theory and so-called mechanism design scaled up in very large dimensions in deep learning.
+The term was then Raj Chetty's whose work I have been blogging regularly about since.
Comments