Algorithmic Fairness: Choices, Assumptions, and Definitions

Algorithmic Fairness: Choices, Assumptions, and Definitions Shira Mitchell Eric Potash Solon Barocas Alexander D’Amour Kristian Lum

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Algorithmic Fairness: Choices, Assumptions, and Definitions 2020 Annual Review of Statistics and Its Application 10.1146/annurev-statistics-042720-125902 mitchell2020-algorithmic-fairness-choices-assum mitchell2020-algorithmic-fairness-choices-assum.txt, the project's staged copy d29dcd008a6c1fada70884d36c88db4216f90e38 cc-by social-sciences

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them THE COMPETING QUANTITATIVE DEFINITIONS OF 'FAIRNESS' THEMSELVES - catalogued, put into one notation, and then shown to be mutually unsatisfiable Quotation withheld because this record is not cleared for redistribution. what the definitions of algorithmic fairness are, what each one assumes, and whether they can be held together Quotation withheld because this record is not cleared for redistribution. a notationally consistent catalogue of fairness definitions, the choices and assumptions each rests on, and the impossibility results that separate them Quotation withheld because this record is not cleared for redistribution. critique Quotation withheld because this record is not cleared for redistribution. a young field in which one word carries several formal meanings that cannot all be satisfied at once, so that two competing analyses of the same tool can both be arithmetically correct Quotation withheld because this record is not cleared for redistribution. review review Quotation withheld because this record is not cleared for redistribution. THE INCOMPATIBILITY IS NOT A PROPERTY OF ALGORITHMS. `the trade-off between different notions of fairness would apply even if a human were the one making the prediction`, and `Rejecting model-driven or automated decision-making is not a way to avoid these problems`. It is a property of predicting at all, under inequality Quotation withheld because this record is not cleared for redistribution. and the conditions under which the definitions COULD agree are named and are both unreachable: equal base rates or perfect prediction. The paper calls them `trivial, degenerate, or even utopian (representing two very different utopias)` Quotation withheld because this record is not cleared for redistribution. the paper declines the job most readers want done: `The goal of this article is not to advance axiomatic definitions of fairness but to summarize the definitions and results ... formalized to date`. It catalogues a disagreement rather than settling it, and says so in the introduction Quotation withheld because this record is not cleared for redistribution. and it names the gap its own subject cannot close: `gaps between mathematically convenient formalism and the larger social goals that many of these concepts were introduced to address`. THAT IS THE OPERATIONALISATION GAP, NAMED AS SUCH, by a paper whose subject is operationalisations Quotation withheld because this record is not cleared for redistribution. the object being predicted is already an abstraction of the thing anyone cares about: policy goals are `sometimes clumsily, abstracted and formulated to fit into a prediction task`, and the choices that do it sit outside the model Quotation withheld because this record is not cleared for redistribution. and the decision itself is flattened before any of this begins: a judge's options are many, `in this literature, this decision is often reduced to A BINARY DECISION - RELEASE OR DETAIN`. Every fairness metric in the catalogue is defined over that binary Quotation withheld because this record is not cleared for redistribution. the authors record the standing criticism of their own field without dismissing it: quantitative notions of fairness `can funnel our thinking into narrow silos where we aim to make adjustments to a decision-making process, rather than to address the structural conditions that sustain inequality in society` Quotation withheld because this record is not cleared for redistribution. whether the incompatibility is an artefact of using algorithms at all eliminates: the algorithmic explanation - the trade-off 'would apply even if a human were the one making the prediction', so refusing automation does not avoid it · outcome: held Quotation withheld because this record is not cleared for redistribution. under what conditions the two COMPAS criteria COULD both hold, worked out formally eliminates: any hope of reconciliation in practice - it requires equal recidivism rates across racial groups, or a group that never experiences an outcome, or perfect prediction · outcome: held Quotation withheld because this record is not cleared for redistribution. whether formal fairness work is worth doing at all, against its own critics eliminates: nothing on either side, and the paper says so: the criticisms are 'rightly pointed out', and formalism 'should not be dismissed as necessarily preserving the status quo'. Both are asserted; neither is tested · outcome: partial Quotation withheld because this record is not cleared for redistribution. algorithmic fairness algorithmic fairness Quotation withheld because this record is not cleared for redistribution. equal false positive rates equal false positive rates Quotation withheld because this record is not cleared for redistribution. equal positive predictive values equal positive predictive values Quotation withheld because this record is not cleared for redistribution. calibration within groups calibration within groups Quotation withheld because this record is not cleared for redistribution. sufficiency sufficiency Quotation withheld because this record is not cleared for redistribution. separation separation Quotation withheld because this record is not cleared for redistribution. bias bias Quotation withheld because this record is not cleared for redistribution. quantitative notions of fairness quantitative notions of fairness Quotation withheld because this record is not cleared for redistribution. structural conditions structural conditions Quotation withheld because this record is not cleared for redistribution. equality of true positive rates equality of true positive rates Quotation withheld because this record is not cleared for redistribution. equality of false negative rates equality of false negative rates Quotation withheld because this record is not cleared for redistribution. equal opportunity equal opportunity Quotation withheld because this record is not cleared for redistribution. error rate balance error rate balance Quotation withheld because this record is not cleared for redistribution. equalized odds equalized odds Quotation withheld because this record is not cleared for redistribution. equality of negative predictive value equality of negative predictive value Quotation withheld because this record is not cleared for redistribution. equality of false omission rate equality of false omission rate Quotation withheld because this record is not cleared for redistribution. equality of positive predictive value equality of positive predictive value Quotation withheld because this record is not cleared for redistribution. equality of false discovery rate equality of false discovery rate Quotation withheld because this record is not cleared for redistribution. predictive parity predictive parity Quotation withheld because this record is not cleared for redistribution. avoiding disparate mistreatment avoiding disparate mistreatment Quotation withheld because this record is not cleared for redistribution. treatment equality treatment equality Quotation withheld because this record is not cleared for redistribution. demographic parity demographic parity Quotation withheld because this record is not cleared for redistribution. statistical parity statistical parity Quotation withheld because this record is not cleared for redistribution. group fairness group fairness Quotation withheld because this record is not cleared for redistribution. Conditional demographic parity Conditional demographic parity Quotation withheld because this record is not cleared for redistribution. fairness through unawareness fairness through unawareness Quotation withheld because this record is not cleared for redistribution. anticlassification anticlassification Quotation withheld because this record is not cleared for redistribution. treatment parity treatment parity Quotation withheld because this record is not cleared for redistribution. metric fairness metric fairness Quotation withheld because this record is not cleared for redistribution. individual fairness individual fairness Quotation withheld because this record is not cleared for redistribution. m-Lipschitz property m-Lipschitz property Quotation withheld because this record is not cleared for redistribution. perfect metric fairness perfect metric fairness Quotation withheld because this record is not cleared for redistribution. individual counterfactual fairness individual counterfactual fairness Quotation withheld because this record is not cleared for redistribution. conditional counterfactual fairness conditional counterfactual fairness Quotation withheld because this record is not cleared for redistribution. counterfactual parity counterfactual parity Quotation withheld because this record is not cleared for redistribution. Calibration within groups → sufficiency The paper says calibration within groups implies sufficiency, and describes a converse after applying a function to a sufficient score. Logical implication between criteria is not one of the eight edge types. Quotation withheld because this record is not cleared for redistribution. metric fairness → unawareness When the metric uses only insensitive variables, the paper says metric fairness implies unawareness. Logical implication is not one of the eight edge types. Quotation withheld because this record is not cleared for redistribution. individual counterfactual fairness → conditional counterfactual fairness The paper orders individual counterfactual fairness above conditional counterfactual fairness in strength. A stronger-than relation is not one of the eight edge types. Quotation withheld because this record is not cleared for redistribution. conditional counterfactual fairness → counterfactual parity The paper orders conditional counterfactual fairness above counterfactual parity in strength. A stronger-than relation is not one of the eight edge types. Quotation withheld because this record is not cleared for redistribution. fairness - NOT ONE DEFINITION BUT SEVERAL FORMAL ONES, AND THE PAPER'S RESULT IS THAT THEY CANNOT ALL HOLD: `Although each flavor of fairness definition ... formalizes an intuitive notion of fairness, these definitions are not mathematically or morally compatible in general` Quotation withheld because this record is not cleared for redistribution. `bias` - THE PAPER CALLS IT OVERLOADED AND GIVES THE DRIFT: `a now overloaded word that, in popular media, has come to mean that the model's performance (however defined) unjustifiably differs along social axes such as race, gender, and class`. The parenthesis is the point Quotation withheld because this record is not cleared for redistribution. THE COMPAS DISPUTE - two analyses of one tool, both correct. ProPublica: it `does not satisfy equal false positive rates by race`. Northpointe: it `satisfies equal positive predictive values` and calibration within groups. NEITHER SIDE MISCALCULATED. They used different definitions of the same word Quotation withheld because this record is not cleared for redistribution. `no harmony` - the paper's verdict on the attempts to reconcile them: `Much of the subsequent conversation consisted of either trying to harmonize these definitions ... or asserting that one or the other is correct. As it turns out, there can be no harmony among definitions in a world where inequality and imperfect prediction are the reality` Quotation withheld because this record is not cleared for redistribution. the formal conditions for agreement, stated exactly: SUFFICIENCY implies equality of positive predictive values, SEPARATION implies equality of false positive rates, and both hold together only if recidivism rates and score distributions are the same across racial groups, or some group never experiences an outcome at all Quotation withheld because this record is not cleared for redistribution. algorithmic fairness - a name for a research field, and the paper notes the questions are older than it: `it resembles older work ... since the 1960s, psychometricians have studied the fairness of educational tests based on their ability to predict performance` Quotation withheld because this record is not cleared for redistribution. equality of true positive rates - equality across groups in the true-positive rate Quotation withheld because this record is not cleared for redistribution. equality of false negative rates - equality across groups in the false-negative rate Quotation withheld because this record is not cleared for redistribution. equal opportunity - the paper's name for equality of true-positive and false-negative rates considered alone Quotation withheld because this record is not cleared for redistribution. error rate balance - Chouldechova's name for the combined outcome-conditioned fairness notions Quotation withheld because this record is not cleared for redistribution. equalized odds - Hardt and colleagues' name for the combined outcome-conditioned fairness notions Quotation withheld because this record is not cleared for redistribution. equality of negative predictive value - equality across groups in negative predictive value Quotation withheld because this record is not cleared for redistribution. equality of false omission rate - equality across groups in false omission rate Quotation withheld because this record is not cleared for redistribution. equality of positive predictive value - equality across groups in positive predictive value Quotation withheld because this record is not cleared for redistribution. equality of false discovery rate - equality across groups in false discovery rate Quotation withheld because this record is not cleared for redistribution. predictive parity - another name for equality of positive predictive value and false discovery rate Quotation withheld because this record is not cleared for redistribution. avoiding disparate mistreatment - Zafar and colleagues' collective name for the four predictive-measure pairs plus equal accuracy Quotation withheld because this record is not cleared for redistribution. treatment equality - a fairness definition based on the ratio of false negatives to false positives Quotation withheld because this record is not cleared for redistribution. demographic parity - equal decision rates across groups irrespective of outcome Quotation withheld because this record is not cleared for redistribution. statistical parity - an alternative name for demographic parity Quotation withheld because this record is not cleared for redistribution. group fairness - an alternative name for demographic parity Quotation withheld because this record is not cleared for redistribution. Conditional demographic parity - parity within strata, expressed as conditional independence of decision and sensitive group Quotation withheld because this record is not cleared for redistribution. fairness through unawareness - conditional demographic parity when conditioning on insensitive variables Quotation withheld because this record is not cleared for redistribution. anticlassification - conditional demographic parity when conditioning on insensitive variables Quotation withheld because this record is not cleared for redistribution. treatment parity - conditional demographic parity when conditioning on insensitive variables Quotation withheld because this record is not cleared for redistribution. metric fairness - requiring closeness under a similarity metric to imply closeness in decisions Quotation withheld because this record is not cleared for redistribution. individual fairness - an alternative name for metric fairness Quotation withheld because this record is not cleared for redistribution. m-Lipschitz property - an alternative mathematical name for metric fairness Quotation withheld because this record is not cleared for redistribution. perfect metric fairness - Rothblum and Yona's name for metric fairness Quotation withheld because this record is not cleared for redistribution. individual counterfactual fairness - the strongest listed counterfactual criterion, requiring decision equality for every individual Quotation withheld because this record is not cleared for redistribution. conditional counterfactual fairness - the intermediate counterfactual criterion, requiring conditional mean-decision equality Quotation withheld because this record is not cleared for redistribution. counterfactual parity - the weakest listed counterfactual criterion, requiring population mean-decision equality Quotation withheld because this record is not cleared for redistribution. contradicts → both positions in the COMPAS dispute, insofar as either claimed to be THE fairness answer - 'there can be no harmony among definitions in a world where inequality and imperfect prediction are the reality' Quotation withheld because this record is not cleared for redistribution. extends → the impossibility results of Kleinberg et al. and Chouldechova, restated here in one notation and set against a public case Quotation withheld because this record is not cleared for redistribution. supports → the psychometric literature on test fairness running back to the 1960s, named as the older form of the same question Quotation withheld because this record is not cleared for redistribution. equal false positive rates —distinct-from→ equal positive predictive values equal false positive rates equal positive predictive values Quotation withheld because this record is not cleared for redistribution. quantitative notions of fairness —contrasted-with→ structural conditions quantitative notions of fairness structural conditions Quotation withheld because this record is not cleared for redistribution. equality of true positive rates —renamed-across-fields→ equal opportunity equality of true positive rates equal opportunity the relation itself is separately evidenced · strength of the relation: 1 · note: The paper reports Hardt et al.'s unhedged name for this condition. Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says this condition 'has been called equal opportunity' by one author; a name, and renamed-across-fields is loose. Quotation withheld because this record is not cleared for redistribution. separation —renamed-across-fields→ error rate balance separation error rate balance the relation itself is separately evidenced · strength of the relation: 1 · note: The paper gives source-specific names for the same combined outcome-conditioned fairness notions. Quotation withheld because this record is not cleared for redistribution. separation —renamed-across-fields→ equalized odds separation equalized odds the relation itself is separately evidenced · strength of the relation: 1 · note: The paper gives source-specific names for the same combined outcome-conditioned fairness notions. Quotation withheld because this record is not cleared for redistribution. Conditional demographic parity —same-as→ separation Conditional demographic parity separation holds when: Data = Y · the relation itself is separately evidenced · strength of the relation: 1 · note: The equivalence is asserted without epistemic hedge but explicitly scoped to conditioning on outcome Y. Audit 2026-09-28: DOUBTFUL - read against its span and kept, but conditional demographic parity equals separation only when Data = Y; same-as is conditional. Quotation withheld because this record is not cleared for redistribution. Conditional demographic parity —same-as→ fairness through unawareness Conditional demographic parity fairness through unawareness holds when: Data = X (the insensitive variables) · the relation itself is separately evidenced · strength of the relation: 1 · note: The equivalence is asserted without epistemic hedge under the paper's explicit Data = X condition. Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'it' = conditional demographic parity (previous sentence); the equivalence holds only when Data = X. Quotation withheld because this record is not cleared for redistribution. Conditional demographic parity —same-as→ anticlassification Conditional demographic parity anticlassification holds when: Data = X (the insensitive variables) · the relation itself is separately evidenced · strength of the relation: 1 · note: The equivalence is asserted without epistemic hedge under the paper's explicit Data = X condition. Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'it' = conditional demographic parity (previous sentence); the equivalence holds only when Data = X. Quotation withheld because this record is not cleared for redistribution. Conditional demographic parity —same-as→ treatment parity Conditional demographic parity treatment parity holds when: Data = X (the insensitive variables) · the relation itself is separately evidenced · strength of the relation: 1 · note: The equivalence is asserted without epistemic hedge under the paper's explicit Data = X condition. Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'it' = conditional demographic parity (previous sentence); the equivalence holds only when Data = X. Quotation withheld because this record is not cleared for redistribution. metric fairness —same-as→ individual fairness metric fairness individual fairness the relation itself is separately evidenced · strength of the relation: 1 · note: The paper unhedgedly says metric fairness is also known as individual fairness. Quotation withheld because this record is not cleared for redistribution. metric fairness —same-as→ m-Lipschitz property metric fairness m-Lipschitz property the relation itself is separately evidenced · strength of the relation: 1 · note: The paper unhedgedly gives the m-Lipschitz property as another name for metric fairness. Quotation withheld because this record is not cleared for redistribution. metric fairness —same-as→ perfect metric fairness metric fairness perfect metric fairness the relation itself is separately evidenced · strength of the relation: 1 · note: The paper unhedgedly gives perfect metric fairness as another name for metric fairness. Quotation withheld because this record is not cleared for redistribution. THE FIELD'S OWN ABSTRACT DIAGNOSES IT: rapid growth has led to `wildly inconsistent motivations, terminology, and notation, presenting a serious challenge for cataloging and comparing definitions` THE FIELD'S OWN ABSTRACT DIAGNOSES IT: rapid growth has led to `wildly inconsistent motivations, terminology, and notation, presenting a serious challenge for cataloging and comparing definitions` Quotation withheld because this record is not cleared for redistribution. AND THE DEFINITIONS ARE NOT MERELY INCONSISTENT, THEY ARE INCOMPATIBLE: `these definitions are not mathematically or morally compatible in general` AND THE DEFINITIONS ARE NOT MERELY INCONSISTENT, THEY ARE INCOMPATIBLE: `these definitions are not mathematically or morally compatible in general` Quotation withheld because this record is not cleared for redistribution. ProPublica found COMPAS fails equal false positive rates by race; Northpointe showed it satisfies equal positive predictive values and calibration within groups ProPublica found COMPAS fails equal false positive rates by race; Northpointe showed it satisfies equal positive predictive values and calibration within groups Quotation withheld because this record is not cleared for redistribution. there can be no harmony among fairness definitions where inequality and imperfect prediction are real there can be no harmony among fairness definitions where inequality and imperfect prediction are real Quotation withheld because this record is not cleared for redistribution. the two criteria can hold together only if recidivism rates and score distributions are equal across racial groups, or some group never experiences an outcome the two criteria can hold together only if recidivism rates and score distributions are equal across racial groups, or some group never experiences an outcome Quotation withheld because this record is not cleared for redistribution. and those conditions are named as unreachable: `trivial, degenerate, or even utopian (representing two very different utopias)` and those conditions are named as unreachable: `trivial, degenerate, or even utopian (representing two very different utopias)` Quotation withheld because this record is not cleared for redistribution. THE TRADE-OFF IS NOT ABOUT ALGORITHMS: it `would apply even if a human were the one making the prediction`, and rejecting automation `is not a way to avoid these problems` THE TRADE-OFF IS NOT ABOUT ALGORITHMS: it `would apply even if a human were the one making the prediction`, and rejecting automation `is not a way to avoid these problems` Quotation withheld because this record is not cleared for redistribution. the paper names the operationalisation gap in its own introduction: `gaps between mathematically convenient formalism and the larger social goals that many of these concepts were introduced to address` the paper names the operationalisation gap in its own introduction: `gaps between mathematically convenient formalism and the larger social goals that many of these concepts were introduced to address` Quotation withheld because this record is not cleared for redistribution. policy goals are `sometimes clumsily, abstracted and formulated to fit into a prediction task` before any fairness metric applies policy goals are `sometimes clumsily, abstracted and formulated to fit into a prediction task` before any fairness metric applies Quotation withheld because this record is not cleared for redistribution. a judge's many options are often reduced in this literature to a binary decision, release or detain a judge's many options are often reduced in this literature to a binary decision, release or detain Quotation withheld because this record is not cleared for redistribution. the questions are older than the field's name: `since the 1960s, PSYCHOMETRICIANS have studied the fairness of educational tests based on their ability to predict performance` the questions are older than the field's name: `since the 1960s, PSYCHOMETRICIANS have studied the fairness of educational tests based on their ability to predict performance` Quotation withheld because this record is not cleared for redistribution. the paper declines to arbitrate: `the goal ... is not to advance axiomatic definitions of fairness but to summarize` what has been formalized the paper declines to arbitrate: `the goal ... is not to advance axiomatic definitions of fairness but to summarize` what has been formalized Quotation withheld because this record is not cleared for redistribution. declared A NOW OVERLOADED WORD which, in popular media, has come to mean that a model's performance - HOWEVER DEFINED - unjustifiably differs along social axes such as race, gender and class bias Quotation withheld because this record is not cleared for redistribution. false positive rates false positive rates Quotation withheld because this record is not cleared for redistribution. positive predictive values positive predictive values Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1146/annurev-statistics-042720-125902.