Equality of Opportunity in Supervised Learning

Equality of Opportunity in Supervised Learning Moritz Hardt Eric Price Nathan Srebro

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Equality of Opportunity in Supervised Learning 2016 NeurIPS / arXiv 1610.02413 hardt2016-equality-opportunity hardt2016-equality-opportunity.txt, the project's staged copy 3c77c2aa41ec542e7e1e89a65fbd7b6eada105cd arxiv-open law

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them method Quotation withheld because this record is not cleared for redistribution. discrimination against protected attributes in supervised learning Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. mechanisms A NAMED APPROACH IS SHOWN NOT TO DO WHAT ITS NAME PROMISES: `fairness through unawareness` - simply ignoring the protected attributes - is called INEFFECTIVE, because redundant encodings let the protected attribute be predicted from other features anyway. Removing the variable does not remove the information Quotation withheld because this record is not cleared for redistribution. mechanisms AN ESTABLISHED CRITERION IS SATISFIED BY EXACTLY THE BEHAVIOUR IT WAS MEANT TO FORBID: demographic parity PERMITS accepting qualified applicants in one group and UNQUALIFIED individuals in the other, so long as the acceptance percentages match - and the authors add that this can arise NATURALLY when one group has little or no training data. The criterion is not merely weak; it is passed by the discrimination it names Quotation withheld because this record is not cleared for redistribution. mechanisms A CRITERION IS SHOWN TO FORBID THE GROUND TRUTH: demographic parity would not allow the ideal predictor that simply equals the true outcome - which, the authors say, CAN HARDLY BE CONSIDERED DISCRIMINATORY as it represents the actual outcome. The case arises whenever the target is correlated with the protected attribute about this quote: The span contains '(cid:98)Y', which is Y-hat - the circumflex accent failed to map and the extraction emitted a glyph code. 367 of these are in this file and almost all are this same hat. The quote is an exact substring and anchors correctly; the mathematics is unreadable in it. Damage shape 6. Unavoidable here: the sentence's subject IS the predictor. Quotation withheld because this record is not cleared for redistribution. mechanisms THE NEW CRITERION IS ENGINEERED SO THE PERFECT PREDICTOR ALWAYS PASSES, AND SO THAT IT GETS EASIER AS ACCURACY RISES: the authors state that unlike demographic parity theirs always allows the perfectly accurate solution, and that the criterion is easier to achieve the more accurate the predictor is - explicitly ALIGNING fairness with the existing objective of supervised learning rather than trading against it about this quote: The span contains '(cid:98)Y', which is Y-hat - the circumflex accent failed to map and the extraction emitted a glyph code. 367 of these are in this file and almost all are this same hat. The quote is an exact substring and anchors correctly; the mathematics is unreadable in it. Damage shape 6. Unavoidable here: the sentence's subject IS the predictor. Quotation withheld because this record is not cleared for redistribution. mechanisms THE CRITERION PERMITS ONE ROUTE AND PROHIBITS ANOTHER BETWEEN THE SAME TWO VARIABLES: the prediction may depend on the protected attribute but ONLY THROUGH the target, so the definition ENCOURAGES features that predict the target directly and PROHIBITS abusing the protected attribute as a proxy for it. What the definition constrains is the PATH the dependence takes about this quote: The span contains '(cid:98)Y', which is Y-hat - the circumflex accent failed to map and the extraction emitted a glyph code. 367 of these are in this file and almost all are this same hat. The quote is an exact substring and anchors correctly; the mathematics is unreadable in it. Damage shape 6. Unavoidable here: the sentence's subject IS the predictor. Quotation withheld because this record is not cleared for redistribution. mechanisms A PROPERTY IS DEFINED FOR PROPERTIES, NOT FOR PREDICTORS: `oblivious` is given a formal definition one level up - a PROPERTY of a predictor or score is oblivious if it depends only on the joint distribution of outcome, protected attribute and prediction. The definition's subject is other definitions about this quote: The span contains '(cid:98)Y', which is Y-hat - the circumflex accent failed to map and the extraction emitted a glyph code. 367 of these are in this file and almost all are this same hat. The quote is an exact substring and anchors correctly; the mathematics is unreadable in it. Damage shape 6. Unavoidable here: the sentence's subject IS the predictor. Quotation withheld because this record is not cleared for redistribution. mechanisms THE CRITERION IS RESTRICTED TO WHAT IS OBSERVABLE, AND THE RESTRICTION IS JUSTIFIED BY OBSERVABILITY RATHER THAN BY CORRECTNESS: in many cases only oblivious analysis is POSSIBLE, because the functional form of the score and the training data are not public - and going beyond it would require SUBJECTIVE INTERPRETATION or, as the paper prints it, `casual` assumptions about specific features (causal is meant), WHICH THE AUTHORS AIM TO AVOID. The definition is cut to fit what can be seen Quotation withheld because this record is not cleared for redistribution. mechanisms A WEAKER CRITERION IS OBTAINED BY RESTRICTING THE STRONGER ONE TO ONE OUTCOME, AND WHICH OUTCOME IS AN INTERPRETIVE CHOICE: the authors say they OFTEN THINK OF one value as the ADVANTAGED outcome, then relax the requirement to hold only within that group, WITHOUT SPECIFYING ANY REQUIREMENT for those who will ultimately default - and name the result `equal opportunity`. The asymmetry that makes the weaker notion weaker is supplied by the reading of the outcome, not by the mathematics Quotation withheld because this record is not cleared for redistribution. mechanisms THE CRITERION IS PRESENTED IN TERMS OF WHERE IT PUTS THE COST: it is described as SHIFTING THE BURDEN OF UNCERTAINTY from the protected class to the decision maker, and thereby INCENTIVISING the collection of better features - ones depending more directly on the target than on the protected attribute - and of data allowing better prediction for all groups Quotation withheld because this record is not cleared for redistribution. mechanisms THE PAPER PROVES THE LIMIT OF ITS OWN ENTIRE FAMILY OF DEFINITIONS: a NON-IDENTIFIABILITY result shows that different dependency structures, WITH POSSIBLY DIFFERENT INTUITIVE NOTIONS OF FAIRNESS, cannot be separated by ANY oblivious notion or test - not merely by theirs. The contribution is a proof about what the class of measures cannot see Quotation withheld because this record is not cleared for redistribution. mechanisms TWO SITUATIONS THAT DIFFER IN EXACTLY THE WAY THE CONCEPT IS SUPPOSED TO TRACK ARE SHOWN TO BE INDISTINGUISHABLE: the authors construct two dependency structures which can have FUNDAMENTALLY DIFFERENT INTERPRETATIONS FROM THE POINT OF VIEW OF FAIRNESS and yet be indistinguishable from their joint distribution - so no oblivious test can resolve which applies Quotation withheld because this record is not cleared for redistribution. mechanisms ACCESS TO THE FEATURES DOES NOT RESCUE IT - THE PAPER CLOSES ITS OWN LOOP: without ATTACHING MEANING to the features or making CAUSAL ASSUMPTIONS about them, no further insight into the two scores is gained. Subjective interpretation was excluded in order to make the criterion objective, and here the authors state that without it nothing more can be learned about this quote: RE-SLICED. My first cut ran straight through a WORD-ORDER SCRAMBLE (damage shape 5): the staged text reads "since the distributions , (cid:101)R) agree ... The over (Y , A, R features are correlated", with two clauses interleaved and a superscript hoisted. It would have been perfectly verbatim and unreadable. The span now keeps only the closing sentence, which carries the claim and is clean. Quotation withheld because this record is not cleared for redistribution. mechanisms A PROPERTY IS NAMED, GIVEN A USEFUL MEANING, AND THEN SHOWN NOT TO MEAN WHAT IT SOUNDS LIKE: having MATCHING conditional ROC curves does correspond to being deterministically correctable to non-discrimination - but the authors state plainly that it DOES NOT AT ALL mean the score is itself non-discriminatory, since it can be biased according to the protected attribute and need correcting Quotation withheld because this record is not cleared for redistribution. mechanisms A PROPERTY OFFERED AS A FAIRNESS MEASURE IS EQUIVALENT TO THE SCORE USING EVERYTHING THE PROTECTED ATTRIBUTE OFFERS: matching conditional frequencies means the score includes ALL POSSIBLE INFORMATION the protected attribute can provide about the target, so the score is IN A SENSE OPTIMALLY DEPENDENT on that attribute Quotation withheld because this record is not cleared for redistribution. the criterion is oblivious - it uses only joint statistics, not the interpretation of individual features what this limits: everything the criterion can detect: Quotation withheld Quotation withheld because this record is not cleared for redistribution. Satisfying the proposed oblivious criterion is not a proof of fairness (nor is violating it a proof of unfairness). what this limits: any use of the criterion as a fairness certificate: SATISFYING OUR NOTION (OR ANY OTHER OBLIVIOUS MEASURE) SHOULD NOT BE CONSIDERED A CONCLUSIVE PROOF OF FAIRNESS Quotation withheld because this record is not cleared for redistribution. The criterion requires reliable observed outcome labels, which are not always available and may themselves be biased. what this limits: every application of the method: HAVING ACCESS TO RELIABLE LABELED DATA IS NOT ALWAYS POSSIBLE, and the labels may themselves carry the bias being measured Quotation withheld because this record is not cleared for redistribution. The near-optimality guarantee is relative to other constrained predictors only; the derived predictor need not be good in absolute terms. what this limits: the optimality claim: THIS DOES NOT MEAN THAT THE PREDICTOR IS NECESSARILY GOOD IN AN ABSOLUTE SENSE - near-optimality is relative to other constrained predictors Quotation withheld because this record is not cleared for redistribution. No test of this kind can tell two situations apart, and the authors prove it about their own class of test. what this limits: a verdict of non-discrimination obtained from this criterion: the authors show no oblivious test can resolve which of two dependency structures applies, so two scenarios with different intuitive fairness are indistinguishable to it Quotation withheld because this record is not cleared for redistribution. Access to the features does not rescue the test, without assumptions the authors decline to make. what this limits: the suggestion that inspecting the model's features would settle what this test cannot: the authors state that without attaching meaning to the features or making causal assumptions about them, no further insight is gained Quotation withheld because this record is not cleared for redistribution. Meeting the criterion can require deciding individual cases partly at random. what this limits: the deployment of this criterion on individual decisions: achieving it cannot be done with a single threshold per group and requires randomization Quotation withheld because this record is not cleared for redistribution. Who pays for the fairness is a property of the design, and the authors name the payer. what this limits: the incidence of the utility cost reported here: the authors state the cost is borne by the company using the classifier, which can invest in better classification, rather than by the classified group, which cannot Quotation withheld because this record is not cleared for redistribution. equalized odds equalized odds about this quote: DAMAGED GLYPHS IN THIS QUOTE: (cid:98). These are raw PDF font glyph indices that leaked out in place of the characters they stand for - usually a mathematical operator. The quote is verbatim to the staged text and its anchor is sound; the STAGED TEXT is what is damaged. NOT repaired: the cid-to-character mapping is defined by the font embedded in that particular PDF, so guessing it would produce a clean and unverifiable quote. Repair needs the source PDF and is a re-staging job. Quotation withheld because this record is not cleared for redistribution. equal opportunity equal opportunity Quotation withheld because this record is not cleared for redistribution. demographic parity demographic parity Quotation withheld because this record is not cleared for redistribution. oblivious oblivious Quotation withheld because this record is not cleared for redistribution. fairness through unawareness fairness through unawareness Quotation withheld because this record is not cleared for redistribution. equal opportunity vs equalized odds - TWO CRITERIA, ONE STRICTLY WEAKER, AND THE PAPER PRICES THE GAP: equal opportunity relaxes equalized odds to the advantaged outcome group only, and in the case study equal opportunity keeps 92.8% of maximal profit against equalized odds' 80.2%, so a claim citing 'Hardt et al.' without saying which criterion is a claim about neither Quotation withheld because this record is not cleared for redistribution. oblivious - IN A FORMAL SENSE THE PAPER DEFINES: a property is oblivious if it depends only on the joint distribution of target, protected attribute and prediction, which is what makes the criterion checkable from published statistics and also what makes it blind Quotation withheld because this record is not cleared for redistribution. demographic parity - THE CRITERION THIS ONE IS DEFINED AGAINST: it requires the decision to be independent of the protected attribute, which the authors reject because it forbids the perfectly accurate predictor and costs the most utility of the four thresholds they compare Quotation withheld because this record is not cleared for redistribution. contradicts → fairness through unawareness (ignoring protected attributes) Quotation withheld because this record is not cleared for redistribution. builds-on → Dwork et al. [DHP+12], "Fairness through Awareness" Quotation withheld because this record is not cleared for redistribution. builds-on → Kleinberg, Mullainathan and Raghavan [KMR16] on calibration within groups Quotation withheld because this record is not cleared for redistribution. contradicts → demographic parity as a non-discrimination criterion differs in: THE AXIS IS THE FAIRNESS CRITERION. 'Quotation withheld' Two criteria, and the disagreement is over which one MEASURES discrimination - not over any fact about a dataset. Quotation withheld because this record is not cleared for redistribution. 2 — ways demographic parity is flawed Quotation withheld because this record is not cleared for redistribution. May 2014 — Obama Big Data report warning on discrimination Quotation withheld because this record is not cleared for redistribution. demographic parity is seriously flawed on two counts Quotation withheld because this record is not cleared for redistribution. fairness through unawareness fails because of redundant encodings Quotation withheld because this record is not cleared for redistribution. their criterion always permits the perfectly accurate predictor Quotation withheld because this record is not cleared for redistribution. the criterion aligns fairness with predictive accuracy Quotation withheld because this record is not cleared for redistribution. protected attribute A protected attribute A Quotation withheld because this record is not cleared for redistribution. target Y target Y Quotation withheld because this record is not cleared for redistribution. predictor predictor Quotation withheld because this record is not cleared for redistribution. features X features X Quotation withheld because this record is not cleared for redistribution. demographic parity demographic parity Quotation withheld because this record is not cleared for redistribution. fairness through unawareness fairness through unawareness Quotation withheld because this record is not cleared for redistribution. oblivious notion oblivious notion Quotation withheld because this record is not cleared for redistribution. equal opportunity by design equal opportunity by design Quotation withheld because this record is not cleared for redistribution. Bayesian Bayesian Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://arxiv.org/abs/1610.02413.