(cid:NN). The PDF this text was extracted from encodes 17 characters in a way that names no letter, so the extractor could not say what they were. The quotes keep them because a quote is reproduced exactly or it proves nothing — the marks are the source’s, not the tagging’s.standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.| relation to other work | claim_relations | how this paper stands to another: agrees, contradicts, extends, qualifies |
| claim type | claim_type | the shape of the claim: existence, mechanism, method, critique, prediction, null |
| concept | concepts | an idea the paper works with, quoted where the paper uses it |
| content tag | content_tags | a finding, definition, quantity, statistic or variable named in the paper |
| limitation | limitations | what the paper says its evidence cannot support |
| mechanism | mechanisms | how the reported effect is said to come about |
| method | method | how the paper produced its evidence |
| phenomenon | phenomenon | the effect or regularity under study |
| sense | senses | a distinct use of a term, quoted where the paper fixes it |
This record holds no relations between concepts or controls. A layer is empty when the tagger recorded nothing in it — which is not the same as the paper having nothing to record.
| facet | tag | field | conf. | whose words |
|---|---|---|---|---|
| claim_type | method Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | claim_type | the paper's words | |
| phenomenon | discrimination against protected attributes in supervised learning Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | phenomenon | the paper's words | |
| method | system-or-tool-building Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: system-or-tool-building | method | the paper's words | |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[0] | 0.95 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[1] | 0.95 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: The span contains '(cid:98)Y', which is Y-hat - the circumflex accent failed to map and the extraction emitted a glyph code. 367 of these are in this file and almost all are this same hat. The quote is an exact substring and anchors correctly; the mathematics is unreadable in it. Damage shape 6. Unavoidable here: the sentence's subject IS the predictor. | mechanisms[2] | 0.85 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: The span contains '(cid:98)Y', which is Y-hat - the circumflex accent failed to map and the extraction emitted a glyph code. 367 of these are in this file and almost all are this same hat. The quote is an exact substring and anchors correctly; the mathematics is unreadable in it. Damage shape 6. Unavoidable here: the sentence's subject IS the predictor. | mechanisms[3] | 0.95 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: The span contains '(cid:98)Y', which is Y-hat - the circumflex accent failed to map and the extraction emitted a glyph code. 367 of these are in this file and almost all are this same hat. The quote is an exact substring and anchors correctly; the mathematics is unreadable in it. Damage shape 6. Unavoidable here: the sentence's subject IS the predictor. | mechanisms[4] | 0.85 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: The span contains '(cid:98)Y', which is Y-hat - the circumflex accent failed to map and the extraction emitted a glyph code. 367 of these are in this file and almost all are this same hat. The quote is an exact substring and anchors correctly; the mathematics is unreadable in it. Damage shape 6. Unavoidable here: the sentence's subject IS the predictor. | mechanisms[5] | 0.85 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[6] | 0.85 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[7] | 0.95 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[8] | 0.85 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[9] | 0.95 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[10] | 0.95 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: RE-SLICED. My first cut ran straight through a WORD-ORDER SCRAMBLE (damage shape 5): the staged text reads "since the distributions , (cid:101)R) agree ... The over (Y , A, R features are correlated", with two clauses interleaved and a superscript hoisted. It would have been perfectly verbatim and unreadable. The span now keeps only the closing sentence, which carries the claim and is clean. | mechanisms[11] | 0.85 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[12] | 0.95 | the tagger's words |
| mechanisms | mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | mechanisms[13] | 0.85 | the tagger's words |
| limitations | the criterion is oblivious - it uses only joint statistics, not the interpretation of individual features Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: everything the criterion can detect: Quotation withheld | limitations[0] | the tagger's words | |
| limitations | Satisfying the proposed oblivious criterion is not a proof of fairness (nor is violating it a proof of unfairness). Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: any use of the criterion as a fairness certificate: SATISFYING OUR NOTION (OR ANY OTHER OBLIVIOUS MEASURE) SHOULD NOT BE CONSIDERED A CONCLUSIVE PROOF OF FAIRNESS | limitations[1] | the tagger's words | |
| limitations | The criterion requires reliable observed outcome labels, which are not always available and may themselves be biased. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: every application of the method: HAVING ACCESS TO RELIABLE LABELED DATA IS NOT ALWAYS POSSIBLE, and the labels may themselves carry the bias being measured | limitations[2] | the tagger's words | |
| limitations | The near-optimality guarantee is relative to other constrained predictors only; the derived predictor need not be good in absolute terms. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the optimality claim: THIS DOES NOT MEAN THAT THE PREDICTOR IS NECESSARILY GOOD IN AN ABSOLUTE SENSE - near-optimality is relative to other constrained predictors | limitations[3] | the tagger's words | |
| limitations | No test of this kind can tell two situations apart, and the authors prove it about their own class of test. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: a verdict of non-discrimination obtained from this criterion: the authors show no oblivious test can resolve which of two dependency structures applies, so two scenarios with different intuitive fairness are indistinguishable to it | limitations[4] | 1.0 | the tagger's words |
| limitations | Access to the features does not rescue the test, without assumptions the authors decline to make. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the suggestion that inspecting the model's features would settle what this test cannot: the authors state that without attaching meaning to the features or making causal assumptions about them, no further insight is gained | limitations[5] | 1.0 | the tagger's words |
| limitations | Meeting the criterion can require deciding individual cases partly at random. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the deployment of this criterion on individual decisions: achieving it cannot be done with a single threshold per group and requires randomization | limitations[6] | 1.0 | the tagger's words |
| limitations | Who pays for the fairness is a property of the design, and the authors name the payer. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the incidence of the utility cost reported here: the authors state the cost is borne by the company using the classifier, which can invest in better classification, rather than by the classified group, which cannot | limitations[7] | 1.0 | the tagger's words |
| concepts | equalized odds Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: DAMAGED GLYPHS IN THIS QUOTE: (cid:98). These are raw PDF font glyph indices that leaked out in place of the characters they stand for - usually a mathematical operator. The quote is verbatim to the staged text and its anchor is sound; the STAGED TEXT is what is damaged. NOT repaired: the cid-to-character mapping is defined by the font embedded in that particular PDF, so guessing it would produce a clean and unverifiable quote. Repair needs the source PDF and is a re-staging job. also in: equalized odds | concepts[1] | the paper's words | |
| concepts | equal opportunity Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: equal opportunity | concepts[5] | the paper's words | |
| concepts | demographic parity Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: demographic parity | concepts[6] | the paper's words | |
| concepts | oblivious Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: oblivious | concepts[7] | the paper's words | |
| concepts | fairness through unawareness Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: fairness through unawareness | concepts[8] | the paper's words | |
| senses | equal opportunity vs equalized odds - TWO CRITERIA, ONE STRICTLY WEAKER, AND THE PAPER PRICES THE GAP: equal opportunity relaxes equalized odds to the advantaged outcome group only, and in the case study equal opportunity keeps 92.8% of maximal profit against equalized odds' 80.2%, so a claim citing 'Hardt et al.' without saying which criterion is a claim about neither Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | senses[0] | 1.0 | the tagger's words |
| senses | oblivious - IN A FORMAL SENSE THE PAPER DEFINES: a property is oblivious if it depends only on the joint distribution of target, protected attribute and prediction, which is what makes the criterion checkable from published statistics and also what makes it blind Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | senses[1] | 1.0 | the tagger's words |
| senses | demographic parity - THE CRITERION THIS ONE IS DEFINED AGAINST: it requires the decision to be independent of the protected attribute, which the authors reject because it forbids the perfectly accurate predictor and costs the most utility of the four thresholds they compare Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | senses[2] | 1.0 | the tagger's words |
| claim_relations | contradicts → fairness through unawareness (ignoring protected attributes) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | claim_relations[0] | the tagger's words | |
| claim_relations | builds-on → Dwork et al. [DHP+12], "Fairness through Awareness" Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | claim_relations[1] | the tagger's words | |
| claim_relations | builds-on → Kleinberg, Mullainathan and Raghavan [KMR16] on calibration within groups Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | claim_relations[2] | the tagger's words | |
| claim_relations | contradicts → demographic parity as a non-discrimination criterion Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. differs in: THE AXIS IS THE FAIRNESS CRITERION. 'Quotation withheld' Two criteria, and the disagreement is over which one MEASURES discrimination - not over any fact about a dataset. | claim_relations[3] | the tagger's words | |
| content_tags | 2 — ways demographic parity is flawed Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | content_tags.quantities[0] | the tagger's words | |
| content_tags | May 2014 — Obama Big Data report warning on discrimination Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | content_tags.quantities[1] | the tagger's words | |
| content_tags | demographic parity is seriously flawed on two counts Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | content_tags.findings[0] | the tagger's words | |
| content_tags | fairness through unawareness fails because of redundant encodings Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | content_tags.findings[1] | the tagger's words | |
| content_tags | their criterion always permits the perfectly accurate predictor Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | content_tags.findings[2] | the tagger's words | |
| content_tags | the criterion aligns fairness with predictive accuracy Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. | content_tags.findings[3] | the tagger's words | |
| content_tags | protected attribute A Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: protected attribute A | content_tags.variables[0] | the tagger's words | |
| content_tags | target Y Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: target Y | content_tags.variables[1] | the tagger's words | |
| content_tags | predictor Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: predictor | content_tags.variables[2] | the tagger's words | |
| content_tags | features X Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: features X | content_tags.variables[3] | the tagger's words | |
| content_tags | demographic parity Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: demographic parity | content_tags.definitions[0] | the tagger's words | |
| content_tags | fairness through unawareness Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: fairness through unawareness | content_tags.definitions[1] | the tagger's words | |
| content_tags | oblivious notion Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: oblivious notion | content_tags.definitions[2] | the tagger's words | |
| content_tags | equal opportunity by design Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: equal opportunity by design | content_tags.definitions[3] | the tagger's words | |
| content_tags | Bayesian Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: Bayesian | content_tags.statistics[0] | the tagger's words |
Tags that could not be placed in the text:
What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.