phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | fairness conditions for probabilistic risk scores— “Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups.” | |
| claim-type | existence— “Our main result, however, is that these conditions are in general incompatible with each other; they can only be simultaneously satisfied in certain highly constrained cases.” | |
| method — evidence | — “We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously.” | |
| transfer | from statistics / machine learning: probabilistic fairness criteria (calibration, error-rate balance) applied to risk scores used in consequential legal / policy decisions— “concern over discrimination in machine learning has led to a large body of new work seeking to better understand and prevent it” | transfers[0] |
| mechanism | an impossibility theorem: calibration and balanced error rates across groups cannot all hold simultaneously except when base rates are equal or prediction is perfect— “there is no method that can satisfy these three conditions simultaneously” | mechanisms[0] |
| contradicts | the assumption that fairness criteria can be jointly satisfied— “there is no method that can satisfy these three conditions simultaneously” | claim_relations[0] |
| extends | Dietrich et al.'s empirical observation that error-rate symmetry across groups is hard to achieve when base rates differ— “Our formulation of the balance conditions for the positive and negative classes, and our result showing the incompatibility of these conditions with calibration, provides a theoretical basis for such observations.” | claim_relations[1] |
| builds-on | the Angwin et al. critique of the COMPAS risk tool and its counter-arguments— “For example, in the analysis of the COMPAS risk tool for criminal defendants, the critique by Angwin et al. focused on the risk tool’s violation of conditions (B) and (C); the counter-arguments established that it satisfies condition (A).” | claim_relations[2] |
| extends | the concurrent binary-classifier analysis of Hardt et al. [19]— “In recent work concurrent with ours, Hardt et al. consider the natural analogues of our conditions (B) and (C), balance for the negative and positive classes, in the case of classifiers that output binary “yes/no” predictions rather than real-valued scores as in our case [19].” | claim_relations[3] |
| sense | fairness - AS CALIBRATION WITHIN GROUPS (condition A): within each group, the people in a bin scored v really do belong to the positive class at rate v, i.e. the scores mean what they claim to mean when read separately in each group— “Calibration within groups requires that for each group t, and each bin b with associated score vb, the expected number of people from group t in b who belong to the positive class should be a vb fraction of the expected number of people from group t assigned to b” | senses[0] |
| sense | fairness - AS BALANCE FOR THE NEGATIVE CLASS (condition B): people who do NOT have the property get the same average score whichever group they are in, so the errors are not systematically worse for one group's negative instances— “Balance for the negative class requires that the average score assigned to people of group 1 who belong to the negative class should be the same as the average score assigned to people of group 2 who belong to the negative class. In other words, the assignment of scores shouldn’t be systematically more inaccurate for negative instances in one… | senses[1] |
| sense | fairness - AS BALANCE FOR THE POSITIVE CLASS (condition C): people who DO have the property get the same average score whichever group they are in - the condition the COMPAS critique turned on— “Balance for the positive class symmetrically requires that the average score assigned to people of group 1 who belong to the positive class should be the same as the average score assigned to people of group 2 who belong to the positive class” | senses[2] |
| sense | the impossibility - AS A FACT ABOUT BASE RATES, NOT ABOUT A DOMAIN: the authors state that the trade-off is not a fact about medicine but about risk estimates whenever base rates differ, which is the paper stating its own transfer condition— “The point is that this trade-off among (a), (b), and (c) is not a fact about medicine; it is simply a fact about risk estimates when the base rates differ between two groups” | senses[3] |
| edge | balance for the positive class —distinct-from→ statistical parity— “Balance for the positive and negative classes, however, is a goal that can be discussed independently of statistical parity, since these two balance conditions simply ask that once we condition on the “correct” answer for a person, the chance of making a mistake on them should not depend on which group they belong to.” | concept_edges[0] |
| edge | balance for the negative class —distinct-from→ statistical parity— “Balance for the positive and negative classes, however, is a goal that can be discussed independently of statistical parity, since these two balance conditions simply ask that once we condition on the “correct” answer for a person” | concept_edges[1] |
| note | Balance for the positive and negative classes —distinct-from→ statistical parity — RE-GROUNDS another row of this record: span stated the relation, endpoint names did not match its words.— “Balance for the positive and negative classes, however, is a goal that can be discussed independently of statistical parity, since these two balance conditions simply ask that once we condition on the “correct” answer for a person, the chance of making a mistake on them should not depend on which group they belong to.” | concept_edges[2] |
| relation | Balance for the positive and negative classes —distinct-from→ statistical parity — the balance conditions are a goal that can be pursued INDEPENDENTLY of statistical parity— “Balance for the positive and negative classes, however, is a goal that can be discussed independently of statistical parity, since these two balance conditions simply ask that once we condition on the “correct” answer for a person, the chance of making a mistake on them should not depend on which group they belong to.”— “can be discussed independently of statistical parity” | concept_edges[2] |
| for which use | limitation — the incompatibility result itself: it holds EXCEPT IN HIGHLY CONSTRAINED SPECIAL CASES— “that except in highly constrained special cases, there” | limitations[0] |
| for which use | limitation — any algorithmic use of the fairness conditions: IT IS AN OPEN QUESTION WHETHER THERE IS A POLYNOMIAL-TIME ALGORITHM TO FIND A FAIR ASSIGNMENT OF MINIMUM LOSS, OR EVEN TO DETERMINE WHETHER A NON-TRIVIAL FAIR SOLUTION EXISTS— “It is an open question whether there is a polynomial-time algorithm to find a fair assignment of minimum loss, or even to determine whether a non-trivial fair solution exists.” | limitations[1] |
| for which use | limitation — the NP-completeness result only: IT APPLIES ONLY TO THE CASE IN WHICH EVERYONE WITH A GIVEN FEATURE VECTOR MUST BE ASSIGNED TO THE SAME BIN— “(Our NP-completeness result applies only to the case in which everyone with a given feature vector must be assigned to the same bin.)” | limitations[2] |
| for which use | limitation — any applied reading of the impossibility result: NOR IS OUR INTENTION TO ANALYZE WHICH DEFINITIONS OF FAIRNESS ARE VIOLATED IN PARTICULAR APPLICATIONS OR DATASETS— “Nor is our intention to analyze which definitions of fairness are violated in particular applications or datasets.” | limitations[3] |
| for which use | limitation — the two-group presentation: WE FOCUS ON THE CASE OF TWO GROUPS FOR SIMPLICITY OF EXPOSITION, with extension described as straightforward— “We focus on the case of two groups for simplicity of exposition, but it is straightforward to extend all of our definitions to the case of more than two groups.” | limitations[4] |
| for which use | limitation — the known-p-sigma assumption, and the direction it cuts: THE CASE IN WHICH THE VALUE OF p-sigma IS UNKNOWN IS AN IMPORTANT VERSION OF THE PROBLEM AS WELL; HOWEVER, SINCE OUR MAIN RESULTS ESTABLISH STRONG LIMITATIONS ON WHAT IS ACHIEVABLE, THESE LIMI[tations carry over]— “Clearly the case in which the value of pσ is unknown is an important version of the problem as well; however, since our main results establish strong limitations on what is achievable, these limitations are only stronger because they apply even to the case of known pσ.” | limitations[5] |
| for which use | limitation — a claim that a procedure is unbiased because it never sees group membership: this model's rule has access only to the feature vector, and the authors state that this does not mean the rule is incapable of exhibiting bias between the two groups— “Note that the rule does not have access to the group t of the person being considered, only their feature vector σ. (As we will see, this does not mean that the rule is incapable of exhibiting bias between the two groups.)” | limitations[6] |
| for which use | limitation — the generality claimed for this model: the assumption that group adds no information beyond the feature vector is justified by allowing instances where people of different groups never share a feature vector, so that the feature vector implicitly conveys the group— “the assumption that the group provides no additional information beyond σ does not restrict the generality of the model, since we can always consider instances in which people of different groups never have the same feature vector σ, and hence σ implicitly conveys perfect information about a person’s group” | limitations[7] |
| for which use | limitation — this theorem as covering fairness generally: statistical parity is excluded from the three conditions, on the stated ground that classification and risk assessment are broader activities where it is often neither feasible nor desirable— “In some cases statistical parity is a central goal (and in some it is legally mandated), but the examples considered so far suggest that classification and risk assessment are much broader activities where statistical parity is often neither feasible nor desirable” | limitations[8] |
| replication | untested | |
| limitation | the incompatibility holds except in highly constrained special cases— “that except in highly constrained special cases, there” | limitations[0] |
| limitation | The computational tractability of finding a minimum-loss fair assignment (or even deciding whether a non-trivial one exists) is left open.— “It is an open question whether there is a polynomial-time algorithm to find a fair assignment of minimum loss, or even to determine whether a non-trivial fair solution exists.” | limitations[1] |
| limitation | The NP-completeness result covers only the integral case, not the general model where a feature vector may be split across bins.— “(Our NP-completeness result applies only to the case in which everyone with a given feature vector must be assigned to the same bin.)” | limitations[2] |
| limitation | The paper deliberately does not analyse which fairness definitions are violated in real applications or datasets.— “Nor is our intention to analyze which definitions of fairness are violated in particular applications or datasets.” | limitations[3] |
| limitation | The exposition is restricted to two groups.— “We focus on the case of two groups for simplicity of exposition, but it is straightforward to extend all of our definitions to the case of more than two groups.” | limitations[4] |
| limitation | The model assumes the true positive-class rate for each feature vector is known to the procedure; the unknown case is not analysed. | limitations[5] |
| limitation | Withholding the group attribute from the procedure does not stop it exhibiting bias between groups, and the paper says so in a parenthesis as it defines the model. | limitations[6] |
| limitation | The modelling assumption that group membership adds nothing beyond the feature vector is defended by a construction in which the feature vector conveys group perfectly. | limitations[7] |
| limitation | A fourth and legally mandated notion of fairness is set aside as often neither feasible nor desirable, so the impossibility result does not speak to it. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[8] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | Except in highly constrained special cases, no method can satisfy the three fairness conditions simultaneously. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[0] |
| finding | Even satisfying all three conditions approximately requires the data to lie in an approximate version of one of the constrained special cases. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[1] |
| quantity | three fairness conditions — the number of fairness conditions proven jointly unsatisfiable Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[0] |
| variable | base rates (independent) Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.variables[0] |
| definition | balance for the positive class Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[1] |
| key-terms | fairness, groups, calibrated, risk, conditions, classification, recidivism, defendants |
| concept | calibration Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[1] |
| concept | risk score Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[2] |
| concept | balance for the positive class Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[4] |
| concept | balance for the negative class Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[5] |
| concept | statistical parity Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[6] |
| concept | base rates Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[7] |
| concept | Balance for the positive and negative classes Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[8] |
| concept | statistical parity Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[9] |
| mean μ | as written: average value Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[1] |
| evidence status note | mean — QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. | content_tags.statistics[1] |
When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
Tag layer © the Meta-Tagging Project, CC BY 4.0. The paper text is © its authors/publisher
and is linked, not reproduced. Excerpts shown are short quotations for identification and verification.
· back to index
· concepts · authors
· עברית
© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824