phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | the recipient of an algorithmic classification rarely has any concrete sense of how or why it was arrived at— “They are opaque in the sense that if one is a recipient of the output of the algorithm (the classification decision), rarely does one have any concrete sense of how or why a particular classification has been arrived at from inputs.” | |
| claim-type | method— “I argue that recognizing the distinct forms of opacity that may be coming into play in a given application is a key to determining which of a variety of technical and non-technical solutions could help to prevent harm.” | |
| method — evidence | — “I cite existing literatures in computer science, known industry practices (as they are publicly presented), and do some testing and manipulation of code as a form of lightweight code audit.” | |
| object of study — evidence | — “This article considers the issue of opacity as a problem for socially consequential mechanisms of classification and ranking, such as spam filters, credit card fraud detection, search engines, news trends, market segmentation and advertising, insurance or loan qualification, and credit scoring. These mechanisms of classification all… | |
| research question — evidence | — “They are opaque in the sense that if one is a recipient of the output of the algorithm (the classification decision), rarely does one have any concrete sense of how or why a particular classification has been arrived at from inputs.” | |
| contribution — evidence | — “In this article, I draw a distinction between three forms of opacity: (1) opacity as intentional corporate or state secrecy, (2) opacity as technical illiteracy, and (3) an opacity that arises from the characteristics of machine learning algorithms and the scale required to apply them usefully.” | |
| mechanism | THE MACHINE BUILDS ITS OWN REPRESENTATION WITHOUT REGARD FOR HUMAN COMPREHENSION - optimisation over training data does not naturally accord with human semantic explanation, so the opacity is a property of the method and not of anyone's choice or skill— “When a computer learns and consequently builds its own representation of a classification decision, it does so without regard for human comprehension. Machine optimizations based on training data do not naturally accord with human semantic explanations.” | mechanisms[0] |
| mechanism | USEFULNESS REQUIRES COMPLEXITY: a machine learning algorithm CAN be implemented so its logic is fully comprehensible, but such an instance is unlikely to be useful - accuracy carries a degree of unavoidable complexity— “Though a machine learning algorithm can be implemented simply in such a way that its logic is almost fully comprehensible, in practice, such an instance is unlikely to be particularly useful. Machine learning models that prove useful (specifically, in terms of the ‘accuracy’ of classification) possess a degree of unavoidable complexity.” | mechanisms[1] |
| mechanism | THE OPACITY IS IN THE INTERPLAY, NOT IN EITHER PART: the dataset may be large but comprehensible and the code may be written with clarity, and the complexity arises between them, in the mechanism of the algorithm operating on data— “While datasets may be extremely large but possible to comprehend and code may be written with clarity, the interplay between the two in the mechanism of the algorithm is what yields the complexity (and thus opacity).” | mechanisms[2] |
| mechanism | THE DECISION LOGIC CHANGES AS THE ALGORITHM LEARNS, so reading the code as written does not tell you what it does after training— “The internal decision logic of the algorithm is altered as it ‘learns’ on training data.” | mechanisms[3] |
| mechanism | DIMENSIONALITY MANIPULATION BREAKS THE FEATURE-TO-DIMENSION CORRESPONDENCE: with PCA or the kernel trick the relation is not one-to-one, so a 'weighted input' can no longer be traced to a human-readable property— “it is often the case that the relationship between a feature and a dimension in the model is not one-to-one.” | mechanisms[4] |
| mechanism | INTUITION FAILS AT HIGH DIMENSIONS - each of thousands of features subtly and imperceptibly shifts the classification, so debugging and improvement get harder as inputs are added— “Domingos (2012) notes that ‘intuition fails at high-dimensions.’ In other words, reasoning about, debugging, or improving the algorithm becomes more difficult with more qualities or characteristics provided as inputs, each subtly and imperceptibly shifting the resulting classification.” | mechanisms[5] |
| mechanism | SOME ALGORITHMS ARE OPAQUE IN ORDER TO WORK: spam, scam and fraud detection face an adversary who adapts, so disclosure would defeat the system - opacity here is functional rather than merely protective— “Network security applications of machine learning deal explicitly with spam, scams, and fraud and remain opaque in order to be effective.” | mechanisms[6] |
| mechanism | SECRECY MAY BE COVER RATHER THAN COMPETITIVE NECESSITY: Pasquale's reading is that the adversary is regulation itself, which makes the same observable opacity a different problem with a different remedy— “he argues that a kind of adversarial situation is indeed in play, one where the adversary is regulation itself.” | mechanisms[7] |
| critiques | proposals for code or algorithm audits as a way to evaluate for discriminatory classification, which the paper argues underestimate what reading the code of a complicated system entails and are non-specific about classes of algorithm— “A call for code ‘audits’ (where this means reading the code) and the employment of ‘auditors’ may underestimate what this would entail as far as the number of hours required to untangle the logic of the code within a complicated software system.” | claim_relations[0] |
| qualifies | Pasquale's definition of opacity as 'remediable incomprehensibility' and the regulatory-disclosure remedy that follows from it— “In reference to this, he defines ‘opacity’ as ‘remediable incomprehensibility.’” | claim_relations[1] |
| qualifies | the emerging literature on the politics of algorithms, which treats algorithmic opacity as largely intentional corporate self-protection— “One argument in the emerging literature on the ‘politics of algorithms’ is that algorithmic opacity is a largely intentional form of self-protection by corporations intent on maintaining their trade secrets and competitive advantage.” | claim_relations[2] |
| extends | scholarship on digital inequality, which has focused on the distribution of computational resources and skills rather than on subjection to computational classification— “The goal of this article was to look more deeply into machine learning algorithms and the nature of their ‘opacity’, relating this to sociological interests in classification and discrimination.” | claim_relations[3] |
| sense | opacity - THREE DIFFERENT THINGS UNDER ONE WORD, and the paper's whole point is that they are routinely conflated: (1) intentional corporate or state secrecy, (2) the fact that reading code is a specialist skill, (3) the mismatch between high-dimensional mathematical optimisation and human-scale semantic reasoning. Only the third is intrinsic to machine learning— “Three distinct forms of opacity include: (1) opacity as intentional corporate or institutional self-protection and concealment and, along with it, the possibility for knowing deception; (2) opacity stemming from the current state of affairs where writing (and reading) code is a specialist skill and; (3) an opacity that stems from the mismatch… | senses[0] |
| sense | opacity - AND THE SECOND AND THIRD SENSES ARE THE PAIR THAT GET CONFUSED, both reading as 'this is technical and complex'. The paper separates them because they have opposite remedies: education fixes illiteracy and cannot touch dimensionality— “This third form of opacity (often conflated with the second form as part of the general sense that algorithms and code are very technical and complex) is the particular focus of this article.” | senses[1] |
| sense | opacity (Pasquale's sense) - 'REMEDIABLE INCOMPREHENSIBILITY', which builds the remedy into the definition. Burrell's third form is precisely the case that definition cannot cover, since no disclosure makes it comprehensible— “In reference to this, he defines ‘opacity’ as ‘remediable incomprehensibility.’” | senses[2] |
| sense | code audit - IN THIS PAPER, TWO INCOMPATIBLE THINGS: reading the source, which is what the legal literature proposes, and observing the algorithm in action on data, which is what the third form of opacity actually requires. The paper's critique of audits turns entirely on which one is meant— “These are challenges not just of reading and comprehending code, but being able to understand the algorithm in action, operating on data.” | senses[3] |
| control | the code audit found the word 'Nigeria' neutrally weighted in the spam classifier - a check that produced reassuring evidence rather than the expected discriminatory finding, and the paper reports it that way — partial — rules out discriminatory weighting of the word 'Nigeria' as the explanation of the spam classifier's behaviour - the audit found it neutrally weighted | controls[0] |
| control | the author states plainly that the explanation his own audit produced may not be correct: innocuous words like 'visit' and 'want' were weighted as spam indicators and remained unexplained — failed Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. — rules out the audit's own explanation being taken as a description of what the classifier does, when words the author calls innocuous carry spam weight and remain unaccounted for Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | controls[1] |
| audit | curse of dimensionality —causes→ opacity — Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the span shown does not name dimensionality in its opening; causes is loose. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[0] |
| weight | curse of dimensionality —causes→ opacity — 0.9 Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[0] |
| weight | machine learning —causes→ opacity — 0.9 Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[1] |
| weight | opacity —renamed-across-fields→ interpretability — 0.95 Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[2] |
| audit | trade secrets —causes→ opacity — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but trade-secret opacity is 'one argument in the emerging literature'; attributed. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[3] |
| weight | trade secrets —causes→ opacity — 0.9 Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[3] |
| outcome | control — partial Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | controls[0] |
| outcome | control — failed Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | controls[1] |
| evidential status | mechanism — Domingos (2012) notes that ‘intuition fails at high-dimensions.’ Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | mechanisms[5] |
| evidential status | mechanism — he argues that a kind of adversarial situation is indeed in play Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | mechanisms[7] |
| limitation | the audit's explanation was produced by imposing human interpretive reasoning on a process of statistical optimisation - 'machine thinking was resolved to human interpretation' - which the author flags as a limit of his own method Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[0] |
| limitation | ambiguities remained after the audit and the author doubts the explanation was necessarily a correct one Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[1] |
| limitation | end-user explanations that list a few key criteria are at best incomplete and at worst false reassurance - which applies to the remedy the paper is closest to endorsing Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[2] |
| limitation | the paper's own examples are handwriting recognition and spam filtering, where the feature-to-dimension relation IS one-to-one, and it states that this is often not the case Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[3] |
| limitation | no single tool or process will alleviate black-boxed classification, and the right combination is said to depend on the application space rather than being specified Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[4] |
| limitation | the difficulty is stated to persist even for those with specialised training and even for computer scientists, which bounds what any literacy remedy can achieve Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[5] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | Three forms of opacity are distinguished: corporate or institutional concealment, code literacy as a specialist skill, and the mismatch between high-dimensional optimisation and human-scale interpretation. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[0] |
| finding | The form of opacity in an application must be identified before choosing a course of action to mitigate it. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[1] |
| finding | The challenge is not only reading code but understanding the algorithm in action on data. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[2] |
| finding | machine learning opacity persists even for those with specialised training, even for computer scientists Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[3] |
| definition | opacity Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[0] |
| definition | remediable incomprehensibility Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[1] |
| definition | interpretability Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[2] |
| definition | curse of dimensionality Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[3] |
| concept | opacity Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[0] |
| concept | machine learning Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[1] |
| concept | classification Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[2] |
| concept | algorithm Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[3] |
| concept | interpretability Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[4] |
| concept | remediable incomprehensibility Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[5] |
| concept | curse of dimensionality Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[6] |
| concept | training data Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[7] |
| concept | features Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[8] |
| concept | code audit Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[9] |
| concept | auditors Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[10] |
| concept | discrimination Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[11] |
| concept | principal component analysis Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[12] |
| concept | kernel trick Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[13] |
| concept | trade secrets Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[14] |
| concept | open source Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[15] |
| concept | semantic interpretation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[16] |
| concept | spam Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[17] |
When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
Tag layer © the Meta-Tagging Project, CC BY 4.0. The paper text is © its authors/publisher
and is linked, not reproduced. Excerpts shown are short quotations for identification and verification.
· back to index
· concepts · authors
· עברית
© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824