The Ethics of AI Ethics: An Evaluation of Guidelines

The Ethics of AI Ethics: An Evaluation of Guidelines Thilo Hagendorff

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

The Ethics of AI Ethics: An Evaluation of Guidelines 2020 Minds and Machines 10.1007/s11023-020-09517-8 hagendorff2020-the-ethics-of-ai-ethics-an-evaluat hagendorff2020-the-ethics-of-ai-ethics-an-evaluat.txt, the project's staged copy c5298cb847e1b7837bcac87e85db4f9f1fbcf978 cc-by ai-ethics

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them 22 AI ethics guidelines - which issues each one covers, which it omits, and whether any of it changes what practitioners do Quotation withheld because this record is not cleared for redistribution. whether ethical guidelines have an actual impact on human decision-making in AI and machine learning Quotation withheld because this record is not cleared for redistribution. a comparison of 22 guidelines that tallies coverage AND absence, sorts the absences into justified and unjustified, and states that the selection method decided the list Quotation withheld because this record is not cleared for redistribution. critique Quotation withheld because this record is not cleared for redistribution. a normative vocabulary assembled by many bodies at once, whose commonest terms turn out to be the ones with engineering solutions Quotation withheld because this record is not cleared for redistribution. review review Quotation withheld because this record is not cleared for redistribution. accountability, privacy and fairness appear in about 80% of the guidelines, and the author's observation is that these are exactly the aspects for which technical fixes exist or are being built. A NORMATIVE VOCABULARY SELECTED BY TRACTABILITY: what can be engineered gets written down, and what cannot - political abuse, ecological cost, democratic erosion - does not the vocabulary is shaped by what is solvable Quotation withheld because this record is not cleared for redistribution. excluding robot ethics guidelines means the list of issues lacks robotics issues. The author states this BEFORE reporting the omissions, so every absence in the paper is already qualified by the selection that produced it - which is the discipline a negative finding needs and almost never gets the sample decides the absence list Quotation withheld because this record is not cleared for redistribution. an issue can be missing without any good reason - political abuse, hidden social and ecological costs - or missing for a reason that can be stated. The author refuses to count them together, which turns a list of gaps into two different claims two kinds of absence Quotation withheld because this record is not cleared for redistribution. a marker goes in only where an issue is explicitly discussed in one or more paragraphs - unless the guideline is so short that it is nothing but brief mentions, in which case brief mentions count. THE EXCEPTION IS WHERE THE MEASUREMENT WOULD OTHERWISE PENALISE BREVITY RATHER THAN SILENCE a coding rule with its exception printed Quotation withheld because this record is not cleared for redistribution. industry guidelines suggest to legislators that self-governance suffices and no specific laws are needed. So the existence of an ethical vocabulary is itself an argument against a binding one - the guidelines are not only a record of values, they are an intervention in whether values become rules ethics as a substitute for law Quotation withheld because this record is not cleared for redistribution. the sampling method has an effect on the list of issues and omissions - excluding robot ethics guidelines removes robotics issues from the result Quotation withheld because this record is not cleared for redistribution. the selection is biased towards western and northern documents and excludes guidelines not written in English Quotation withheld because this record is not cleared for redistribution. the evaluation is semi-systematic rather than a full analysis of every available soft-law or non-legal norm document Quotation withheld because this record is not cleared for redistribution. documents about big data, algorithms or robotics, and papers on single aspects of AI ethics, were excluded about this quote: PAGE FURNITURE IS SPLICED INTO THE 'evidence' QUOTE - the publisher's page mark, a page number and the author running head: '1 3 102 T. Hagendorff'. The words on either side of it were never continuous on the page; read the quote with it deleted. Quotation withheld because this record is not cleared for redistribution. the overview of missing issues is stated as non-exhaustive Quotation withheld because this record is not cleared for redistribution. systematic-review methodology from evidence-based medicine, applied semi-systematically to normative documents rather than to trials Quotation withheld because this record is not cleared for redistribution. a stated marking criterion - explicit discussion in one or more paragraphs - applied before any tally, with the exception for guidelines that are entirely brief mentions eliminates: a word-count masquerading as coverage; a guideline that names privacy once does not score the same as one that discusses it · outcome: held Quotation withheld because this record is not cleared for redistribution. the effect of the sampling method on the omissions list, stated in the same section as the omissions eliminates: an absence being read as a property of the field when it is a property of the sample - the robotics example is given as the worked case · outcome: held Quotation withheld because this record is not cleared for redistribution. omissions sorted into those without any good reason and those that can be justified, rather than counted as one figure eliminates: a gap count in which a deliberate scope decision and a genuine blind spot are the same number · outcome: held Quotation withheld because this record is not cleared for redistribution. selection criterion declared as the discernible intention of comprehensive mapping rather than depth of detail eliminates: the longest documents dominating the sample, which would make coverage a function of document length · outcome: held Quotation withheld because this record is not cleared for redistribution. the author's own selection bias towards western, northern and English-language documents, named as a shortcoming eliminates: guidelines from outside the western, northern, English-language world - the author names the bias in his own corpus and does not correct it · outcome (stated by the tagger, no quotation): absent-declared-unreported Quotation withheld because this record is not cleared for redistribution. omissions omissions Quotation withheld because this record is not cleared for redistribution. sampling method sampling method Quotation withheld because this record is not cleared for redistribution. political abuse political abuse Quotation withheld because this record is not cleared for redistribution. technical fixes technical fixes Quotation withheld because this record is not cleared for redistribution. accountability accountability Quotation withheld because this record is not cleared for redistribution. green markers green markers Quotation withheld because this record is not cleared for redistribution. Isolated mentions Isolated mentions Quotation withheld because this record is not cleared for redistribution. AI ethics AI ethics about this quote: PAGE FURNITURE IS SPLICED INTO THE 'evidence' QUOTE - the publisher's page mark, a page number and the author running head: '1 3 102 T. Hagendorff'. The words on either side of it were never continuous on the page; read the quote with it deleted. Quotation withheld because this record is not cleared for redistribution. big data big data about this quote: PAGE FURNITURE IS SPLICED INTO THE 'evidence' QUOTE - the publisher's page mark, a page number and the author running head: '1 3 102 T. Hagendorff'. The words on either side of it were never continuous on the page; read the quote with it deleted. Quotation withheld because this record is not cleared for redistribution. depth of detail depth of detail Quotation withheld because this record is not cleared for redistribution. machine consciousness machine consciousness about this quote: PAGE FURNITURE IS SPLICED INTO THE 'evidence' QUOTE - the publisher's page mark, the title running head and a page number: '1 3 The Ethics of AI Ethics: An Evaluation of Guidelines 105'. The words on either side of it were never continuous on the page; read the quote with it deleted. Quotation withheld because this record is not cleared for redistribution. an omission - UNJUSTIFIED: missing or severely underrepresented without any good reason, such as political abuse or the hidden social and ecological costs of AI Quotation withheld because this record is not cleared for redistribution. an omission - JUSTIFIED: absent for a reason the analysis can state, which the author separates out rather than counting alongside the first Quotation withheld because this record is not cleared for redistribution. an issue covered - AS CODED: explicitly discussed in one or more paragraphs, with isolated mentions not counted Quotation withheld because this record is not cleared for redistribution. an issue covered - THE EXCEPTION TO THE CODING: unless the guideline is so short that it consists entirely of brief mentions, in which case brief mentions count Quotation withheld because this record is not cleared for redistribution. `AI` - AS USED IN THE GUIDELINES: the only term they use, referred to exclusively and almost never replaced by more specific terminology Quotation withheld because this record is not cleared for redistribution. `AI` - AS THE AUTHOR READS IT: a collective term for a wide range of technologies, or an abstract large-scale phenomenon - which is why no guideline goes into technical detail Quotation withheld because this record is not cleared for redistribution. critiques → the AI ethics guidelines themselves, on the grounds that they lack any mechanism to reinforce their own normative claims Quotation withheld because this record is not cleared for redistribution. critiques → industry self-governance, as a device for suggesting to legislators that no specific laws are necessary Quotation withheld because this record is not cleared for redistribution. critiques → the guidelines' exclusive use of the term AI, which is a collective term for a wide range of technologies and lets every document avoid technical detail Quotation withheld because this record is not cleared for redistribution. qualifies → its own omissions list, by stating that the sampling method produced it and that the selection is western, northern and English-language Quotation withheld because this record is not cleared for redistribution. green markers —distinct-from→ Isolated mentions green markers Isolated mentions differs in: whether the issue was discussed or merely named · the relation itself is separately evidenced · strength of the relation: 0.95 · differs in (stated by the tagger, no quotation): whether the issue was discussed or merely named · the relation stated: an issue counts as covered only where it is explicitly discussed in one or more paragraphs; a bare mention does not - unless the whole guideline is bare mentions Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span explains which issues got a green marker; distinct-from is loose. Quotation withheld because this record is not cleared for redistribution. AI ethics —distinct-from→ big data AI ethics big data differs in: which technology the document is about, which is the boundary the selection drew · the relation itself is separately evidenced · strength of the relation: 0.95 · differs in (stated by the tagger, no quotation): which technology the document is about, which is the boundary the selection drew · about this quote: PAGE FURNITURE IS SPLICED INTO THE 'evidence' QUOTE - the publisher's page mark, a page number and the author running head: '1 3 102 T. Hagendorff'. The words on either side of it were never continuous on the page; read the quote with it deleted. PAGE FURNITURE IS SPLICED INTO THE relation evidence QUOTE - the publisher's page mark, a page number and the author running head: '1 3 102 T. Hagendorff'. The words on either side of it were never continuous on the page; read the quote with it deleted. · the relation stated: documents about big data, algorithms or robotics were excluded from the sample even where they are ethics documents, so the issues peculiar to them are absent from the list by construction Quotation withheld because this record is not cleared for redistribution. the most frequently mentioned ethical aspects are exactly those for which technical fixes exist or are being developed the most frequently mentioned ethical aspects are exactly those for which technical fixes exist or are being developed Quotation withheld because this record is not cleared for redistribution. the sampling method used to select the guidelines has an effect on the list of issues and omissions the sampling method used to select the guidelines has an effect on the list of issues and omissions Quotation withheld because this record is not cleared for redistribution. not all omissions can be treated equally - some are missing without any good reason and some can be justified not all omissions can be treated equally - some are missing without any good reason and some can be justified Quotation withheld because this record is not cleared for redistribution. hardly any guideline discusses political abuse of AI - propaganda, bots, fake news, deepfakes, micro targeting, election fraud hardly any guideline discusses political abuse of AI - propaganda, bots, fake news, deepfakes, micro targeting, election fraud Quotation withheld because this record is not cleared for redistribution. machine consciousness is omitted from all examined guidelines machine consciousness is omitted from all examined guidelines about this quote: PAGE FURNITURE IS SPLICED INTO THE 'evidence' QUOTE - the publisher's page mark, the title running head and a page number: '1 3 The Ethics of AI Ethics: An Evaluation of Guidelines 105'. The words on either side of it were never continuous on the page; read the quote with it deleted. Quotation withheld because this record is not cleared for redistribution. Only the AI Now 2019 Report notes that AI producer and consumer practices may themselves contradict sustainability goals. Only the AI Now 2019 Report notes that AI producer and consumer practices may themselves contradict sustainability goals. Quotation withheld because this record is not cleared for redistribution. the guidelines refer exclusively to the term AI and never or very seldom use more specific terminology the guidelines refer exclusively to the term AI and never or very seldom use more specific terminology Quotation withheld because this record is not cleared for redistribution. AI ethics lacks mechanisms to reinforce its own normative claims, and deviations from the codes have no consequences AI ethics lacks mechanisms to reinforce its own normative claims, and deviations from the codes have no consequences Quotation withheld because this record is not cleared for redistribution. industry ethics guidelines serve to suggest to legislators that self-governance is sufficient and no specific laws are necessary industry ethics guidelines serve to suggest to legislators that self-governance is sufficient and no specific laws are necessary Quotation withheld because this record is not cleared for redistribution. issues absent from the guidelines, divided into those missing or severely underrepresented without any good reason and those that can be justified omissions Quotation withheld because this record is not cleared for redistribution. inserted only where an issue is explicitly discussed in one or more paragraphs; isolated mentions without further explanation do not count, unless the guideline consists entirely of brief mentions green markers Quotation withheld because this record is not cleared for redistribution. AI ethics guidelines analysed and compared 22 Quotation withheld because this record is not cleared for redistribution. of all guidelines mention accountability, privacy or fairness - the three aspects with technical fixes 80% Quotation withheld because this record is not cleared for redistribution. omissions omissions Quotation withheld because this record is not cleared for redistribution. sampling method sampling method Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1007/s11023-020-09517-8.