On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation Sebastian Bach Alexander Binder Grégoire Montavon Frederick Klauschen Klaus‐Robert Müller Wojciech Samek

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation 2015 PLoS ONE 10.1371/journal.pone.0130140 bach2015-on-pixel-wise-explanations-for-non bach2015-on-pixel-wise-explanations-for-non.txt, the project's staged copy 61074252fd380df41485a39bf257484518c82c81 cc-by machine-learning

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them the contribution of each individual input pixel to one non-linear classifier's decision on one image Quotation withheld because this record is not cleared for redistribution. how to explain why a non-linear image classifier reached a particular decision, at the resolution of single pixels Quotation withheld because this record is not cleared for redistribution. layer-wise relevance propagation - a heatmapping concept defined as a set of constraints rather than an algorithm - together with a Taylor-type alternative, both applicable to pre-trained Bag-of-Words and neural classifiers Quotation withheld because this record is not cleared for redistribution. method Quotation withheld because this record is not cleared for redistribution. classifier opacity: a non-linear model gives a decision without saying what in the input produced it Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. the relevance held by a unit is redistributed onto the units that fed it, and the total is preserved from one layer to the next, so the prediction survives intact all the way down to the pixels relevance conservation between layers Quotation withheld because this record is not cleared for redistribution. relevance is computed one layer at a time rather than globally, which is cheap but can strand relevance in a lower layer where it is then distributed randomly greedy layer-at-a-time propagation Quotation withheld because this record is not cleared for redistribution. the gradient at the prediction point points toward the nearest local optimum, which may carry the SAME SIGN as the prediction - so it does not point at the boundary the explanation is relative to Quotation withheld because this record is not cleared for redistribution. a root is found by taking any candidate with the opposite prediction sign and searching the line between them - the segment must cross zero, so interval intersection locates it Quotation withheld because this record is not cleared for redistribution. the root point is specified by CONSTRAINTS rather than by a formula - it must be a root and must lie near the data point, with the rest deferred Quotation withheld because this record is not cleared for redistribution. the two approaches differ in what they conserve: the earlier work rectifies activations to keep feature maps non-negative, this one uses SIGNED activations to conserve the relevance quantity across layers about this quote: THE QUOTE'S SUBJECT IS A CITATION THE TEXT EXTRACTION DROPPED: the paper's sentence opens with a reference to earlier work, so what 'uses rectified linear units' is that work, which the paragraph compares with this paper's approach - not this paper. Quotation withheld because this record is not cleared for redistribution. the propagation is greedy - computed one layer at a time - and some relevance may hit a dead end in the lower layers and be distributed randomly Quotation withheld because this record is not cleared for redistribution. the proposed propagation rules are heuristic: whether relevance proportional to weighted neuron activations can be analytically justified is open Quotation withheld because this record is not cleared for redistribution. there is no way to evaluate the quality of a heatmap beyond looking at it Quotation withheld because this record is not cleared for redistribution. no universal metric for quantifying heatmap quality exists; the paper defers it to future work Quotation withheld because this record is not cleared for redistribution. a decomposition satisfying the conservation equation is neither unique nor guaranteed to be meaningful - the constraint set does not pin down an answer Quotation withheld because this record is not cleared for redistribution. the Taylor approach needs a second point besides the input, and finding it is called a technical difficulty in the paper's own words Quotation withheld because this record is not cleared for redistribution. where several minimal-norm roots exist the method must average or integrate over them rather than pick one Quotation withheld because this record is not cleared for redistribution. a competing method is specific to convolutional architectures, which the paper contrasts with its own range of applicability Quotation withheld because this record is not cleared for redistribution. the conservation law, taken from physics and imposed on relevance flowing between network layers Quotation withheld because this record is not cleared for redistribution. a quantitative test that can separate two heatmaps which look equally good to a human eliminates: two heatmapping methods being ranked on how good they look to a human, which is the judgement the paper exists to replace · outcome: partial Quotation withheld because this record is not cleared for redistribution. a counterexample testing whether the conservation constraint alone yields a meaningful decomposition eliminates: the conservation constraint being credited with producing a meaningful decomposition when it constrains only the total · outcome: failed Quotation withheld because this record is not cleared for redistribution. application to a classifier the authors did not train, to show no retraining is needed eliminates: a heatmapping result that depends on the authors having trained the classifier themselves · outcome: held Quotation withheld because this record is not cleared for redistribution. pixel-wise decomposition pixel-wise decomposition Quotation withheld because this record is not cleared for redistribution. layer-wise relevance propagation layer-wise relevance propagation Quotation withheld because this record is not cleared for redistribution. relevance relevance Quotation withheld because this record is not cleared for redistribution. heatmap heatmap Quotation withheld because this record is not cleared for redistribution. conservation law conservation law Quotation withheld because this record is not cleared for redistribution. black box black box Quotation withheld because this record is not cleared for redistribution. pixel-flipping pixel-flipping Quotation withheld because this record is not cleared for redistribution. segmentation segmentation Quotation withheld because this record is not cleared for redistribution. pre-trained classifier pre-trained classifier Quotation withheld because this record is not cleared for redistribution. non-linearity non-linearity Quotation withheld because this record is not cleared for redistribution. Taylor reference point Taylor reference point Quotation withheld because this record is not cleared for redistribution. Taylor-type decomposition Taylor-type decomposition Quotation withheld because this record is not cleared for redistribution. root point root point Quotation withheld because this record is not cleared for redistribution. sensitivity maps sensitivity maps Quotation withheld because this record is not cleared for redistribution. Taylor residuum Taylor residuum Quotation withheld because this record is not cleared for redistribution. locally linear locally linear Quotation withheld because this record is not cleared for redistribution. layer-wise relevance propagation = whatever satisfies the constraints; several different algorithms qualify Quotation withheld because this record is not cleared for redistribution. R_d < 0 = evidence AGAINST the structure being present; R_d > 0 = evidence FOR it Quotation withheld because this record is not cleared for redistribution. relevance at the output layer = the prediction f(x) itself Quotation withheld because this record is not cleared for redistribution. heatmap = relevances mapped to a colour space - named, then explicitly put out of scope Quotation withheld because this record is not cleared for redistribution. 'admissible' = satisfying the constraints, which is weaker than 'correct' and the paper says so Quotation withheld because this record is not cleared for redistribution. claim_type: method a second reading reaching the same value. Recorded: method — note: A BLIND second reading by the project on 2026-08-25: the record's own claim type/method/phenomenon were not looked at until every value here was written and substring-verified. Not applied - a person decides. Sampling frame: the 504 staged records carrying no alternatives block, so this never re-measures the 202. All 11 records with 130+ tags were taken as a census and 19 more spread one per discipline across 22 disciplines - the dense non-neuroscience cell is the one the 202 do not contain (inside them, 94-100% of the densest bands are neuroscience; among these 11, one is)., Quotation withheld because this record is not cleared for redistribution. method: system-or-tool-building a second reading DIFFERING from the recorded value. Recorded: system-or-tool-building — note: A BLIND second reading by the project on 2026-08-25: the record's own claim type/method/phenomenon were not looked at until every value here was written and substring-verified. Not applied - a person decides. Sampling frame: the 504 staged records carrying no alternatives block, so this never re-measures the 202. All 11 records with 130+ tags were taken as a census and 19 more spread one per discipline across 22 disciplines - the dense non-neuroscience cell is the one the 202 do not contain (inside them, 94-100% of the densest bands are neuroscience; among these 11, one is)., Quotation withheld because this record is not cleared for redistribution. phenomenon: which parts of an input image drive a nonlinear classifier's decision a second reading DIFFERING from the recorded value. Recorded: classifier opacity: a non-linear model gives a decision without saying what in the input produced it — note: A BLIND second reading by the project on 2026-08-25: the record's own claim type/method/phenomenon were not looked at until every value here was written and substring-verified. Not applied - a person decides. Sampling frame: the 504 staged records carrying no alternatives block, so this never re-measures the 202. All 11 records with 130+ tags were taken as a census and 19 more spread one per discipline across 22 disciplines - the dense non-neuroscience cell is the one the 202 do not contain (inside them, 94-100% of the densest bands are neuroscience; among these 11, one is)., Quotation withheld because this record is not cleared for redistribution. distinguishes-from → the standard task of optimising a classifier for accuracy and generalisation note: the paper's framing move: explaining one decision is ORTHOGONAL to training a good classifier, so a better classifier does not make a better explanation Quotation withheld because this record is not cleared for redistribution. critiques → the claim that non-linear models' success justifies their opacity note: highly successful AND prevented from contributing understanding - the paper puts those in one sentence Quotation withheld because this record is not cleared for redistribution. non-linearity —causes→ black box non-linearity black box direction on the measure: increases · strength of the relation: 0.95 · direction on the measure (stated by the tagger, no quotation): increases Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'This lack of interpretability' = the classifier acting as a black box (previous sentence). Quotation withheld because this record is not cleared for redistribution. relevance —part-of→ conservation law relevance conservation law strength of the relation: 0.85 · note: the conservation law is a property required OF relevance, which is why it constrains rather than computes it Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the formula is read as a conservation law for relevance; part-of is loose. Quotation withheld because this record is not cleared for redistribution. a decomposition satisfying the layer-wise equation is neither unique nor necessarily meaningful Quotation withheld because this record is not cleared for redistribution. a trivially valid decomposition makes every input point the same way as the prediction, which is not a realistic interpretation Quotation withheld because this record is not cleared for redistribution. The method was applied to a third-party pre-trained ImageNet network without retraining. Quotation withheld because this record is not cleared for redistribution. Further specification of the relevance propagation rule is deferred to the user or to future work. Quotation withheld because this record is not cleared for redistribution. there is no direct relationship between a classifier's value at a point and its derivative at the same point, which is the first reason a gradient map is not an explanation Quotation withheld because this record is not cleared for redistribution. Derivatives at the prediction point are not useful for the paper's aim. Quotation withheld because this record is not cleared for redistribution. layer-wise relevance propagation needs no second reference point at all, which is the practical advantage claimed over Taylor decomposition Quotation withheld because this record is not cleared for redistribution. In the example, the local-gradient explanation gives too large a contribution to an irrelevant direction. Quotation withheld because this record is not cleared for redistribution. pixel-wise decomposition pixel-wise decomposition Quotation withheld because this record is not cleared for redistribution. layer-wise relevance propagation layer-wise relevance propagation Quotation withheld because this record is not cleared for redistribution. relevance relevance Quotation withheld because this record is not cleared for redistribution. heatmap heatmap Quotation withheld because this record is not cleared for redistribution. pixel-flipping pixel-flipping Quotation withheld because this record is not cleared for redistribution. the sign convention that makes a root the state of maximal uncertainty - positive means presence, negative means absence state of maximal uncertainty Quotation withheld because this record is not cleared for redistribution. conservation of total relevance across layers conservation of total relevance across layers Quotation withheld because this record is not cleared for redistribution. relevance relevance Quotation withheld because this record is not cleared for redistribution. prediction prediction Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1371/journal.pone.0130140.