phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | classifier opacity: a non-linear model gives a decision without saying what in the input produced it— “However, like many methods in machine learning, these models often lack a straightforward interpretability of the classifier predictions. In other words the classifier acts as a black box and does not provide detailed information about why it reaches a certain classification decision.” | |
| claim-type | method— “The overall idea of pixel-wise decomposition is to understand the contribution of a single pixel of an image x to the prediction f(x) made by a classifier f in an image classification task. We would like to find out, separately for each image x, which pixels contribute to what extent to a positive or negative classification result.” | |
| method — evidence | — “The underlying Formula (2) can be interpreted as a conservation law for the relevance R in between layers of the feature processing.” | |
| object of study — evidence | — “The overall idea of pixel-wise decomposition is to understand the contribution of a single pixel of an image x to the prediction f(x) made by a classifier f in an image classification task. We would like to find out, separately for each image x, which pixels contribute to what extent to a positive or negative classification result.” | |
| research question — evidence | — “However, like many methods in machine learning, these models often lack a straightforward interpretability of the classifier predictions. In other words the classifier acts as a black box and does not provide detailed information about why it reaches a certain classification decision.” | |
| contribution — evidence | — “We will introduce layer-wise relevance propagation as a concept defined by a set of constraints. Any solution satisfying the constraints will be considered to follow the concept of layer-wise relevance propagation.” | |
| transfer | from physics: the conservation law, taken from physics and imposed on relevance flowing between network layers— “The underlying Formula (2) can be interpreted as a conservation law for the relevance R in between layers of the feature processing.” | transfers[0] |
| mechanism | relevance conservation between layers the relevance held by a unit is redistributed onto the units that fed it, and the total is preserved from one layer to the next, so the prediction survives intact all the way down to the pixels— “The underlying Formula (2) can be interpreted as a conservation law for the relevance R in between layers of the feature processing.” | mechanisms[0] |
| mechanism | greedy layer-at-a-time propagation relevance is computed one layer at a time rather than globally, which is cheap but can strand relevance in a lower layer where it is then distributed randomly— “A first aspect that needs to be investigated is the greediness of the layer-wise relevance propagation procedure, in the sense that it is computed one layer at a time, and its potential impact on the quality of heatmaps: While computationally advantageous, some of the backpropagated relevance might encounter a dead-end in the lower layers and… | mechanisms[1] |
| mechanism | the gradient at the prediction point points toward the nearest local optimum, which may carry the SAME SIGN as the prediction - so it does not point at the boundary the explanation is relative to— “The differential Df(x) at the prediction point does not necessarily point to a root which is close under the Euclidean norm.” | mechanisms[2] |
| mechanism | a root is found by taking any candidate with the opposite prediction sign and searching the line between them - the segment must cross zero, so interval intersection locates it— “It is clear that the line l(a) = ax + (1 − a)x′ must contain a root of f which can be found by interval intersection.” | mechanisms[3] |
| mechanism | the root point is specified by CONSTRAINTS rather than by a formula - it must be a root and must lie near the data point, with the rest deferred— “In particular, the root point x 0 at which the Taylor decomposition is computed is constrained to satisfy f(x 0) = 0 and to lie not too far (e.g. within a fixed radius) from the actual data point x.” | mechanisms[4] |
| mechanism | the two approaches differ in what they conserve: the earlier work rectifies activations to keep feature maps non-negative, this one uses SIGNED activations to conserve the relevance quantity across layers— “uses rectified linear units to project information from the unfolded maps towards the inputs with the aim to ensure the feature maps to be non-negative.” | mechanisms[5] |
| distinguishes-from | the standard task of optimising a classifier for accuracy and generalisation— “Making, say, a nonlinear classification decision for one particular novel data point transparent to the user is essentially orthogonal to the standard task of optimizing for an excellent and well generalizing classifier.” | claim_relations[0] |
| critiques | the claim that non-linear models' success justifies their opacity— “While highly successful in e.g. hard classification, regression or ranking problems, their non-linearity so far has prevented these models to additionally explain and thus contribute to a better understanding about the nature of the solved problem.” | claim_relations[1] |
| sense | layer-wise relevance propagation = whatever satisfies the constraints; several different algorithms qualify— “It is important to note that the definition is not given as an algorithm or a solution of an objective with a distinct minimum. Instead, it is given as a set of constraints that the solution should satisfy. Thus, different algorithms with different resulting solutions may be admissible under these constraints.” | senses[0] |
| sense | R_d < 0 = evidence AGAINST the structure being present; R_d > 0 = evidence FOR it— “The qualitative interpretation is that R d < 0 contributes evidence against the presence of a structure which is to be classified while R d > 0 contributes evidence for its presence.” | senses[1] |
| sense | relevance at the output layer = the prediction f(x) itself— “The above example gives furthermore an intuition about what relevance R is, namely, the local contribution to the prediction function f(x). In that sense the relevance of the output layer is the prediction itself f(x).” | senses[2] |
| sense | heatmap = relevances mapped to a colour space - named, then explicitly put out of scope— “In terms of subsequent visualization, which however will not be the scope of this paper, the resulting relevances R d for each input pixel x (d) can be mapped to a color space and visualized in that way as a conventional heatmap.” | senses[3] |
| sense | 'admissible' = satisfying the constraints, which is weaker than 'correct' and the paper says so— “It is important to note that the definition is not given as an algorithm or a solution of an objective with a distinct minimum. Instead, it is given as a set of constraints that the solution should satisfy. Thus, different algorithms with different resulting solutions may be admissible under these constraints.” | senses[4] |
| control | a quantitative test that can separate two heatmaps which look equally good to a human — partial— “as a starting point” — rules out two heatmapping methods being ranked on how good they look to a human, which is the judgement the paper exists to replace— “In this paper we have proposed as a starting point a pixel-flipping method that allows to discriminate between two heatmapping methods that may otherwise look of similar quality to the human.” | controls[0] |
| control | a counterexample testing whether the conservation constraint alone yields a meaningful decomposition — failed— “This is for many classification problems not a realistic interpretation.” — rules out the conservation constraint being credited with producing a meaningful decomposition when it constrains only the total— “In terms of pixel-wise decomposition interpretation, all inputs point towards the presence of a structure if f(x) > 0 and towards the absence of a structure if f(x) < 0. This is for many classification problems not a realistic interpretation.” | controls[1] |
| control | application to a classifier the authors did not train, to show no retraining is needed — held— “without any further training or retraining” — rules out a heatmapping result that depends on the authors having trained the classifier themselves— “This desirable property is demonstrated in this paper by the heatmapping of images classified by the third-party GPU-trained ImageNet neural network. In particular, our heatmapping procedure was applied to this network without any further training or retraining.” | controls[2] |
| note | the standard task of optimising a classifier for accuracy and generalisation — the paper's framing move: explaining one decision is ORTHOGONAL to training a good classifier, so a better classifier does not make a better explanation— “Making, say, a nonlinear classification decision for one particular novel data point transparent to the user is essentially orthogonal to the standard task of optimizing for an excellent and well generalizing classifier.” | claim_relations[0] |
| weight | the standard task of optimising a classifier for accuracy and generalisation — 0.95— “Making, say, a nonlinear classification decision for one particular novel data point transparent to the user is essentially orthogonal to the standard task of optimizing for an excellent and well generalizing classifier.” | claim_relations[0] |
| note | the claim that non-linear models' success justifies their opacity — highly successful AND prevented from contributing understanding - the paper puts those in one sentence— “While highly successful in e.g. hard classification, regression or ranking problems, their non-linearity so far has prevented these models to additionally explain and thus contribute to a better understanding about the nature of the solved problem.” | claim_relations[1] |
| weight | the claim that non-linear models' success justifies their opacity — 0.85— “While highly successful in e.g. hard classification, regression or ranking problems, their non-linearity so far has prevented these models to additionally explain and thus contribute to a better understanding about the nature of the solved problem.” | claim_relations[1] |
| audit | non-linearity —causes→ black box — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'This lack of interpretability' = the classifier acting as a black box (previous sentence).— “This lack of interpretability is due to the non-linearity of the various mappings that process the raw image pixels to its feature representation and from that to the final classifier function.” | concept_edges[1] |
| effect direction (stated by the tagger, no quotation) | non-linearity —causes→ black box — increases | concept_edges[1] |
| weight | non-linearity —causes→ black box — 0.95— “This lack of interpretability is due to the non-linearity of the various mappings that process the raw image pixels to its feature representation and from that to the final classifier function.” | concept_edges[1] |
| audit | relevance —part-of→ conservation law — Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the formula is read as a conservation law for relevance; part-of is loose.— “The underlying Formula (2) can be interpreted as a conservation law for the relevance R in between layers of the feature processing.” | concept_edges[2] |
| note | relevance —part-of→ conservation law — the conservation law is a property required OF relevance, which is why it constrains rather than computes it— “The underlying Formula (2) can be interpreted as a conservation law for the relevance R in between layers of the feature processing.” | concept_edges[2] |
| weight | relevance —part-of→ conservation law — 0.85— “The underlying Formula (2) can be interpreted as a conservation law for the relevance R in between layers of the feature processing.” | concept_edges[2] |
| outcome | control — partial— “In this paper we have proposed as a starting point a pixel-flipping method that allows to discriminate between two heatmapping methods that may otherwise look of similar quality to the human.”— “as a starting point” | controls[0] |
| outcome | control — failed— “In terms of pixel-wise decomposition interpretation, all inputs point towards the presence of a structure if f(x) > 0 and towards the absence of a structure if f(x) < 0. This is for many classification problems not a realistic interpretation.”— “This is for many classification problems not a realistic interpretation.” | controls[1] |
| outcome | control — held— “This desirable property is demonstrated in this paper by the heatmapping of images classified by the third-party GPU-trained ImageNet neural network. In particular, our heatmapping procedure was applied to this network without any further training or retraining.”— “without any further training or retraining” | controls[2] |
| evidence status note | mechanism — THE QUOTE'S SUBJECT IS A CITATION THE TEXT EXTRACTION DROPPED: the paper's sentence opens with a reference to earlier work, so what 'uses rectified linear units' is that work, which the paragraph compares with this paper's approach - not this paper.— “uses rectified linear units to project information from the unfolded maps towards the inputs with the aim to ensure the feature maps to be non-negative.” | mechanisms[5] |
| limitation | the propagation is greedy - computed one layer at a time - and some relevance may hit a dead end in the lower layers and be distributed randomly— “A first aspect that needs to be investigated is the greediness of the layer-wise relevance propagation procedure, in the sense that it is computed one layer at a time, and its potential impact on the quality of heatmaps: While computationally advantageous, some of the backpropagated relevance might encounter a dead-end in the lower layers and… | limitations[0] |
| limitation | the proposed propagation rules are heuristic: whether relevance proportional to weighted neuron activations can be analytically justified is open— “Another open question relates to the heuristic nature of the proposed instances of relevance propagation, in particular, whether the distributed relevance messages being proportional to the weighted neuron activations can be analytically justified.” | limitations[1] |
| limitation | there is no way to evaluate the quality of a heatmap beyond looking at it— “Finally, it is not clear how to evaluate the quality of a heatmap beyond simple visual assessment.” | limitations[2] |
| limitation | no universal metric for quantifying heatmap quality exists; the paper defers it to future work— “Future work will need to explore the many domain- and data-specific degrees of freedom in the heatmapping process in order to ultimately propose a universal metric for quantification.” | limitations[3] |
| limitation | a decomposition satisfying the conservation equation is neither unique nor guaranteed to be meaningful - the constraint set does not pin down an answer— “As we will show now, a decomposition satisfying Eq (2) per se is neither unique, nor it is guaranteed that it yields a meaningful interpretation of the classifier prediction.” | limitations[4] |
| limitation | the Taylor approach needs a second point besides the input, and finding it is called a technical difficulty in the paper's own words— “One technical difficulty is to find a root point x 0.” | limitations[5] |
| limitation | where several minimal-norm roots exist the method must average or integrate over them rather than pick one— “In case of multiple existing roots x 0 with minimal norm, they can be averaged or integrated in order to get an average over all these solutions.” | limitations[6] |
| limitation | a competing method is specific to convolutional architectures, which the paper contrasts with its own range of applicability | limitations[7] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | a decomposition satisfying the layer-wise equation is neither unique nor necessarily meaningful | content_tags.findings[0] |
| finding | a trivially valid decomposition makes every input point the same way as the prediction, which is not a realistic interpretation | content_tags.findings[1] |
| finding | The method was applied to a third-party pre-trained ImageNet network without retraining. | content_tags.findings[2] |
| finding | Further specification of the relevance propagation rule is deferred to the user or to future work. | content_tags.findings[3] |
| finding | there is no direct relationship between a classifier's value at a point and its derivative at the same point, which is the first reason a gradient map is not an explanation | content_tags.findings[4] |
| finding | Derivatives at the prediction point are not useful for the paper's aim. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[5] |
| finding | layer-wise relevance propagation needs no second reference point at all, which is the practical advantage claimed over Taylor decomposition Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[6] |
| finding | In the example, the local-gradient explanation gives too large a contribution to an irrelevant direction. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[7] |
| variable | relevance Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.variables[0] |
| variable | prediction Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.variables[1] |
| definition | pixel-wise decomposition Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[0] |
| definition | layer-wise relevance propagation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[1] |
| definition | relevance Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[2] |
| definition | heatmap Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[3] |
| definition | pixel-flipping Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[4] |
| definition | state of maximal uncertainty Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[5] |
| concept | pixel-wise decomposition — not defined in this paper's own words; splitting a classifier's scalar prediction into one signed number per input pixel Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[0] |
| concept | layer-wise relevance propagation — not defined in this paper's own words; a concept defined by constraints any admissible solution must satisfy, not by a procedure Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[1] |
| concept | relevance — not defined in this paper's own words; the local contribution of a unit to the prediction function; at the output layer it IS the prediction Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[2] |
| concept | heatmap — not defined in this paper's own words; the per-pixel relevances mapped to a colour space; explicitly out of scope in this paper Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[3] |
| concept | conservation law — not defined in this paper's own words; the requirement that total relevance is preserved between adjacent layers Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[4] |
| concept | black box — not defined in this paper's own words; a classifier that returns a decision without detail about why Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[5] |
| concept | pixel-flipping — not defined in this paper's own words; the paper's own proposed way to tell two heatmaps apart when they look alike Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[6] |
| concept | segmentation — not defined in this paper's own words; named to be excluded - the method needs none, and no pixel-wise labels either Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[7] |
| concept | pre-trained classifier — not defined in this paper's own words; what the method is built on top of; it does not train or retrain anything Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[8] |
| concept | non-linearity — not defined in this paper's own words; named as the cause of the opacity, not merely as a property of the models Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[9] |
| concept | Taylor reference point — not defined in this paper's own words; left unspecified beyond two constraints: a root, close to the data point Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[10] |
| concept | Taylor-type decomposition — not defined in this paper's own words; explaining a prediction by first-order expansion around a ROOT of the classifier - the alternative this paper positions its own method against Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[11] |
| concept | root point — not defined in this paper's own words; a point where the classifier is maximally uncertain, f(x0)=0 - the reference against which a prediction is explained, and the free parameter of the whole approach Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[12] |
| concept | sensitivity maps — not defined in this paper's own words; visualisations built from partial derivatives AT the prediction point - argued here to be the wrong tool for explaining a prediction Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[13] |
| concept | Taylor residuum — not defined in this paper's own words; how much the first-order expansion misses; used to choose among candidate root points Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[14] |
| concept | locally linear — not defined in this paper's own words; the decomposition is linear given the root point but the root point itself depends on the input, so the algorithm as a whole is not linear Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[15] |
| conservation of total relevance across layers | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[0] |
| claim type | method — a second reading reaching the same value; recorded: method Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | alternatives.claim_type |
| method | system-or-tool-building — a second reading DIFFERING from the recorded value; recorded: system-or-tool-building Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | alternatives.method |
| phenomenon | which parts of an input image drive a nonlinear classifier's decision — a second reading DIFFERING from the recorded value; recorded: classifier opacity: a non-linear model gives a decision without saying what in the input produced it Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | alternatives.phenomenon |
When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
Tag layer © the Meta-Tagging Project, CC BY 4.0. The paper text is © its authors/publisher
and is linked, not reproduced. Excerpts shown are short quotations for identification and verification.
· back to index
· concepts · authors
· עברית
© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824