phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | the degradation problem in very deep neural networks— “When deeper networks are able to start converging, a degradation problem has been exposed: with the network depth increasing, accuracy gets saturated (which might be unsurprising) and then degrades rapidly.” | |
| claim-type | method— “We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously.” | |
| method — evidence | — “We present comprehensive experiments on ImageNet [36] to show the degradation problem and evaluate our method.” | |
| mechanism | reformulate each layer to learn a residual function via identity shortcut connections, so very deep networks are easier to optimize and avoid the depth-degradation problem— “these residual networks are easier to optimize, and can gain accuracy from considerably increased depth” | mechanisms[0] |
| extends | plain deep convolutional networks (the degradation problem)— “with the network depth increasing, accuracy gets saturated” | claim_relations[0] |
| replicates | prior reports [11, 42] that adding layers to a deep model raises training error— “Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported in [11, 42] and thoroughly verified by our experiments.” | claim_relations[1] |
| builds-on | VGG nets [41]— “Our plain baselines (Fig. 3, middle) are mainly inspired by the philosophy of VGG nets [41] (Fig. 3, left).” | claim_relations[2] |
| builds-on | Faster R-CNN [32]— “We adopt Faster R-CNN [32] as the detection method.” | claim_relations[3] |
| extends | highway networks [42, 43]— “These gates are data-dependent and have parameters, in contrast to our identity shortcuts that are parameter-free.” | claim_relations[4] |
| sense | the degradation problem - IDENTIFIED BY TRAINING ERROR, NOT TEST ERROR: accuracy saturates then degrades rapidly with depth, and the authors' point is that this is NOT overfitting because adding layers raises the TRAINING error - the distinction is what makes it an optimisation problem rather than a generalisation one— “Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported in [11, 42] and thoroughly verified by our experiments” | senses[0] |
| sense | an identity mapping - AN IDEALISATION THE PAPER WITHDRAWS: the argument for residual learning runs through the case where identity mappings are optimal and the solver can drive the weights to zero, and the authors then state in their own words that in real cases this is unlikely— “In real cases, it is unlikely that identity mappings are optimal” | senses[1] |
| sense | the residual mapping - A REFORMULATION, PRESENTED AS A HYPOTHESIS: instead of fitting the desired underlying mapping the layers fit its residual, and the claim that this is easier to optimise is given in the authors' own words as something they hypothesize— “We hypothesize that it is easier to optimize the residual mapping than to optimize the original, unreferenced mapping” | senses[2] |
| control | overfitting as the cause of the degradation problem — held — rules out the reading that deeper networks degrade because they overfit— “Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported in [11, 42] and thoroughly verified by our experiments” | controls[0] |
| control | vanishing gradients as the cause of the optimisation difficulty in plain nets — partial — rules out the standard explanation that deep plain nets fail because signals vanish— “We argue that this optimization difficulty is unlikely to be caused by vanishing gradients. These plain networks are trained with BN [16], which ensures forward propagated signals to have non-zero variances. We also verify that the backward propagated gradients exhibit healthy norms with BN. So neither forward nor backward signals… | controls[1] |
| effect direction | vanishing gradients —causes→ optimization difficulty — no-change— “We argue that this optimization difficulty is unlikely to be caused by vanishing gradients. These plain networks are trained with BN [16], which ensures forward propagated signals to have non-zero variances. We also verify that the backward propagated gradients exhibit healthy norms with BN. So neither forward nor backward signals…— “We argue that this optimization difficulty is unlikely to be caused by vanishing gradients” | concept_edges[1] |
| weight | vanishing gradients —causes→ optimization difficulty — 0.8— “We argue that this optimization difficulty is unlikely to be caused by vanishing gradients. These plain networks are trained with BN [16], which ensures forward propagated signals to have non-zero variances. We also verify that the backward propagated gradients exhibit healthy norms with BN. So neither forward nor backward signals… | concept_edges[1] |
| outcome (stated by the tagger, no quotation) | control — held | controls[0] |
| outcome (stated by the tagger, no quotation) | control — partial | controls[1] |
| for which use | limitation — the explanation of why residual learning works: THIS HYPOTHESIS, HOWEVER, IS STILL AN OPEN QUESTION— “This hypothesis, however, is still an open question.” | limitations[0] |
| for which use | limitation — the account of what goes wrong in plain deep nets: THE REASON FOR SUCH OPTIMIZATION DIFFICULTIES WILL BE STUDIED IN THE FUTURE— “The reason for such optimization difficulties will be studied in the future.” | limitations[1] |
| for which use | limitation — the very deep regime: THERE ARE STILL OPEN PROBLEMS ON SUCH AGGRESSIVELY DEEP MODELS— “But there are still open problems on such aggressively deep models.” | limitations[2] |
| for which use | limitation — the 1202-layer result specifically: THE 1202-LAYER NETWORK MAY BE UNNECESSARILY LARGE (19.4M) FOR THIS SMALL DATASET— “The 1202-layer network may be unnecessarily large (19.4M) for this small dataset.” | limitations[3] |
| for which use | limitation — the approximation premise rather than the optimisation question of item 0: THE PREMISE THAT MULTIPLE NONLINEAR LAYERS CAN ASYMPTOTICALLY APPROXIMATE COMPLICATED FUNCTIONS is stated as a hypothesis that is STILL AN OPEN QUESTION— “This hypothesis, however, is still an open question.” | limitations[4] |
| for which use | limitation — the CIFAR-10 results: COMBINING WITH STRONGER REGULARIZATION MAY IMPROVE RESULTS, WHICH WE WILL STUDY IN THE FUTURE— “But combining with stronger regularization may improve results, which we will study in the future.” | limitations[5] |
| for which use | limitation — the intuition given for residual learning: the authors concede that in real cases— “In real cases, it is unlikely that identity mappings are optimal” | limitations[6] |
| for which use | limitation — the claim that residual mappings are easier to optimize: the authors present it as a hypothesis rather than as a demonstrated result— “We hypothesize that it is easier to optimize the residual mapping than to optimize the original, unreferenced mapping” | limitations[7] |
| replication | untested | |
| limitation | why residual mappings are easier to optimize remains an open question— “This hypothesis, however, is still an open question.” | limitations[0] |
| limitation | The cause of the optimisation difficulty in deep plain nets is not explained. | limitations[1] |
| limitation | Extremely deep (1202-layer) models remain problematic. | limitations[2] |
| limitation | The 1202-layer net's worse test error is attributed to overfitting from excess capacity on a small dataset. | limitations[3] |
| limitation | The premise that multiple nonlinear layers can asymptotically approximate complicated functions is unproven. | limitations[4] |
| limitation | Stronger regularisation was not explored and might improve the CIFAR-10 results. | limitations[5] |
| limitation | The scenario that motivates the design is conceded to be unlikely in practice. | limitations[6] |
| limitation | The paper's central premise is labelled a hypothesis by its authors. | limitations[7] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | Reformulating layers to learn residual functions with reference to the layer inputs makes very deep networks easier to optimize and able to gain accuracy from considerably increased depth. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[0] |
| evidence status note | finding — 2026-09-15 RE-CHECK: this quote is COMPOSITE - two passages joined by an ellipsis, which the merge gate refuses (rule 2). Against the scrape named in the grounding copy, the SECOND passage resolves and the FIRST does not at any tier tried (exact; whitespace collapsed; line-break hyphenation joined; Unicode NFKC with minus, dash and no-break-space folding; all whitespace removed; case-folded); the cause was not established. The grounding_copy_note's statement that the quote resolves holds for one part only. | content_tags.findings[0] |
| not machine-vouched | finding — not-in-our-copy - The quote is in NEITHER copy we hold - not in the text we tag against and not in the page we fetched ('fulltext.html') - checked with ligatures, hyphens, spacing and punctuation set aside. This body carries 18 '(cid:NNN)' glyph ids, 1 detached diacritics. We do not know what it was taken from, and no class is claimed for it. | content_tags.findings[0] |
| finding | An ensemble of residual nets reached 3.57% error on the ImageNet test set, winning 1st place in the ILSVRC 2015 classification task. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[1] |
| quantity | 152 layers — depth of the residual nets evaluated (8x deeper than VGG, lower complexity) Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[0] |
| quantity | 3.57% error — ImageNet test-set error of the ResNet ensemble (1st place ILSVRC 2015) Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[1] |
| quantity | 100 and 1000 layers — network depths analysed on CIFAR-10 Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[2] |
| variable | network depth (independent) Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.variables[0] |
| definition | residual learning Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[0] |
| evidence status note | definition — QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. | content_tags.definitions[0] |
| definition | shortcut connection Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[1] |
| definition | degradation problem Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[2] |
| key-terms | residual, networks, layers, depth, ImageNet, identity, shortcut, accuracy, error |
| concept | degradation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[4] |
| concept | overfitting Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[5] |
| concept | vanishing gradients Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[6] |
| concept | optimization difficulty Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[7] |
| concept | residual mapping Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[8] |
| concept | identity mapping Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[9] |
| mean μ | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[0] |
| regression β | as written: logistic regression Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[1] |
| not machine-vouched | regression — not-in-our-copy - The quote is in the original web page we fetched for this paper, but not in the text we tag against, which was extracted differently. Checked with ligatures, hyphens, spacing and punctuation set aside. | content_tags.statistics[1] |
| standard-deviation SD/σ | as written: Standard deviations Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[2] |
When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
Tag layer © the Meta-Tagging Project, CC BY 4.0. The paper text is © its authors/publisher
and is linked, not reproduced. Excerpts shown are short quotations for identification and verification.
· back to index
· concepts · authors
· עברית
© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824