Deep Residual Learning for Image Recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun · IEEE CVPR / arXiv · 2015 · artificial-intelligence · record he2015-resnet
tag layer · index · concepts · authors · עברית

Where this record's text came from. faceted 2026-08 by reading the locally held text (pdftext); every evidence string verified as an exact substring of that text before merging. method is not on the suggested list; the work is empirical architecture comparison on ImageNet/CIFAR-10/COCO benchmarks, so 'computational-experiment' fits better than 'simulation' or 'system-or-tool-building'. The highway-networks relation is a differentiation of a concurrent method, not a factual disagreement, so I used 'extends' rather than 'contradicts' — the paper labels it explicitly ('Concurrent with our work, “highway networks” [42, 43] present shortcut connections with gating functions [15].'). Most other citations are bare numeric references with no author names in running text and so were not coded as relations. 2026-08-14 and 2026-08-15 schema-0.3 passes: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node. Update, 15 September 2026: EXCEPTION, measured with the merge gate's own predicate (Paper.locate): 2 quote(s) on this record are not found as one exact string in the current staged text - content_tags.findings[0].evidence, content_tags.statistics[1].evidence. The sentence above does not hold for them as written. Where a row records why - the quote is grounded in another copy of the paper, joins two passages with an ellipsis, or differs from the staged text by extraction noise - that note stays on the row.

The quality of the text these quotes were cut from. symbol-font-not-extractable: 18 '(cid:NNN)' glyph-index markers remain in a staged text of 40,477 characters, where the PDF's embedded font had no ToUnicode map. THE QUOTES ON THIS RECORD ARE THE PAPER'S AND THE STAGED COPY IS THE DAMAGED ONE - which is the opposite of the usual case and is why it went unnoticed: 2 of its 50 live quotes are not exact substrings of this text, and NONE of them could be located in it even by aligning the quote's WORD SEQUENCE under this damage, so the defect recorded here is present on the record but does not by itself account for the miss. Not repaired in place: editing the staged text moves every offset after the edit and re-hashes every anchor on the record. Measured at 113aae60fefc3eb1, re-measured unchanged at 54b63482778ddf56.

2 quotes on this page break a word across a line. The PDF this text was extracted from hyphenated those words at the end of a printed line — adversar- ial, repre- sentations — and the extractor kept the hyphen and the space. The quotes keep them because a quote is reproduced exactly or it proves nothing: the break is the typesetter’s, not the author’s. A search of this page will not find the affected words.

2 quotes on this page were cut from a different copy of this article. This project holds two extractions of this paper: the plain text every other quote here is anchored in, and the publisher’s own HTML edition, which we are not licensed to republish. They are verbatim in the second and cannot be found in the first — not because the tagging is wrong, but because the extraction damaged the passage: a page number dropped into the middle of a number, a formula moved out of its sentence, a ligature where two letters belong. The record names the copy for each one. The quote is the paper’s; the extraction is not.

How to read this page. The article itself is not reproduced here — this paper is not cleared for redistribution, so what follows is the tag layer alone. The evidence sentences are withheld with the article. The table at the foot lists every tag the record carries; each row names the facet it belongs to, and the whose words column says whether the tag records the paper's own wording or the tagger's sentence about the paper. Every tag links to each other paper in the corpus carrying it. The machine-readable edition of the same thing is the TEI file beside this one, where the tags live in standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.

Other fields define these differently the same word, defined another way in another discipline
deep learning 11 fields · accuracy 9 fields · overfitting 6 fields · logistic regression 3 fields · standard-deviation 3 fields · identity 2 fields

Tags on this paper

Every tag the record carries, each linking to every other paper in the corpus that carries it. A tag appears here whether or not its words occur in the article — the spans below are only the ones a sentence in this paper licenses. Of the 10 concept tags on this paper, 6 are grounded in a quoted sentence.

residual network deep learning skip connection image recognition degradation overfitting vanishing gradients optimization difficulty residual mapping identity mapping reformulate each layer to learn a residual function via identity shortcut connections, so very deep networks are easier to optimize and avoid the depth-degradation problem residual networks layers depth ImageNet identity shortcut accuracy error residual learning shortcut connection degradation problem network depth mean regression logistic regression standard-deviation Standard deviations system-or-tool-building

Method, as described: computational-experiment

Legend

relation to other work	`claim_relations`	how this paper stands to another: agrees, contradicts, extends, qualifies
claim type	`claim_type`	the shape of the claim: existence, mechanism, method, critique, prediction, null
concept edge	`concept_edges`	an asserted relation between two concepts, with the sentence asserting it
concept	`concepts`	an idea the paper works with, quoted where the paper uses it
content tag	`content_tags`	a finding, definition, quantity, statistic or variable named in the paper
control	`controls`	an experiment run to eliminate a rival explanation
limitation	`limitations`	what the paper says its evidence cannot support
mechanism	`mechanisms`	how the reported effect is said to come about
method	`method`	how the paper produced its evidence
phenomenon	`phenomenon`	the effect or regularity under study
sense	`senses`	a distinct use of a term, quoted where the paper fixes it

The paper

The full text is not reproduced here. This article is not cleared for redistribution, so the page carries the tag layer only. Its evidence sentences are withheld; the grounding status remains with each tag, and the article itself is at https://arxiv.org/abs/1512.03385. The reading view exists for the papers whose licence permits it.

Every tag on this page

claim_relations 5 · claim_type 1 · concept_edges 1 · concepts 6 · content_tags 10 · controls 2 · limitations 8 · mechanisms 1 · method 1 · phenomenon 1 · senses 3

Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; none of 1 relation here carry that mark. The rest are the tagger’s reading of the sentence quoted beside them.

facet	tag	field	conf.	whose words
claim_type	method Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_type`		the paper's words
phenomenon	the degradation problem in very deep neural networks Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`phenomenon`		the paper's words
method	system-or-tool-building Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: system-or-tool-building	`method`		the paper's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: reformulate each layer to learn a residual function via identity shortcut connections, so very deep networks are easier to optimize and avoid the depth-degradation problem	`mechanisms[0]`		the tagger's words
limitations	why residual mappings are easier to optimize remains an open question Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the explanation of why residual learning works: THIS HYPOTHESIS, HOWEVER, IS STILL AN OPEN QUESTION	`limitations[0]`		the tagger's words
limitations	The cause of the optimisation difficulty in deep plain nets is not explained. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the account of what goes wrong in plain deep nets: THE REASON FOR SUCH OPTIMIZATION DIFFICULTIES WILL BE STUDIED IN THE FUTURE	`limitations[1]`		the tagger's words
limitations	Extremely deep (1202-layer) models remain problematic. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the very deep regime: THERE ARE STILL OPEN PROBLEMS ON SUCH AGGRESSIVELY DEEP MODELS	`limitations[2]`		the tagger's words
limitations	The 1202-layer net's worse test error is attributed to overfitting from excess capacity on a small dataset. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the 1202-layer result specifically: THE 1202-LAYER NETWORK MAY BE UNNECESSARILY LARGE (19.4M) FOR THIS SMALL DATASET	`limitations[3]`		the tagger's words
limitations	The premise that multiple nonlinear layers can asymptotically approximate complicated functions is unproven. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the approximation premise rather than the optimisation question of item 0: THE PREMISE THAT MULTIPLE NONLINEAR LAYERS CAN ASYMPTOTICALLY APPROXIMATE COMPLICATED FUNCTIONS is stated as a hypothesis that is STILL AN OPEN QUESTION	`limitations[4]`		the tagger's words
limitations	Stronger regularisation was not explored and might improve the CIFAR-10 results. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the CIFAR-10 results: COMBINING WITH STRONGER REGULARIZATION MAY IMPROVE RESULTS, WHICH WE WILL STUDY IN THE FUTURE	`limitations[5]`		the tagger's words
limitations	The scenario that motivates the design is conceded to be unlikely in practice. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the intuition given for residual learning: the authors concede that in real cases	`limitations[6]`	1.0	the tagger's words
limitations	The paper's central premise is labelled a hypothesis by its authors. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the claim that residual mappings are easier to optimize: the authors present it as a hypothesis rather than as a demonstrated result	`limitations[7]`	1.0	the tagger's words
controls	overfitting as the cause of the degradation problem Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. eliminates: the reading that deeper networks degrade because they overfit · outcome (stated by the tagger, no quotation): held	`controls[0]`	1.0	the tagger's words
controls	vanishing gradients as the cause of the optimisation difficulty in plain nets Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. eliminates: the standard explanation that deep plain nets fail because signals vanish · outcome (stated by the tagger, no quotation): partial	`controls[1]`	0.8	the tagger's words
concepts	degradation Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: degradation	`concepts[4]`		the paper's words
concepts	overfitting Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: overfitting	`concepts[5]`		the paper's words
concepts	vanishing gradients Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: vanishing gradients	`concepts[6]`		the paper's words
concepts	optimization difficulty Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: optimization difficulty	`concepts[7]`		the paper's words
concepts	residual mapping Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: residual mapping	`concepts[8]`		the paper's words
concepts	identity mapping Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: identity mapping	`concepts[9]`		the paper's words
senses	the degradation problem - IDENTIFIED BY TRAINING ERROR, NOT TEST ERROR: accuracy saturates then degrades rapidly with depth, and the authors' point is that this is NOT overfitting because adding layers raises the TRAINING error - the distinction is what makes it an optimisation problem rather than a generalisation one Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[0]`	1.0	the tagger's words
senses	an identity mapping - AN IDEALISATION THE PAPER WITHDRAWS: the argument for residual learning runs through the case where identity mappings are optimal and the solver can drive the weights to zero, and the authors then state in their own words that in real cases this is unlikely Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[1]`	1.0	the tagger's words
senses	the residual mapping - A REFORMULATION, PRESENTED AS A HYPOTHESIS: instead of fitting the desired underlying mapping the layers fit its residual, and the claim that this is easier to optimise is given in the authors' own words as something they hypothesize Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[2]`	1.0	the tagger's words
claim_relations	extends → plain deep convolutional networks (the degradation problem) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[0]`		the tagger's words
claim_relations	replicates → prior reports [11, 42] that adding layers to a deep model raises training error Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[1]`		the tagger's words
claim_relations	builds-on → VGG nets [41] Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[2]`		the tagger's words
claim_relations	builds-on → Faster R-CNN [32] Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[3]`		the tagger's words
claim_relations	extends → highway networks [42, 43] Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[4]`		the tagger's words
concept_edges	vanishing gradients —causes→ optimization difficulty Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. direction on the measure: no-change · strength of the relation: 0.8 also in: vanishing gradients · optimization difficulty	`concept_edges[1]`		the tagger's words
content_tags	152 — depth of the residual nets evaluated (8x deeper than VGG, lower complexity) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[0]`		the tagger's words
content_tags	3.57% — ImageNet test-set error of the ResNet ensemble (1st place ILSVRC 2015) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[1]`		the tagger's words
content_tags	100 and 1000 — network depths analysed on CIFAR-10 Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[2]`		the tagger's words
content_tags	An ensemble of residual nets reached 3.57% error on the ImageNet test set, winning 1st place in the ILSVRC 2015 classification task. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[1]`		the tagger's words
content_tags	network depth Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: network depth	`content_tags.variables[0]`		the tagger's words
content_tags	residual learning Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. also in: residual learning	`content_tags.definitions[0]`		the tagger's words
content_tags	shortcut connection Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: shortcut connection	`content_tags.definitions[1]`		the tagger's words
content_tags	degradation problem Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: degradation problem	`content_tags.definitions[2]`		the tagger's words
content_tags	mean Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: mean	`content_tags.statistics[0]`		the tagger's words
content_tags	standard-deviation Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: standard-deviation	`content_tags.statistics[2]`		the tagger's words

Interpretive choices

Tags that could not be placed in the text:

content_tags.findings[0] — quotation withheld — the page has already reproduced as much of this paper as the rights budget allows
about this quote: 2026-09-15 RE-CHECK: this quote is COMPOSITE - two passages joined by an ellipsis, which the merge gate refuses (rule 2). Against the scrape named in the grounding copy, the SECOND passage resolves and the FIRST does not at any tier tried (exact; whitespace collapsed; line-break hyphenation joined; Unicode NFKC with minus, dash and no-break-space folding; all whitespace removed; case-folded); the cause was not established. The grounding_copy_note's statement that the quote resolves holds for one part only. · NOT MACHINE-VOUCHED: not-in-our-copy - The quote is in NEITHER copy we hold - not in the text we tag against and not in the page we fetched ('fulltext.html') - checked with ligatures, hyphens, spacing and punctuation set aside. This body carries 18 '(cid:NNN)' glyph ids, 1 detached diacritics. We do not know what it was taken from, and no class is claimed for it.
content_tags.statistics[1] — quotation withheld — the page has already reproduced as much of this paper as the rights budget allows
NOT MACHINE-VOUCHED: not-in-our-copy - The quote is in the original web page we fetched for this paper, but not in the text we tag against, which was extracted differently. Checked with ligatures, hyphens, spacing and punctuation set aside.

What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.