| phenomenon | deep learning in artificial neural networks— “The present survey, however, will focus on the narrower, but now commercially important, subfield of Deep Learning (DL) in Artificial Neural Networks (NNs).” |
| claim-type | method— “To measure whether credit assignment in a given NN application is of the deep or shallow type, I introduce the concept of Credit Assignment Paths or CAPs, which are chains of possibly causal links between the events of Sec. 2, e.g., from input through hidden to output layers in FNNs, or through transformations over time in RNNs.” |
| method — evidence | — “This historical survey compactly summarises relevant work, much of it from the previous millennium.” |
| transfer | from neuroscience (neurophysiology): incorporating neurophysiological insights into deep neural-network architectures (e.g. the Neocognitron)— “Apart from deep GMDH networks (Sec. 5.3), the Neocognitron (Fukushima, 1979, 1980, 2013a) was perhaps the first artificial NN that deserved the attribute deep, and the first to incorporate the neurophysiological insights of Sec. 5.2.” | transfers[0] |
| mechanism | deep learning = long credit-assignment paths: assign credit or blame for outcomes back through many layers of causal links (done by backpropagation)— “the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects” | mechanisms[0] |
| builds-on | half a century of prior neural-network and machine-learning research— “This historical survey compactly summarises relevant work, much of it from the previous millennium.” | claim_relations[0] |
| sense | credit assignment path - chains of possibly learnable causal links between actions and effects, whose DEPTH is what separates shallow from deep learners— “Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects.” | senses[0] |
| sense | the shallow/deep boundary - not a settled criterion: the author reports that experts could not agree and then stipulates a threshold for this overview— “At which problem depth does Shallow Learning end, and Deep Learning begin? Discussions with DL experts have not yet yielded a conclusive response to this question.” | senses[1] |
| sense | Very Deep Learning - a threshold defined by stipulation for this survey alone, at problem depth greater than ten— “let me just define for the purposes of this overview: problems of depth > 10 require Very Deep Learning.” | senses[2] |
| sense | credit assignment - also what the survey itself performs, on researchers rather than network links, and which the author says cannot be fully achieved— “this work should be viewed as merely a snapshot of an ongoing credit assignment process” | senses[3] |
| sense | 'solution depth' / 'problem depth' - A STIPULATIVE CASCADE THAT MAKES 'DEPTH' WELL-DEFINED WHERE THE FIELD'S OWN BOUNDARY IS NOT. Three definitions in one breath: a 'solution' is the set of used weights; the 'solution depth' is the depth of the deepest CAP in that sequence; and the 'problem depth' is 'the smallest depth of any solution'. THE THIRD IS A MINIMUM OVER SOLUTIONS, NOT A PROPERTY OF A NETWORK - so two networks solving one problem can have different solution depths while the problem depth is a single number about the PROBLEM. This record already records, in another of its senses, that experts could not agree where shallow ends and deep begins; this is the same author making the quantity precise by fiat instead of waiting for the field.— “Then the set of used weights is called a solution to the problem, and the depth of the deepest CAP within the sequence is called the solution depth. There may be other solutions (yielding different event sequences) with different depths. Given some fixed NN topology, the smallest depth of any solution is called the problem depth.” | senses[4] |
| sense | `deep learning` - THE SURVEY MARKS ITS OWN TERM'S OUTER EDGE AND DECLINES TO CROSS IT: `universal RL methods, which go beyond what is usually called DL`. A scope statement about the EXTENSION of the term rather than its content, and it is deferred to ordinary usage - `what is USUALLY called` - by an author who elsewhere stipulates his own thresholds. THE TERM IS FIXED BY STIPULATION AT ITS CENTRE AND BY CONVENTION AT ITS EDGE.— “Here I won’t further discuss universal RL methods, which go beyond what is usually called DL.” | senses[5] |
| audit | deep learning —distinct-from→ shallow learning — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says experts have NOT YET given a conclusive answer where shallow learning ends; hedged.— “At which problem depth does Shallow Learning end, and Deep Learning begin? Discussions with DL experts have not yet yielded a conclusive response to this question.” | concept_edges[0] |
| for which use | limitation — the credit-assignment aim of the survey: I ACKNOWLEDGE THE LIMITATIONS OF ATTEMPTING TO ACHIEVE THIS GOAL— “I acknowledge the limitations of attempting to achieve this goal.” | limitations[0] |
| for which use | limitation — the same scope as limitations[0] - the goal is stated as not fully achievable— “I acknowledge the limitations of attempting to achieve this goal.” | limitations[1] |
| for which use | limitation — the coverage of the survey: THROUGH AN EXPERT SELECTION BIAS I MAY HAVE MISSED IMPORTANT WORK— “Nevertheless, through an expert selection bias I may have missed important work.” | limitations[2] |
| for which use | limitation — the balance of attribution: A RELATED BIAS WAS SURELY INTRODUCED BY MY SPECIAL FAMILIARITY WITH THE WORK OF MY OWN DL RESEARCH GROUP IN THE PAST QUARTER-CENTURY— “A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century.” | limitations[3] |
| for which use | limitation — every use of the shallow/deep distinction: AT WHICH PROBLEM DEPTH DOES SHALLOW LEARNING END, AND DEEP LEARNING BEGIN? DISCUSSIONS WITH DL EXPERTS HAVE NOT YET YIELDED A CONCLUSIVE RESPONSE— “At which problem depth does Shallow Learning end, and Deep Learning begin? Discussions with DL experts have not yet yielded a conclusive response to this question.” | limitations[4] |
| for which use | limitation — the reinforcement-learning sections: THEY CANNOT SERVE AS A GENERAL RL SURVEY— “The following subsections mostly focus on certain obvious intersections between DL and RL—they cannot serve as a general RL survey.” | limitations[5] |
| replication | untested |
| limitation | fully and fairly assigning credit to all contributors is not achievable— “I acknowledge the limitations of attempting to achieve this goal.” | limitations[0] |
| limitation | The author states that the credit-assignment goal of the survey cannot be fully achieved.— “I acknowledge the limitations of attempting to achieve this goal.” | limitations[1] |
| limitation | Coverage may be incomplete because of expert selection bias.— “Nevertheless, through an expert selection bias I may have missed important work.” | limitations[2] |
| limitation | The survey is biased toward the author's own research group's work.— “A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century.” | limitations[3] |
| limitation | The boundary between shallow and deep learning is set by stipulation, not by a settled criterion.— “At which problem depth does Shallow Learning end, and Deep Learning begin? Discussions with DL experts have not yet yielded a conclusive response to this question.” | limitations[4] |
| limitation | The reinforcement-learning part is explicitly not a general survey of that field.— “The following subsections mostly focus on certain obvious intersections between DL and RL—they cannot serve as a general RL survey.” | limitations[5] |