phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | a field whose knowledge is fragmented across partial reviews, so that assembling a working understanding requires reading past the reviews— “Several published DL review papers have been presented in the last few years. However, all of them have only been addressed one side focusing on one application or topic such as the review of CNN architectures, DL for classification of plant diseases, DL for object detection, DL applications in medical image analysis, and etc. Although these… | |
| claim-type | critique— “Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it.” | |
| method — evidence | — “We have reviewed more than 300 papers on various DL topics. There are 108 papers from the year 2020, 76 papers from the year 2019, and 48 papers from the year 2018. This indicates that this review focused on the latest publications in the field of DL.” | |
| object of study — evidence | — “We review current challenges (limitations) of Deep Learning including lack of training data, Imbalanced Data, Interpretability of data, Uncertainty scaling, Catastrophic forgetting, Model compression, Overfitting, Vanishing gradient problem, Exploding Gradient Problem, and Underspecification.” | |
| research question — evidence | — “Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it.” | |
| contribution — evidence | — “We review current challenges (limitations) of Deep Learning including lack of training data, Imbalanced Data, Interpretability of data, Uncertainty scaling, Catastrophic forgetting, Model compression, Overfitting, Vanishing gradient problem, Exploding Gradient Problem, and Underspecification.” | |
| transfer | from neuroscience: the architecture of the mammalian visual cortex, carried into network design - the CNN's local receptive fields are presented as a simulation of cells that sense only small regions of a scene— “DL, a subset of ML (Fig. 2), is inspired by the information processing patterns found in the human brain. DL does not require any human-designed rules to operate; rather, it uses a large amount of data to map the given input to specific labels.” | transfers[0] |
| transfer | from computer vision: pre-trained representations carried from natural-image datasets into medical imaging, and the paper's own report that this particular transfer does not work— “Nevertheless, although TL enhances the accuracy of several tasks in the fields of pattern recognition and computer vision, there is an essential issue related to the source data type used by the TL as compared to the target dataset. For instance, enhancing the medical image classification performance of CNN models is achieved by training the… | transfers[1] |
| mechanism | vanishing gradient through repeated multiplication back-propagation multiplies each layer's derivative down the network, so with a squashing activation like the sigmoid, N small derivatives multiplied together make the gradient decline exponentially - and the first layers, which recognise the essential elements of the input, are the ones that stop updating— “However, this weight updating may not occur in some cases due to a vanishingly small gradient, which in the worst case means that no extra training is possible and the neural network will stop completely.” | mechanisms[0] |
| mechanism | catastrophic forgetting by fine-tuning incorporating new information into a trained model interferes with what it already learned; fine-tuning on only the new class leaves performance on the old classes unsuccessful, so continually renewed data forces retraining from scratch— “This is defined as incorporating new information into a plain DL model, made possible by interfering with the learned information. For instance, consider a case where there are 1000 types of flowers and a model is trained to classify these flowers, after which a new type of flower is introduced; if the model is fine-tuned only with this new… | mechanisms[1] |
| mechanism | domain mismatch defeating transfer pre-training transfers representations learned from the source distribution; where the source is natural images and the target is raw medical images the learned representation does not apply, so the transferred model performs no better than one trained from scratch— “Nevertheless, although TL enhances the accuracy of several tasks in the fields of pattern recognition and computer vision, there is an essential issue related to the source data type used by the TL as compared to the target dataset. For instance, enhancing the medical image classification performance of CNN models is achieved by training the… | mechanisms[2] |
| mechanism | underspecification selecting among equivalent solutions many parameter settings satisfy the training objective equally well, so the pipeline does not determine which one is chosen; small modifications force the model toward a completely different solution and hence different predictions in deployment— “It has been shown that small modifications can force a model towards a completely different solution as well as lead to different predictions in deployment domains.” | mechanisms[3] |
| critiques | existing DL review papers, each of which addresses a single application or topic and so leaves an overall lack of knowledge about the field— “Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it.” | claim_relations[0] |
| builds-on | the 2020 Google identification of underspecification as a distinct failure mode of ML pipelines— “In 2020, a team of computer scientists at Google has identified a new challenge called underspecification. ML models including DL models often show surprisingly poor behavior when they are tested in real-world applications such as computer vision, medical imaging, natural language processing, and medical genomics. The reason behind the weak… | claim_relations[1] |
| qualifies | transfer learning from ImageNet as the standard remedy for insufficient training data in medical imaging— “It has further been proven that TL from different domains does not significantly affect performance on medical imaging tasks, as lightweight models trained from scratch perform nearly as well as standard ImageNet-transferred models. Therefore, there exists scenarios in which using pre-trained models do not become an affordable solution.” | claim_relations[2] |
| sense | RL - REPRESENTATION LEARNING. Given in the introduction as another name for deep learning itself— “Among the different ML algorithms, deep learning (DL) is very commonly employed in these applications. Another name for DL is representation learning (RL).” | senses[0] |
| sense | RL - REINFORCEMENT LEARNING. Given 16,670 characters later as another name for deep reinforcement learning, a distinct technique the same paper classifies separately. Neither definition acknowledges the other— “DL techniques are classified into three major categories: unsupervised, partially supervised (semi-supervised) and supervised. Furthermore, deep reinforcement learning (DRL), also known as RL, is another type of learning technique, which is mostly considered to fall into the category of partially supervised (and occasionally unsupervised)… | senses[1] |
| sense | transfer learning - AS RECOMMENDED: enhances accuracy across pattern recognition and computer vision, and is the paper's answer to insufficient training data— “Nevertheless, although TL enhances the accuracy of several tasks in the fields of pattern recognition and computer vision, there is an essential issue related to the source data type used by the TL as compared to the target dataset. For instance, enhancing the medical image classification performance of CNN models is achieved by training the… | senses[2] |
| sense | transfer learning - AS MEASURED ACROSS DISSIMILAR DOMAINS: does not significantly affect performance, with models trained from scratch performing nearly as well. Same technique, opposite verdict, and the disambiguator is source-target similarity— “It has further been proven that TL from different domains does not significantly affect performance on medical imaging tasks, as lightweight models trained from scratch perform nearly as well as standard ImageNet-transferred models. Therefore, there exists scenarios in which using pre-trained models do not become an affordable solution.” | senses[3] |
| sense | Feature engineering versus deep representation learning - conventional ML constructs task-specific features manually, whereas DL extracts layered discriminative features automatically— “Usually, the effectiveness of an ML algorithm is highly dependent on the integrity of the input-data representation. It has been shown that a suitable data representation provides an improved performance when compared to a poor data representation. Thus, a significant research trend in ML for many years has been feature engineering, which has… | senses[4] |
| sense | Recurrent state - the hidden layer functions as short-term memory carrying information from earlier elements of a sequence— “RNNs are a commonly employed and familiar algorithm in the discipline of DL. RNN is mainly applied in the area of speech processing and NLP contexts. Unlike conventional networks, RNN uses sequential data in the network. Since the embedded structure in the sequence of the data delivers valuable information, this feature is fundamental to a… | senses[5] |
| sense | CNN locality - sparse local connections and shared weights exploit image structure while reducing parameter and memory requirements— “In the field of DL, the CNN is the most famous and commonly employed algorithm. The main benefit of CNN compared to its predecessors is that it automatically identifies the relevant features without any human supervision. CNNs have been extensively applied in a range of different fields, including computer vision, speech processing, Face… | senses[6] |
| sense | Pooling trade-off - subsampling reduces parameters and computation but can discard the location of a detected feature— “Convolutional Layer: In CNN architecture, the most significant component is the convolutional layer. It consists of a collection of convolutional filters (so-called kernels). The input image, expressed as N-dimensional metrics, is convolved with these filters to generate the output feature map. Three types of pooling operations Pooling Layer… | senses[7] |
| sense | Activation-function family - ReLU reduces computation but can permanently deactivate neurons; leaky, noisy, and parametric variants modify that behavior— “Activation Function (non-linearity) Mapping the input to the output is the core function of all types of activation function in all types of neural network. The input value is determined by computing the weighted summation of the neuron input along with its bias (if present). This means that the activation function makes the decision as to… | senses[8] |
| sense | Fitting-state distinction - over-fitting fails on unseen data, under-fitting fails to learn enough, and just-fitting succeeds on both training and test sets— “Over-fitting and under-fitting issues For CNN models, over-fitting represents the central issue associated with obtaining well-behaved generalization. The model is entitled over-fitted in cases where the model executes especially well on training data and does not succeed on test data (unseen data) which is more explained in the latter… | senses[9] |
| sense | Dense connectivity - DenseNet concatenates all preceding feature maps, improving information flow at the cost of more feature maps and parameter expense— “To solve the problem of the vanishing gradient, DenseNet was presented, following the same direction as ResNet and the Highway network. One of the drawbacks of ResNet is that it clearly conserves information by means of preservative individuality transformations, as several layers contribute extremely little or no information. In addition… | senses[10] |
| sense | Uncertainty calibration - raw softmax probabilities may be overconfident and require post-scaling before they function as reliable confidence estimates— “Commonly, the final prediction label is not the only label required when employing DL techniques to achieve the prediction; the score of confidence for every inquiry from the model is also desired. The score of confidence is defined as how confident the model is in its prediction. Since the score of confidence prevents belief in unreliable… | senses[11] |
| sense | Compression taxonomy - parameter pruning, knowledge distillation, compact filters, and low-rank factorization reduce different sources of model cost— “To obtain well-trained models that can still be employed productively, DL models have intensive memory and computational requirements due to their huge complexity and large numbers of parameters. One of the fields that is characterized as data-intensive is the field of healthcare and environmental science. These needs reduce the deployment of… | senses[12] |
| sense | Application-task boundary - localization uses one bounding box, detection may use many, segmentation outlines pixels, and registration aligns images— “Presently, various DL applications are widespread around the world. These applications include healthcare, social network analysis, audio and speech processing (like recognition and enhancement), visual data processing methods (such as multimedia data analysis and computer vision), and NLP (translation and sentence classification), among… | senses[13] |
| sense | Hardware specialization - CPU favors memory and generality, GPU favors parallel DNN training, and FPGA favors configurable low-latency inference— “The well-behaved performance of the CPU nodes usually assists robust network connectivity, storage abilities, and large memory. Although CPU nodes are more common-purpose than those of FPGA or GPU, they lack the ability to match them in unprocessed computation facilities, since this requires increased network ability and a larger memory… | senses[14] |
| sense | Metric-role distinction - evaluation metrics guide classifier selection during training and assess efficiency on hidden data during testing— “Accuracy: Calculates the ratio of correct predicted classes to the total number of samples evaluated (Eq. 20). Sensitivity or Recall: Utilized to calculate the fraction of positive patterns that are correctly classified (Eq. 21). Specificity: Utilized to calculate the fraction of negative patterns that are correctly classified (Eq. 22).… | senses[15] |
| control | a declared corpus for the review - a stated floor on papers read and their distribution by year - so that 'we focused on recent work' is a number rather than a claim — held— “We have reviewed more than 300 papers on various DL topics. There are 108 papers from the year 2020, 76 papers from the year 2019, and 48 papers from the year 2018. This indicates that this review focused on the latest publications in the field of DL.” — rules out a review's coverage claim resting on the reviewers' sense of having read enough— “We have reviewed more than 300 papers on various DL topics. There are 108 papers from the year 2020, 76 papers from the year 2019, and 48 papers from the year 2018. This indicates that this review focused on the latest publications in the field of DL.” | controls[0] |
| control | declared review objectives - list and define approaches, list architectures, present challenges, assess applications and computational approaches - fixed in advance of the reading — held — rules out a review whose scope is settled after the reading, by fixing the five objectives in advance— “The selected papers were analyzed and reviewed to (1) list and define the DL approaches and network types, (2) list and explain CNN architectures, (3) present the challenges of DL and suggest the alternate solutions, (4) assess the applications of DL, (5) assess computational approaches.” | controls[1] |
| control | cross-domain transfer learning checked against models trained from scratch - the control that decides whether transfer contributed anything, run by others and reported here — absent-stated-about-other— “It has further been proven that TL from different domains does not significantly affect performance on medical imaging tasks, as lightweight models trained from scratch perform nearly as well as standard ImageNet-transferred models. Therefore, there exists scenarios in which using pre-trained models do not become an affordable solution.” — rules out that the reported performance is owed to the pre-trained source domain rather than to the architecture and the target data - models trained from scratch are the comparison that decides it— “It has further been proven that TL from different domains does not significantly affect performance on medical imaging tasks, as lightweight models trained from scratch perform nearly as well as standard ImageNet-transferred models. Therefore, there exists scenarios in which using pre-trained models do not become an affordable solution.” | controls[2] |
| weight | existing DL review papers, each of which addresses a single application or topic and so leaves an overall lack of knowledge about the field — 0.9— “Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it.” | claim_relations[0] |
| weight | the 2020 Google identification of underspecification as a distinct failure mode of ML pipelines — 0.9— “In 2020, a team of computer scientists at Google has identified a new challenge called underspecification. ML models including DL models often show surprisingly poor behavior when they are tested in real-world applications such as computer vision, medical imaging, natural language processing, and medical genomics. The reason behind the weak… | claim_relations[1] |
| weight | transfer learning from ImageNet as the standard remedy for insufficient training data in medical imaging — 0.9— “It has further been proven that TL from different domains does not significantly affect performance on medical imaging tasks, as lightweight models trained from scratch perform nearly as well as standard ImageNet-transferred models. Therefore, there exists scenarios in which using pre-trained models do not become an affordable solution.” | claim_relations[2] |
| relation | DL —part-of→ ML — deep learning is a subset of machine learning— “DL, a subset of ML (Fig. 2), is inspired by the information processing patterns found in the human brain. DL does not require any human-designed rules to operate; rather, it uses a large amount of data to map the given input to specific labels.”— “DL, a subset of ML” | concept_edges[0] |
| weight | DL —part-of→ ML — 0.95— “DL, a subset of ML (Fig. 2), is inspired by the information processing patterns found in the human brain. DL does not require any human-designed rules to operate; rather, it uses a large amount of data to map the given input to specific labels.” | concept_edges[0] |
| relation | representation learning —same-as→ RL — given in apposition in the introduction— “Among the different ML algorithms, deep learning (DL) is very commonly employed in these applications. Another name for DL is representation learning (RL).”— “Another name for DL is representation learning (RL).” | concept_edges[1] |
| weight | representation learning —same-as→ RL — 0.9— “Among the different ML algorithms, deep learning (DL) is very commonly employed in these applications. Another name for DL is representation learning (RL).” | concept_edges[1] |
| relation | deep reinforcement learning —same-as→ RL — given as another name 16,670 characters later, for a different concept— “DL techniques are classified into three major categories: unsupervised, partially supervised (semi-supervised) and supervised. Furthermore, deep reinforcement learning (DRL), also known as RL, is another type of learning technique, which is mostly considered to fall into the category of partially supervised (and occasionally unsupervised)…— “deep reinforcement learning (DRL), also known as RL” | concept_edges[2] |
| weight | deep reinforcement learning —same-as→ RL — 0.9— “DL techniques are classified into three major categories: unsupervised, partially supervised (semi-supervised) and supervised. Furthermore, deep reinforcement learning (DRL), also known as RL, is another type of learning technique, which is mostly considered to fall into the category of partially supervised (and occasionally unsupervised)… | concept_edges[2] |
| differs in (stated by the tagger, no quotation) | TL —contrasted-with→ lightweight models trained from scratch — performance on medical imaging tasks when source and target domains differ | concept_edges[3] |
| relation | TL —contrasted-with→ lightweight models trained from scratch — cross-domain transfer does not beat training from scratch on medical imaging— “It has further been proven that TL from different domains does not significantly affect performance on medical imaging tasks, as lightweight models trained from scratch perform nearly as well as standard ImageNet-transferred models. Therefore, there exists scenarios in which using pre-trained models do not become an affordable solution.”— “It has further been proven that TL from different domains does not significantly affect performance on medical imaging tasks, as lightweight models trained from scratch perform nearly as well as standard ImageNet-transferred models.” | concept_edges[3] |
| weight | TL —contrasted-with→ lightweight models trained from scratch — 0.9— “It has further been proven that TL from different domains does not significantly affect performance on medical imaging tasks, as lightweight models trained from scratch perform nearly as well as standard ImageNet-transferred models. Therefore, there exists scenarios in which using pre-trained models do not become an affordable solution.” | concept_edges[3] |
| note | RNN —contrasted-with→ conventional networks — The review explicitly contrasts recurrent use of sequential state with conventional networks.— “Unlike conventional networks, RNN uses sequential data in the network. Since the embedded structure in the sequence of the data delivers valuable information, this feature is fundamental to a range of different applications. For instance, it is important to understand the context of the sentence in order to determine the meaning of a specific… | concept_edges[4] |
| note | CNN —contrasted-with→ RNN — The paper directly compares their power and feature compatibility.— “Unlike conventional networks, RNN uses sequential data in the network. Since the embedded structure in the sequence of the data delivers valuable information, this feature is fundamental to a range of different applications. For instance, it is important to understand the context of the sentence in order to determine the meaning of a specific… | concept_edges[5] |
| note | gated recurrent units (GRUs) —part-of→ RNN category — The RNN category is explicitly said to include GRUs.— “the RNN category includes gated recurrent units (GRUs) and long short-term memory (LSTM) approaches. The main advantage of this technique is the ability to collect data or generate… | concept_edges[6] |
| note | long short-term memory (LSTM) approaches —part-of→ RNN category — The RNN category is explicitly said to include LSTM approaches. | concept_edges[7] |
| note | convolutional filters —part-of→ convolutional layer — The convolutional layer is stated to consist of a collection of filters. | concept_edges[8] |
| note | max pooling —part-of→ pooling methods — Max pooling is explicitly included among pooling methods. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[9] |
| note | global average pooling (GAP) —part-of→ pooling methods — GAP is explicitly included among pooling methods. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[10] |
| note | increasing the stride value —causes→ a feature map of lower dimensions — The lower-dimensional map is stated to result from increasing stride. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[11] |
| note | sub-sampling layers —causes→ a reduction in the network parameters — Down-sampling is stated to lead to fewer parameters. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[12] |
| note | Cross-Entropy or Softmax Loss Function —same-as→ log loss function — The review explicitly gives log loss as another name. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[13] |
| audit | Drop-Weights —contrasted-with→ Dropout — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but Drop-Weights is 'highly similar to dropout', differing in what is dropped; contrasted-with is loose. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[14] |
| note | Drop-Weights —contrasted-with→ Dropout — Weights rather than neurons are dropped; the paper calls this the only difference. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[14] |
| note | Adam —is-a→ learning strategy — The paper directly classifies Adam as a learning strategy. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[15] |
| note | ResNet —contrasted-with→ highway network — The review contrasts ResNet's always-open parameter-free shortcuts with highway gates. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[16] |
| note | High-Resolution Network (HRNet) —same-as→ HRNet — The long name is explicitly followed by the short name. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[17] |
| audit | catastrophic forgetting —operationalised-as→ incorporating new information into a plain DL model — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'This' = catastrophic forgetting (the heading immediately before); the definition is stated. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[18] |
| note | catastrophic forgetting —operationalised-as→ incorporating new information into a plain DL model — The paper gives an explicit definition. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[18] |
| relation | catastrophic forgetting —operationalised-as→ incorporating new information into a plain DL model — Catastrophic forgetting This is defined as incorporating new information into a plain DL model, made possible by interfering with the learned information Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[18] |
| note | score of confidence —operationalised-as→ how confident the model is in its prediction — The paper explicitly defines the confidence score. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[19] |
| note | FPGA —contrasted-with→ GPU — The processors are directly compared on floating-point performance and configuration. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[20] |
| note | J Score —same-as→ Youdens J statistic — The paper explicitly states the alternate metric name. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concept_edges[21] |
| outcome | control — held Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | controls[0] |
| outcome (stated by the tagger, no quotation) | control — held | controls[1] |
| outcome | control — absent-stated-about-other Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | controls[2] |
| relation | CNN → visual cortex — Convolutional neural networks In the field of DL, the CNN is the most famous and commonly employed algorithm. The main benefit of CNN compared to its predecessors is that it automatically identifies the relevant features without any human supervision. CNNs have been extensively applied in a range of different fields, including computer vision, speech processing, Face Recognition, etc. The structure of CNNs was inspired by neurons in human and animal brains, similar to a conventional neural network. More specifically, in a cat’s brain, a complex sequence of cells forms the visual cortex; this sequence is simulated by the CNN. Goodfellow et al. identified three key benefits of the CNN: equivalent representations, sparse interactions, and parameter sharing. Unlike conventional fully connected (FC) networks, shared weights and local connections in the CNN are employed to make full use of 2D input-data structures like image signals. This operation utilizes an extremely small number of parameters, which both simplifies the training process and speeds up the network. This is the same as in the visual cortex cells. Notably, only small regions of a scene are sensed by these cells rather than the whole scene (i.e., these cells spatially extract the local correlation available in the input, like local filters over the input). An example of CNN architecture for image classification A commonly used type of CNN, which is similar to the multi-layer perceptron (MLP), consists of numerous convolution layers preceding sub-sampling (pooling) layers, while the ending layers are FC layers. An example of CNN architecture for image classification is illustrated in Fig. 7. The input x of each layer in a CNN model is organized in three dimensions: height, width, and depth, or , where the height (m) is equal to the width. The depth is also referred to as the channel number Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[0] |
| weight | CNN → visual cortex — 0.9 Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[0] |
| relation | LSTM → exploding gradient and vanishing problems — Recurrent neural networks Typical unfolded RNN diagram RNNs are a commonly employed and familiar algorithm in the discipline of DL. RNN is mainly applied in the area of speech processing and NLP contexts. Unlike conventional networks, RNN uses sequential data in the network. Since the embedded structure in the sequence of the data delivers valuable information, this feature is fundamental to a range of different applications. For instance, it is important to understand the context of the sentence in order to determine the meaning of a specific word in it. Thus, it is possible to consider the RNN as a unit of short-term memory, where x represents the input layer, y is the output layer, and s represents the state (hidden) layer. For a given input sequence, a typical unfolded RNN diagram is illustrated in Fig. 6. Pascanu et al. introduced three different types of deep RNN techniques, namely “Hidden-to-Hidden”, “Hidden-to-Output”, and “Input-to-Hidden”. A deep RNN is introduced that lessens the learning difficulty in the deep network and brings the benefits of a deeper RNN based on these three techniques. However, RNN’s sensitivity to the exploding gradient and vanishing problems represent one of the main issues with this approach. More specifically, during the training process, the reduplications of several large or small derivatives may cause the gradients to exponentially explode or decay. With the entrance of new inputs, the network stops thinking about the initial ones; therefore, this sensitivity decays over time. Furthermore, this issue can be handled using LSTM. This approach offers recurrent connections to memory blocks in the network. Every memory block contains a number of memory cells, which have the ability to store the temporal states of the network. In addition, it contains gated units for controlling the flow of information. In very deep networks, residual connections also have the ability to considerably reduce the impact of the vanishing gradient issue which explained in later sections. CNN is considered to be more powerful than RNN. RNN includes less feature compatibility when compared to CNN Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[1] |
| relation | parameter pruning → model compression — Model compression To obtain well-trained models that can still be employed productively, DL models have intensive memory and computational requirements due to their huge complexity and large numbers of parameters. One of the fields that is characterized as data-intensive is the field of healthcare and environmental science. These needs reduce the deployment of DL in limited computational-power machines, mainly in the healthcare field. The numerous methods of assessing human health and the data heterogeneity have become far more complicated and vastly larger in size; thus, the issue requires additional computation. Furthermore, novel hardware-based parallel processing solutions such as FPGAs and GPUs have been developed to solve the computation issues associated with DL. Recently, numerous techniques for compressing the DL models, designed to decrease the computational issues of the models from the starting point, have also been introduced. These techniques can be classified into four classes. In the first class, the redundant parameters (which have no significant impact on model performance) are reduced. This class, which includes the famous deep compression method, is called parameter pruning. In the second class, the larger model uses its distilled knowledge to train a more compact model; thus, it is called knowledge distillation. In the third class, compact convolution filters are used to reduce the number of parameters. In the final class, the information parameters are estimated for preservation using low-rank factorization. For model compression, these classes represent the most representative techniques. In, it has been provided a more comprehensive discussion about the topic Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[2] |
| relation | knowledge distillation → model compression — Model compression To obtain well-trained models that can still be employed productively, DL models have intensive memory and computational requirements due to their huge complexity and large numbers of parameters. One of the fields that is characterized as data-intensive is the field of healthcare and environmental science. These needs reduce the deployment of DL in limited computational-power machines, mainly in the healthcare field. The numerous methods of assessing human health and the data heterogeneity have become far more complicated and vastly larger in size; thus, the issue requires additional computation. Furthermore, novel hardware-based parallel processing solutions such as FPGAs and GPUs have been developed to solve the computation issues associated with DL. Recently, numerous techniques for compressing the DL models, designed to decrease the computational issues of the models from the starting point, have also been introduced. These techniques can be classified into four classes. In the first class, the redundant parameters (which have no significant impact on model performance) are reduced. This class, which includes the famous deep compression method, is called parameter pruning. In the second class, the larger model uses its distilled knowledge to train a more compact model; thus, it is called knowledge distillation. In the third class, compact convolution filters are used to reduce the number of parameters. In the final class, the information parameters are estimated for preservation using low-rank factorization. For model compression, these classes represent the most representative techniques. In, it has been provided a more comprehensive discussion about the topic Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[3] |
| relation | registration → Target Selection — Registration Target Selection: it illustrates the determined input image that the second counterpart input image needs to remain accurately superimposed to. Feature Extraction: it computes the set of features extracted from each input image. Feature Matching: it allows finding similarities between the previously obtained features. Pose Optimization: it is aimed to minimize the distance between both input images Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[4] |
| relation | registration → Pose Optimization — Registration Target Selection: it illustrates the determined input image that the second counterpart input image needs to remain accurately superimposed to. Feature Extraction: it computes the set of features extracted from each input image. Feature Matching: it allows finding similarities between the previously obtained features. Pose Optimization: it is aimed to minimize the distance between both input images Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[5] |
| relation | GPU → DNN training — CPU-based approach The well-behaved performance of the CPU nodes usually assists robust network connectivity, storage abilities, and large memory. Although CPU nodes are more common-purpose than those of FPGA or GPU, they lack the ability to match them in unprocessed computation facilities, since this requires increased network ability and a larger memory capacity. ### GPU-based approach GPUs are extremely effective for several basic DL primitives, which include greatly parallel-computing operations such as activation functions, matrix multiplication, and convolutions. Incorporating HBM-stacked memory into the up-to-date GPU models significantly enhances the bandwidth. This enhancement allows numerous primitives to efficiently utilize all computational resources of the available GPUs. The improvement in GPU performance over CPU performance is usually 10-20:1 related to dense linear algebra operations. Maximizing parallel processing is the base of the initial GPU programming model. For example, a GPU model may involve up to sixty-four computational units. There are four SIMD engines per each computational layer, and each SIMD has sixteen floating-point computation lanes. The peak performance is 25 TFLOPS (fp16) and 10 TFLOPS (fp32) as the percentage of the employment approaches 100%. Additional GPU performance may be achieved if the addition and multiply functions for vectors combine the inner production instructions for matching primitives related to matrix operations. For DNN training, the GPU is usually considered to be an optimized design, while for inference operations, it may also offer considerable performance improvements Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[6] |
| relation | FPGA → inference accelerators — FPGA-based approach FPGA is wildly utilized in various tasks including deep learning. Inference accelerators are commonly implemented utilizing FPGA. The FPGA can be effectively configured to reduce the unnecessary or overhead functions involved in GPU systems. Compared to GPU, the FPGA is restricted to both weak-behaved floating-point performance and integer inference. The main FPGA aspect is the capability to dynamically reconfigure the array characteristics (at run-time), as well as the capability to configure the array by means of effective design with little or no overhead. As mentioned earlier, the FPGA offers both performance and latency for every watt it gains over GPU and CPU in DL inference operations. Implementation of custom high-performance hardware, pruned networks, and reduced arithmetic precision are three factors that enable the FPGA to implement DL algorithms and to achieve FPGA with this level of efficiency. In addition, FPGA may be employed to implement CNN overlay engines with over 80% efficiency, eight-bit accuracy, and over 15 TOPs peak performance; this is used for a few conventional CNNs, as Xillinx and partners demonstrated recently. By contrast, pruning techniques are mostly employed in the LSTM context. The sizes of the models can be efficiently minimized by up to 20×, which provides an important benefit during the implementation of the optimal solution, as MLP neural processing demonstrated. A recent study in the field of implementing fixed-point precision and custom floating-point has revealed that lowering the 8-bit is extremely promising; moreover, it aids in supplying additional advancements to implementing peak performance FPGA related to the DNN models Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[7] |
| relation | Sensitivity → Recall — Evaluation metrics Accuracy: Calculates the ratio of correct predicted classes to the total number of samples evaluated (Eq. 20). Sensitivity or Recall: Utilized to calculate the fraction of positive patterns that are correctly classified (Eq. 21). Specificity: Utilized to calculate the fraction of negative patterns that are correctly classified (Eq. 22). Precision: Utilized to calculate the positive patterns that are correctly predicted by all predicted patterns in a positive class (Eq. 23). F1-Score: Calculates the harmonic average between recall and precision rates (Eq. 24). J Score: This metric is also called Youdens J statistic. Eq. 25 represents the metric. False Positive Rate (FPR): This metric refers to the possibility of a false alarm ratio as calculated in Eq. 26 Area Under the ROC Curve: AUC is a common ranking type metric. It is utilized to conduct comparisons between learning algorithms, as well as to construct an optimal learning model. In contrast to probability and threshold metrics, the AUC value exposes the entire classifier ranking performance. The following formula is used to calculate the AUC value for two-class problem (Eq. 27) Here, represents the sum of all positive ranked samples. The number of negative and positive samples is denoted as and , respectively. Compared to the accuracy metrics, the AUC value was verified empirically and theoretically, making it very helpful for identifying an optimized solution and evaluating the classifier performance through classification training. When considering the discrimination and evaluation processes, the AUC performance was brilliant. However, for multiclass issues, the AUC computation is primarily cost-effective when discriminating a large number of created solutions. In addition, the time complexity for computing the AUC is with respect to the Hand and Till AUC model and according to Provost and Domingo’s AUC model. Evaluation metrics adopted within DL tasks play a crucial role in achieving the optimized classifier. They are utilized within a usual data classification procedure through two main stages: training and testing. It is utilized to optimize the classification algorithm during the training stage. This means that the evaluation metric is utilized to discriminate and select the optimized solution, e.g., as a discriminator, which can generate an extra-accurate forecast of upcoming evaluations related to a specific classifier. For the time being, the evaluation metric is utilized to measure the efficiency of the created classifier, e.g. as an evaluator, within the model testing stage using hidden data. As given in Eq. 20, TN and TP are defined as the number of negative and positive instances, respectively, which are successfully classified. In addition, FN and FP are defined as the number of misclassified positive and negative instances respectively. Next, some of the most well-known evaluation metrics are listed below Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[8] |
| relation | CapsuleNet → CapsNet — CapsuleNet CNN is an efficient technique for detecting object features and achieving well-behaved recognition performance in comparison with innovative handcrafted feature detectors. A number of restrictions related to CNN are present, meaning that the CNN does not consider certain relations, orientation, size, and perspectives of features. For instance, when considering a face image, the CNN does not count the various face components (such as mouth, eyes, nose, etc.) positions, and will incorrectly activate the CNN neurons and recognize the face without taking specific relations (such as size, orientation etc.) into account. At this point, consider a neuron that has probability in addition to feature properties such as size, orientation, perspective, etc. A specific neuron/capsule of this type has the ability to effectively detect the face along with different types of information. Thus, many layers of capsule nodes are used to construct the capsule network. An encoding unit, which contains three layers of capsule nodes, forms the CapsuleNet or CapsNet (the initial version of the capsule networks). For example, the MNIST architecture comprises images, applying 256 filters of size and with stride 1. The is the output plus 256 feature maps. Next, these outputs are input to the first capsule layer, while producing an 8D vector rather than a scalar; in fact, this is a modified convolution layer. Note that a stride 2 with filters is employed in the first convolution layer. Thus, the dimension of the output is . The initial capsules employ filters, which generate 32 × 8 × 6 × 6 (32 for groups, 8 for neurons, while 6 × 6 is the neuron size). The complete CapsNet encoding and decoding processes Figure 25 represents the complete CapsNet encoding and decoding processes. In the CNN context, a max-pooling layer is frequently employed to handle the translation change. It can detect the feature moves in the event that the feature is still within the max-pooling window. This approach has the ability to detect the overlapped features; this is highly significant in detection and segmentation operations, since the capsule involves the weighted features sum from the preceding layer. In conventional CNNs, a particular cost function is employed to evaluate the global error that grows toward the back throughout the training process. Conversely, in such cases, the activation of a neuron will not grow further once the weight between two neurons turns out to be zero. Instead of a single size being provided with the complete cost function in repetitive dynamic routing alongside the agreement, the signal is directed based on the feature parameters. Sabour et al. provides more details about this architecture. When using MNIST to recognize handwritten digits, this innovative CNN architecture gives superior accuracy. From the application perspective, this architecture has extra suitability for segmentation and detection approaches when compared with classification approaches Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[9] |
| no type for this | CNN → visual cortex — CNN structure is described as inspired by visual-cortex organization; inspired-by is absent from the eight governed types. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[0] |
| no type for this | LSTM → exploding gradient and vanishing problems — LSTM is presented as handling the recurrent network's exploding/vanishing-gradient issue; mitigates is absent from the eight governed types. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[1] |
| no type for this | parameter pruning → model compression — Parameter pruning is one of four methods used for model compression; method-for is absent, and the prose does not support forcing an ontological part-of edge. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[2] |
| no type for this | knowledge distillation → model compression — Knowledge distillation trains a compact model from a larger model for compression; method-for is absent from the governed types. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[3] |
| no type for this | registration → Target Selection — Target Selection is the first named stage of image registration; stage-of is absent from the eight governed types. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[4] |
| no type for this | registration → Pose Optimization — Pose Optimization is the distance-minimization stage of image registration; stage-of is absent from the eight governed types. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[5] |
| no type for this | GPU → DNN training — The GPU is described as optimized for DNN training; optimized-for is absent from the eight governed types. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[6] |
| no type for this | FPGA → inference accelerators — FPGAs are commonly used to implement inference accelerators; used-for is absent from the governed relation types. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[7] |
| no type for this | Sensitivity → Recall — Sensitivity and Recall are presented as alternate names for one metric; the slash/or naming construction lacks a governed same-as cue, so the alias is filed unhoused. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[8] |
| no type for this | CapsuleNet → CapsNet — CapsNet is the short form used for CapsuleNet; abbreviation expansion is absent from the eight governed types. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | unhoused_relations[9] |
| limitation | the paper recommends transfer learning for scarce data and then reports that cross-domain TL does not significantly affect medical imaging performance - its own central recommendation carries a negative result three paragraphs later Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[0] |
| limitation | `RL` is defined as representation learning in the introduction and used for reinforcement learning in the classification section; a reader carrying the first definition forward misreads the second Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[1] |
| limitation | underspecification puts major constraints on the credibility of ML predictions and the proposed remedy - stress tests - requires already knowing how the model can fail Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[2] |
| limitation | CNN performance is greatly influenced by hyper-parameter selection, so any small change affects the result - which limits how much a reported architecture comparison can be attributed to architecture Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[3] |
| limitation | labelling remains the binding constraint: crowdsourced workers cannot annotate medical or biological images, expert labelling is costly and slow, so producing the volume of labels deep networks need is called unfeasible Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[4] |
| limitation | despite the stated transition toward semi-supervised and unsupervised learning, most current deep-learning models still use supervised learning - so the data constraint above is not being relieved Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[5] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[1] |
| finding | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[2] |
| finding | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[3] |
| finding | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[4] |
| finding | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[5] |
| quantity | papers reviewed from 2020, of more than 300 total Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[0] |
| quantity | the floor on the number of papers reviewed Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[1] |
| variable | hyper-parameter selection Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.variables[0] |
| variable | amount of data Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.variables[1] |
| definition | AUC Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[0] |
| definition | DL Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[1] |
| definition | Sparse Connectivity Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[2] |
| definition | Weight Sharing Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[3] |
| definition | Dropout Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[4] |
| definition | catastrophic forgetting Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[5] |
| definition | underspecification Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[6] |
| definition | Same-domain TL Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[7] |
| concept | representation learning — not defined in this paper's own words; given here as another name for deep learning - and abbreviated RL, which this paper also uses for reinforcement learning Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[0] |
| concept | RL — not defined in this paper's own words; COLLIDES IN THIS PAPER: introduced as representation learning, then used for reinforcement learning. Two unrelated concepts under one abbreviation, neither definition aware of the other Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[1] |
| concept | deep reinforcement learning — not defined in this paper's own words; learning by interaction with an environment; classified here as partially supervised, and given the same abbreviation as representation learning Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[2] |
| concept | DL — not defined in this paper's own words; a subset of machine learning that maps input to labels without human-designed rules Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[3] |
| concept | ML — not defined in this paper's own words; the containing field, from which DL is distinguished by learning its own features Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[4] |
| concept | TL — not defined in this paper's own words; transfer learning - pre-train on a large dataset, fine-tune on a small one. Recommended here and also reported not to help when the domains differ Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[5] |
| concept | Same-domain TL — not defined in this paper's own words; using source images that look like the target, introduced precisely because cross-domain transfer was found not to help Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[6] |
| concept | underspecification — not defined in this paper's own words; a 2020 Google finding that models passing identical validation can behave very differently in deployment - small changes select a completely different solution Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[7] |
| concept | stress tests — not defined in this paper's own words; the proposed remedy for underspecification, which the paper notes requires already understanding how the model can fail Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[8] |
| concept | Sparse Connectivity — not defined in this paper's own words; only a few weights between adjacent layers, unlike fully connected networks - the memory argument for CNNs Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[9] |
| concept | Weight Sharing — not defined in this paper's own words; one set of weights operating over every pixel, so training cost does not scale with neuron count Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[10] |
| concept | Dropout — not defined in this paper's own words; randomly dropping neurons each epoch so feature-selection power is distributed rather than concentrated Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[11] |
| concept | Dying ReLU — not defined in this paper's own words; a large gradient through ReLU updating weights so the neuron never activates again Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[12] |
| concept | catastrophic forgetting — not defined in this paper's own words; fine-tuning on a new class destroying performance on the old ones - the model cannot add without subtracting Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[13] |
| concept | vanishing gradient problem — not defined in this paper's own words; a gradient so small that weight updating stops, so the earliest layers - the ones recognising the essential input elements - never train Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[14] |
| concept | overfitting — not defined in this paper's own words; the model performing well on training data and failing on unseen data, made likely by the sheer parameter count Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[15] |
| concept | black box — not defined in this paper's own words; the interpretability charge against DL, which the paper answers by asserting the techniques ARE interpretable Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[16] |
| concept | imbalanced — not defined in this paper's own words; far more negative than positive samples - the default condition of biological data, and a source of undesirable results Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[17] |
| concept | hyper-parameter selection — not defined in this paper's own words; the choice the conclusion says dominates CNN performance - any small change affects the result Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[18] |
| concept | lightweight models trained from scratch — not defined in this paper's own words; the baseline that performs nearly as well as ImageNet-transferred models on medical imaging, and so the control that undercuts cross-domain transfer Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[19] |
| concept | feature engineering — not defined in this paper's own words; manual construction of task-specific input features Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[20] |
| concept | histogram of oriented gradients (HOG) — not defined in this paper's own words; hand-engineered computer-vision feature Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[21] |
| concept | scale-invariant feature transform (SIFT) — not defined in this paper's own words; hand-engineered scale-invariant feature Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[22] |
| concept | bag of words (BoW) — not defined in this paper's own words; hand-engineered representation used in computer vision Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[23] |
| concept | feature extraction — not defined in this paper's own words; automatic extraction of discriminative representations in DL Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[24] |
| concept | multi-layer data representation architecture — not defined in this paper's own words; hierarchy from low- to high-level features Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[25] |
| concept | low-level features — not defined in this paper's own words; features extracted in early layers Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[26] |
| concept | high-level features — not defined in this paper's own words; features extracted in late layers Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[27] |
| concept | unsupervised — not defined in this paper's own words; major DL supervision category without labels Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[28] |
| concept | partially supervised (semi-supervised) — not defined in this paper's own words; major DL supervision category using partial labels Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[29] |
| concept | supervised — not defined in this paper's own words; major DL supervision category using labeled data Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[30] |
| concept | labeled data — not defined in this paper's own words; input-output supervision used in deep supervised learning Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[31] |
| concept | recurrent neural networks (RNNs) — not defined in this paper's own words; supervised sequence-network family Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[32] |
| concept | convolutional neural networks (CNNs) — not defined in this paper's own words; supervised convolutional-network family Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[33] |
| concept | deep neural networks (DNNs) — not defined in this paper's own words; supervised deep-network family Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[34] |
| concept | gated recurrent units (GRUs) — not defined in this paper's own words; gated member of the recurrent-network family Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[35] |
| concept | long short-term memory (LSTM) approaches — not defined in this paper's own words; memory-gated recurrent-network approaches Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[36] |
| concept | decision boundary — not defined in this paper's own words; class boundary that may be overstrained by missing samples Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[37] |
| concept | generative networks — not defined in this paper's own words; unsupervised network family Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[38] |
| concept | dimensionality reduction — not defined in this paper's own words; unsupervised representation-reduction task Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[39] |
| concept | clustering — not defined in this paper's own words; unsupervised discovery of group structure Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[40] |
| concept | restricted Boltzmann machines — not defined in this paper's own words; unsupervised nonlinear representation model Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[41] |
| concept | auto-encoders — not defined in this paper's own words; unsupervised reconstruction-based representation models Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[42] |
| concept | GANs — not defined in this paper's own words; generative adversarial networks used in unsupervised learning Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[43] |
| concept | reward function — not defined in this paper's own words; feedback supplied to the reinforcement-learning agent Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[44] |
| concept | environment — not defined in this paper's own words; interactive source of state and reward in reinforcement learning Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[45] |
| concept | supervised learning — not defined in this paper's own words; learning from provided samples rather than interaction Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[46] |
| concept | RNN — not defined in this paper's own words; network whose hidden state processes sequential data Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[47] |
| concept | sequential data — not defined in this paper's own words; ordered inputs exploited by recurrent networks Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[48] |
| concept | short-term memory — not defined in this paper's own words; interpretation of recurrent hidden state Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[49] |
| concept | state (hidden) layer — not defined in this paper's own words; recurrent state carrying sequence context Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[50] |
| concept | exploding gradient — not defined in this paper's own words; training instability from repeatedly multiplied large derivatives Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[51] |
| concept | LSTM — not defined in this paper's own words; recurrent architecture using gated memory blocks Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[52] |
| concept | memory blocks — not defined in this paper's own words; LSTM components containing temporal memory cells Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[53] |
| concept | memory cells — not defined in this paper's own words; units storing temporal network states Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[54] |
| concept | gated units — not defined in this paper's own words; controllers of information flow in an LSTM memory block Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[55] |
| concept | conventional networks — not defined in this paper's own words; non-recurrent networks contrasted with RNNs Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[56] |
| concept | CNN — not defined in this paper's own words; convolutional network compared with recurrent networks Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[57] |
| concept | equivalent representations — not defined in this paper's own words; one of the review's three key CNN benefits Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[58] |
| concept | sparse interactions — not defined in this paper's own words; local rather than fully connected CNN interactions Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[59] |
| concept | parameter sharing — not defined in this paper's own words; reuse of convolution weights across spatial locations Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[60] |
| concept | shared weights — not defined in this paper's own words; weights reused over a two-dimensional input Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[61] |
| concept | local connections — not defined in this paper's own words; connections restricted to local input regions Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[62] |
| concept | 2D input-data structures — not defined in this paper's own words; image-like structure exploited by CNNs Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[63] |
| concept | convolution layers — not defined in this paper's own words; feature-extraction layers in a CNN Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[64] |
| concept | sub-sampling (pooling) layers — not defined in this paper's own words; down-sampling layers following convolutions Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[65] |
| concept | FC layers — not defined in this paper's own words; fully connected ending layers of a CNN Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[66] |
| concept | feature maps — not defined in this paper's own words; spatial representations produced by convolution Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[67] |
| concept | softmax — not defined in this paper's own words; ending classifier producing class scores Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[68] |
| concept | support vector machines (SVMs) — not defined in this paper's own words; alternative ending classifier named by the review Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[69] |
| concept | kernel weight — not defined in this paper's own words; learned discrete value in a convolution kernel Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[70] |
| concept | convolutional operation — not defined in this paper's own words; sliding dot-product computation over the image Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[71] |
| concept | feature map — not defined in this paper's own words; output grid of convolution dot products Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[72] |
| concept | stride — not defined in this paper's own words; vertical or horizontal kernel step size Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[73] |
| concept | padding — not defined in this paper's own words; input-border expansion retaining border information Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[74] |
| concept | convolutional layer — not defined in this paper's own words; CNN layer containing convolutional filters Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[75] |
| concept | convolutional filters — not defined in this paper's own words; kernels generating output feature maps Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[76] |
| concept | pooling layer — not defined in this paper's own words; layer that subsamples feature maps Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[77] |
| concept | pooling methods — not defined in this paper's own words; family of feature-map subsampling operations Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[78] |
| concept | max pooling — not defined in this paper's own words; pooling by maximum local activation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[79] |
| concept | min pooling — not defined in this paper's own words; pooling by minimum local activation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[80] |
| concept | global average pooling (GAP) — not defined in this paper's own words; pooling by global channel average Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[81] |
| concept | global max pooling — not defined in this paper's own words; pooling by global maximum activation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[82] |
| concept | activation function — not defined in this paper's own words; nonlinear mapping determining neuron output Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[83] |
| concept | Sigmoid — not defined in this paper's own words; S-shaped activation with output between zero and one Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[84] |
| concept | Tanh — not defined in this paper's own words; activation with output between minus one and one Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[85] |
| concept | ReLU — not defined in this paper's own words; rectifier activation with comparatively low computation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[86] |
| concept | Leaky ReLU — not defined in this paper's own words; rectifier retaining small negative inputs Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[87] |
| concept | Noisy ReLU — not defined in this paper's own words; rectifier perturbed with Gaussian noise Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[88] |
| concept | Parametric Linear Units — not defined in this paper's own words; rectifier with learned leak factor Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[89] |
| concept | Fully Connected Layer — not defined in this paper's own words; ending CNN classifier layer Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[90] |
| concept | loss functions — not defined in this paper's own words; training objectives measuring prediction error Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[91] |
| concept | Cross-Entropy or Softmax Loss Function — not defined in this paper's own words; multiclass probability loss Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[92] |
| concept | log loss function — not defined in this paper's own words; alternate name for cross-entropy loss Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[93] |
| concept | Euclidean Loss Function — not defined in this paper's own words; regression loss based on squared error Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[94] |
| concept | mean square error — not defined in this paper's own words; alternate name for Euclidean loss Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[95] |
| concept | Hinge Loss Function — not defined in this paper's own words; maximum-margin binary-classification loss Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[96] |
| concept | Drop-Weights — not defined in this paper's own words; regularizer dropping connections rather than neurons Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[97] |
| concept | Data Augmentation — not defined in this paper's own words; regularizer expanding the training set Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[98] |
| concept | Batch Normalization — not defined in this paper's own words; layerwise activation normalization Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[99] |
| concept | internal covariance shift — not defined in this paper's own words; training-time change in activation distributions Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[100] |
| concept | over-fitted — not defined in this paper's own words; model state with strong training and poor test performance Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[101] |
| concept | under-fitted model — not defined in this paper's own words; model that learns too little from training data Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[102] |
| concept | just-fitted — not defined in this paper's own words; model state performing well on training and test data Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[103] |
| concept | learning rate — not defined in this paper's own words; step size used in parameter updates Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[104] |
| concept | training epoch — not defined in this paper's own words; one complete pass of parameter updates over the training set Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[105] |
| concept | Batch Gradient Descent — not defined in this paper's own words; one parameter update after the full training set Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[106] |
| concept | Stochastic Gradient Descent — not defined in this paper's own words; parameter updating at every training sample Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[107] |
| concept | Mini-batch Gradient Descent — not defined in this paper's own words; parameter updating after each non-overlapping mini-batch Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[108] |
| concept | Adaptive Moment Estimation (Adam) — not defined in this paper's own words; adaptive per-parameter learning-rate optimizer Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[109] |
| concept | Adam — not defined in this paper's own words; short name used for Adaptive Moment Estimation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[110] |
| concept | learning strategy — not defined in this paper's own words; class to which the review assigns Adam Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[111] |
| concept | residual links — not defined in this paper's own words; shortcut connections that accelerate deep-network convergence Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[112] |
| concept | deep network convergence — not defined in this paper's own words; training progress accelerated by residual links Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[113] |
| concept | highway network — not defined in this paper's own words; gated-shortcut architecture contrasted with ResNet Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[114] |
| concept | ResNet — not defined in this paper's own words; residual architecture with always-open identity shortcuts Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[115] |
| concept | DenseNet — not defined in this paper's own words; densely cross-connected feed-forward architecture Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[116] |
| concept | CapsuleNet — not defined in this paper's own words; capsule-node architecture retaining feature properties Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[117] |
| concept | CapsNet — not defined in this paper's own words; short name for CapsuleNet Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[118] |
| concept | High-Resolution Network (HRNet) — not defined in this paper's own words; parallel multi-resolution vision backbone Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[119] |
| concept | HRNet — not defined in this paper's own words; short name for the High-Resolution Network Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[120] |
| concept | training data — not defined in this paper's own words; examples used to fit a deep-learning model Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[121] |
| concept | simulated data — not defined in this paper's own words; synthetic source used to enlarge a training set Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[122] |
| concept | data augmentation techniques — not defined in this paper's own words; data-space methods increasing limited training data Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[123] |
| concept | data-space solutions — not defined in this paper's own words; class assigned to augmentation techniques Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[124] |
| concept | model interpretability — not defined in this paper's own words; ability to attribute predictions to input portions Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[125] |
| concept | perturbation-based approaches — not defined in this paper's own words; interpretability methods changing input portions Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[126] |
| concept | back-propagation-based techniques — not defined in this paper's own words; interpretability methods propagating output signals backward Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[127] |
| concept | score of confidence — not defined in this paper's own words; model-reported certainty in a prediction Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[128] |
| concept | temperature scaling — not defined in this paper's own words; post-scaling technique for reliable probabilities Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[129] |
| concept | incorporating new information into a plain DL model — not defined in this paper's own words; operation whose interference defines catastrophic forgetting Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[130] |
| concept | parameter pruning — not defined in this paper's own words; compression by removing redundant parameters Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[131] |
| concept | knowledge distillation — not defined in this paper's own words; compression by training a compact model from a larger model Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[132] |
| concept | compact convolution filters — not defined in this paper's own words; compression by parameter-efficient filters Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[133] |
| concept | low-rank factorization — not defined in this paper's own words; compression by low-rank parameter approximation Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[134] |
| concept | lack of training data — not defined in this paper's own words; source of distribution mismatch and overfitting Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[135] |
| concept | stress tests — not defined in this paper's own words; deployment-focused tests proposed for underspecification Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[136] |
| concept | classification — not defined in this paper's own words; categorization of data into classes Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[137] |
| concept | localization — not defined in this paper's own words; locating one object with one bounding box Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[138] |
| concept | detection — not defined in this paper's own words; locating multiple potentially different-class objects Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[139] |
| concept | segmentation — not defined in this paper's own words; outlining and labeling target-object edges Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[140] |
| concept | registration — not defined in this paper's own words; fitting one image onto another Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[141] |
| concept | Target Selection — not defined in this paper's own words; registration stage selecting the fixed counterpart image Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[142] |
| concept | Feature Extraction — not defined in this paper's own words; registration stage computing image features Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[143] |
| concept | Feature Matching — not defined in this paper's own words; registration stage finding feature similarities Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[144] |
| concept | Pose Optimization — not defined in this paper's own words; registration stage minimizing distance between images Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[145] |
| concept | CPU — not defined in this paper's own words; general-purpose processor with memory and storage strengths Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[146] |
| concept | GPU — not defined in this paper's own words; parallel processor optimized for DNN training primitives Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[147] |
| concept | FPGA — not defined in this paper's own words; reconfigurable processor commonly used for inference acceleration Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[148] |
| concept | Accuracy — not defined in this paper's own words; ratio of correctly predicted classes Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[149] |
| concept | Sensitivity or Recall — not defined in this paper's own words; fraction of positive patterns correctly classified Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[150] |
| concept | Specificity — not defined in this paper's own words; fraction of negative patterns correctly classified Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[151] |
| concept | Precision — not defined in this paper's own words; fraction of predicted positives that are correct Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[152] |
| concept | F1-Score — not defined in this paper's own words; harmonic average of recall and precision Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[153] |
| concept | J Score — not defined in this paper's own words; metric also named Youdens J statistic Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[154] |
| concept | Youdens J statistic — not defined in this paper's own words; alternate name for J Score Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[155] |
| concept | Area Under the ROC Curve — not defined in this paper's own words; ranking metric evaluating classifier performance Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[156] |
| concept | RNN category — not defined in this paper's own words; category explicitly including GRUs and LSTM approaches Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[157] |
| concept | increasing the stride value — not defined in this paper's own words; larger convolution step size Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[158] |
| concept | a feature map of lower dimensions — not defined in this paper's own words; output effect of increasing stride Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[159] |
| concept | sub-sampling layers — not defined in this paper's own words; layers that down-sample feature maps Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[160] |
| concept | a reduction in the network parameters — not defined in this paper's own words; parameter-count effect of down-sampling Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[161] |
| concept | how confident the model is in its prediction — not defined in this paper's own words; paper's definition of confidence score Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[162] |
| concept | overfitting problem — not defined in this paper's own words; generalization failure linked to too little training data Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[163] |
| concept | visual cortex — not defined in this paper's own words; biological organization said to inspire CNN structure Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[164] |
| concept | exploding gradient and vanishing problems — not defined in this paper's own words; paired recurrent-network training instabilities Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[165] |
| concept | model compression — not defined in this paper's own words; reduction of model computation and memory cost Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[166] |
| concept | DNN training — not defined in this paper's own words; training workload for deep neural networks Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[167] |
| concept | inference accelerators — not defined in this paper's own words; hardware accelerators for trained-model inference Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[168] |
| concept | Sensitivity — not defined in this paper's own words; positive-pattern classification rate Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[169] |
| concept | Recall — not defined in this paper's own words; alternate name paired with sensitivity Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[170] |
| AUC | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[0] |
| Sensitivity or Recall | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[1] |
| Specificity | Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[2] |
When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
Tag layer © the Meta-Tagging Project, CC BY 4.0. The paper text is © its authors/publisher
and is linked, not reproduced. Excerpts shown are short quotations for identification and verification.
· back to index
· concepts · authors
· עברית
© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824