| phenomenon | overfitting in deep neural networks— “However, overfitting is a serious problem in such networks.” |
| claim-type | method— “Dropout is a technique for addressing this problem.” |
| method — evidence | — “We trained dropout neural networks for classification problems on data sets in different domains.” |
| transfer | from evolutionary biology (theory of sexual reproduction): dropout as a neural-net regularizer inspired by the evolutionary role of sex - breaking co-adaptations, analogous to genetic recombination— “A motivation for dropout comes from a theory of the role of sex in evolution” | transfers[0] |
| mechanism | randomly drop units during training so neurons cannot co-adapt; this approximates averaging an exponential ensemble of thinned networks, regularizing the model— “The key idea is to randomly drop units (along with their connections) from the neural network during training” | mechanisms[0] |
| extends | prior regularization methods for neural networks— “This significantly reduces overfitting and gives major improvements over other regularization methods.” | claim_relations[0] |
| extends | denoising autoencoders (Vincent et al., 2008, 2010), where noise is added to the input units— “Our work extends this idea by showing that dropout can be effectively applied in the hidden layers as well and that it can be interpreted as a form of model averaging.” | claim_relations[1] |
| sense | the motivation for dropout - AN ANALOGY, FLAGGED AS ONE: a theory of the role of sex in evolution, in which combining half of each parent's genes beats copying a good genome directly; the paper presents this as a motivation and not as a derivation, and its only proof of an equivalent regularizer is for linear regression— “A motivation for dropout comes from a theory of the role of sex in evolution (Livnat et al., 2010). Sexual reproduction involves taking half the genes of one parent and half of the other, adding a very small amount of random mutation, and combining them to produce an offspring” | senses[0] |
| sense | the test-time network - A STAND-IN FOR AN ENSEMBLE THAT IS NEVER BUILT: a single unthinned network with scaled weights approximates an equally weighted geometric mean over an exponential number of thinned networks, so the rule everyone implements is a cheap approximation to a computation nobody performs— “We propose to do this by approximating an equally weighted geometric mean of the predictions of an exponential number of learned models that share parameters” | senses[1] |
| sense | a thinned network - ONE SAMPLE FROM 2^n: applying dropout amounts to sampling a thinned network, and a net with n units is described as a collection of that many possible thinned networks all sharing weights, so 'a dropout network' names a distribution over architectures rather than one architecture— “Applying dropout to a neural network amounts to sampling a “thinned” network from it” | senses[2] |
| control | the same architectures trained with and without dropout, differing only in whether a thinned network is sampled for each training case in a mini-batch — held — rules out an improvement owed to the architecture, the data or the training schedule rather than to dropout itself— “The same architectures trained with and without dropout have drastically different test errors as seen as by the two separate clusters of trajectories.” | controls[0] |
| audit | dropout —operationalised-as→ thinned — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says applying dropout AMOUNTS TO sampling a 'thinned' network; operationalised-as is loose.— “Applying dropout to a neural network amounts to sampling a “thinned” network from it” | concept_edges[0] |
| outcome (stated by the tagger, no quotation) | control — held | controls[0] |
| for which use | limitation — the cost side of adopting dropout: ONE OF THE DRAWBACKS OF DROPOUT IS THAT IT INCREASES TRAINING TIME— “One of the drawbacks of dropout is that it increases training time.” | limitations[0] |
| duplicate of | limitation — limitations[0]— “One of the drawbacks of dropout is that it increases training time.” | limitations[1] |
| for which use | limitation — the same scope as limitations[0] - training is substantially slower— “One of the drawbacks of dropout is that it increases training time.” | limitations[1] |
| for which use | limitation — small-data applications, where the benefit is measured at zero: FOR EXTREMELY SMALL DATA SETS (100, 500) DROPOUT DOES NOT GIVE ANY IMPROVEMENTS— “It can be observed that for extremely small data sets (100, 500) dropout does not give any improvements.” | limitations[2] |
| for which use | limitation — text applications specifically: on Reuters THE IMPROVEMENT WAS MUCH SMALLER COMPARED TO THAT FOR THE VISION AND SPEECH DATA SETS— “We found that the improvement was much smaller compared to that for the vision and speech data sets.” | limitations[3] |
| for which use | limitation — any theoretical account of dropout beyond linear regression: FOR MORE COMPLICATED MODELS, IT IS NOT OBVIOUS HOW TO OBTAIN AN EQUIVALENT REGULARIZER— “For more complicated models, it is not obvious how to obtain an equivalent regularizer.” | limitations[4] |
| for which use | limitation — any account of why dropout works: the authors give as motivation a theory of the role of sex in evolution, in which reproduction combines half the genes of each parent with a small amount of random mutation— “A motivation for dropout comes from a theory of the role of sex in evolution (Livnat et al., 2010). Sexual reproduction involves taking half the genes of one parent and half of the other, adding a very small amount of random mutation, and combining them to produce an offspring” | limitations[5] |
| for which use | limitation — any account of why dropout works: the authors offer a second motivation, described as closely related but slightly different, from thinking about successful conspiracies— “A closely related, but slightly different motivation for dropout comes from thinking about successful conspiracies” | limitations[6] |
| for which use | limitation — the weight-scaling rule used at test time: the authors describe it as— “We propose to do this by approximating an equally weighted geometric mean of the predictions of an exponential number of learned models that share parameters” | limitations[7] |
| replication | untested |
| limitation | dropout increases training time— “One of the drawbacks of dropout is that it increases training time.” | limitations[0] |
| limitation | Dropout makes training substantially slower.— “One of the drawbacks of dropout is that it increases training time.” | limitations[1] |
| limitation | On very small training sets dropout gives no benefit at all.— “It can be observed that for extremely small data sets (100, 500) dropout does not give any improvements.” | limitations[2] |
| limitation | The gain on the Reuters text data set was far smaller than on vision and speech data.— “We found that the improvement was much smaller compared to that for the vision and speech data sets.” | limitations[3] |
| limitation | The equivalent deterministic regularizer is only derived for linear regression; for richer models it is unknown.— “For more complicated models, it is not obvious how to obtain an equivalent regularizer.” | limitations[4] |
| limitation | The method's stated motivation is an analogy to sexual reproduction, not a derivation.— “A motivation for dropout comes from a theory of the role of sex in evolution (Livnat et al., 2010). Sexual reproduction involves taking half the genes of one parent and half of the other, adding a very small amount of random mutation, and combining them to produce an offspring” | limitations[5] |
| limitation | A second, separate motivation is offered by analogy to conspiracies.— “A closely related, but slightly different motivation for dropout comes from thinking about successful conspiracies” | limitations[6] |
| limitation | The test-time procedure is an approximation to averaging an exponential number of networks.— “We propose to do this by approximating an equally weighted geometric mean of the predictions of an exponential number of learned models that share parameters” | limitations[7] |