Generative Adversarial Networks

Generative Adversarial Networks Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville Yoshua Bengio

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Generative Adversarial Networks 2014 Advances in Neural Information Processing Systems (NeurIPS) 1406.2661 goodfellow2014-gan goodfellow2014-gan.txt, the project's staged copy d704dd1ffdfe59029e04057059288f83ee161ad2 arxiv-open artificial-intelligence

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them method Quotation withheld because this record is not cleared for redistribution. adversarial training of generative models Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. mechanisms adversarial training: a generator and discriminator compete in a minimax game - the generator learns to produce samples that fool the discriminator (which learns real vs fake), driving the generator toward the data distribution Quotation withheld because this record is not cleared for redistribution. the likelihood-estimation method has high variance and does not perform well in high dimensions what this limits: the reported likelihood numbers: the estimation method DOES NOT PERFORM WELL IN HIGH DIMENSIONAL SPACES Quotation withheld because this record is not cleared for redistribution. The model gives no explicit representation of the generator's density, and the discriminator must be kept synchronised with the generator or the generator collapses (the 'Helvetica scenario'). what this limits: any use requiring a density or stable training: THERE IS NO EXPLICIT REPRESENTATION OF pg(x), AND D MUST BE SYNCHRONIZED WELL WITH G DURING TRAINING Quotation withheld because this record is not cleared for redistribution. The Parzen-window likelihood estimate used to evaluate the model is high-variance and performs poorly in high dimensions. what this limits: the same scope as limitations[0], with the reason it was used anyway: IT HAS SOMEWHAT HIGH VARIANCE AND DOES NOT PERFORM WELL IN HIGH DIMENSIONAL SPACES BUT IT IS THE BEST METHOD AVAILABLE TO OUR KNOWLEDGE Quotation withheld because this record is not cleared for redistribution. The convergence result is proved in the non-parametric limit; using a multilayer perceptron for G breaks the theoretical guarantee. what this limits: the convergence guarantee, which is proved in the non-parametric limit and does not cover the implementation: USING A MULTILAYER PERCEPTRON TO DEFINE G INTRODUCES MULTIPLE CRITICAL POINTS IN PARAMETER SPACE Quotation withheld because this record is not cleared for redistribution. The authors do not claim the generated samples are better than those of existing methods. what this limits: any comparison with existing generative models: WHILE WE MAKE NO CLAIM THAT THESE SAMPLES ARE BETTER THAN SAMPLES GENERATED BY EXISTING METHODS, WE BELIEVE THAT THESE SAMPLES ARE AT LEAST COMPETITIVE Quotation withheld because this record is not cleared for redistribution. The authors state their evaluation method is high-variance and poor in high dimensions, call it the best available, and report the paper's headline numbers on it. what this limits: the log-likelihood figures in Table 1: they come from Gaussian Parzen-window estimation, which the authors say Quotation withheld because this record is not cleared for redistribution. The convergence results are proved in the non-parametric limit, and the paper states the implementation does not satisfy that condition and has no theoretical guarantee. what this limits: the guarantee that pg converges to pdata: it is proved in a non-parametric setting, while the implementation optimises the parameters of a multilayer perceptron, which the paper says lacks theoretical guarantees Quotation withheld because this record is not cleared for redistribution. The objective that is trained is not the objective that is analysed; the substitution is made because the analysed one saturates, and the equivalence claimed is of fixed points only. what this limits: the relation between Eq. 1 and what was run: G is trained to maximize log D(G(z)) instead, which the paper says gives the same fixed point but much stronger gradients early in learning Quotation withheld because this record is not cleared for redistribution. The two stated disadvantages are structural, not incidental: the model has no explicit density, and training can collapse to too few outputs if the two networks fall out of step. what this limits: adversarial nets as a density model or as a stable training procedure: there is no explicit representation of pg(x), and D must be synchronized well with G to avoid G collapsing too many values of z to the same value of x Quotation withheld because this record is not cleared for redistribution. The claim made for the sample quality is a stated belief, and the paper explicitly declines to claim the samples are better than existing methods. what this limits: the figures of generated samples as evidence of quality: the authors make no claim that they are better than existing methods and state only a belief that they are at least competitive Quotation withheld because this record is not cleared for redistribution. a minimax two-player game between a generator and a discriminator, adapting game-theoretic adversarial competition to train generative models Quotation withheld because this record is not cleared for redistribution. the displayed samples declared to be FAIR RANDOM DRAWS rather than selected eliminates: selection of the best outputs as the basis for judging the model - a figure of hand-picked samples shows what the model CAN do at its best, which is not what the distribution looks like, and nothing in the image itself would reveal the difference · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. a column showing the NEAREST TRAINING EXAMPLE beside each generated sample eliminates: memorisation of the training set as the explanation of sample quality - a model that reproduced its training data would look excellent by every visual measure, and the neighbour column is what makes that checkable by the reader · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. discriminator discriminator Quotation withheld because this record is not cleared for redistribution. minimax game minimax game Quotation withheld because this record is not cleared for redistribution. adversarial nets adversarial nets Quotation withheld because this record is not cleared for redistribution. Parzen window Parzen window Quotation withheld because this record is not cleared for redistribution. log-likelihood log-likelihood Quotation withheld because this record is not cleared for redistribution. the training objective - AS ANALYSED vs AS TRAINED: the minimax value function of Eq. 1 is what the theorems are about; the implementation maximises a different function with the same fixed point, because the analysed one saturates early in learning Quotation withheld because this record is not cleared for redistribution. log-likelihood - AS A PARZEN-WINDOW ESTIMATE: not computed from the model, but from a Gaussian fitted to samples drawn from it, because the model has no tractable density Quotation withheld because this record is not cleared for redistribution. adversarial nets - THE PAPER'S OWN NAME, and it names the special case, not the framework: the framework is general and 'adversarial nets' is what the authors call the multilayer-perceptron instance of it Quotation withheld because this record is not cleared for redistribution. the failure mode - NAMED AS A JOKE: 'the Helvetica scenario', in which the generator maps too many inputs to the same output; the field later renamed this mode collapse, and Table 2 records it as the single word Helvetica Quotation withheld because this record is not cleared for redistribution. builds-on → game theory (minimax two-player game) Quotation withheld because this record is not cleared for redistribution. builds-on → SML/PCD training [31, 29] Quotation withheld because this record is not cleared for redistribution. builds-on → the Parzen-window likelihood evaluation procedure of Breuleux et al. [8] Quotation withheld because this record is not cleared for redistribution. extends → the generative stochastic network (GSN) framework [5] Quotation withheld because this record is not cleared for redistribution. extends → noise-contrastive estimation (NCE) [13] Quotation withheld because this record is not cleared for redistribution. log-likelihood —operationalised-as→ Parzen window log-likelihood Parzen window strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. adversarial nets —operationalised-as→ minimax game adversarial nets minimax game strength of the relation: 1 Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span writes 'D and G play the following two-player minimax game', not 'adversarial nets'. Quotation withheld because this record is not cleared for redistribution. D = 1/2 — discriminator output at the optimal solution (cannot distinguish real from fake) Quotation withheld because this record is not cleared for redistribution. A unique solution exists, with G recovering the training data distribution and D equal to 1/2 everywhere; with multilayer perceptrons the whole system can be trained by backpropagation. Quotation withheld because this record is not cleared for redistribution. generative adversarial net generative adversarial net Quotation withheld because this record is not cleared for redistribution. discriminator D discriminator D Quotation withheld because this record is not cleared for redistribution. Bayesian Bayesian Quotation withheld because this record is not cleared for redistribution. standard-error standard-error about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. variance variance Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://arxiv.org/abs/1406.2661.