A Style-Based Generator Architecture for Generative Adversarial Networks

A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras Samuli Laine Timo Aila

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

A Style-Based Generator Architecture for Generative Adversarial Networks 2019 CVPR / arXiv 1812.04948 karras2019-stylegan karras2019-stylegan.txt, the project's staged copy 20cb3a2abd351595caa6f89b0d59bae12c4d3ad4 arxiv-open art-history

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them method Quotation withheld because this record is not cleared for redistribution. latent-space disentanglement in GAN image synthesis Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. mechanisms each style controls exactly one convolution: AdaIN normalises the channel to zero mean and unit variance first, so the incoming style's scale and bias do not depend on the previous statistics and are overridden by the next AdaIN Quotation withheld because this record is not cleared for redistribution. mechanisms style localisation is trained for, not only architected: a percentage of images are generated from two latent codes with a crossover point, which stops the network assuming adjacent styles are correlated Quotation withheld because this record is not cleared for redistribution. the paper's reason - that a traditional generator must otherwise invent spatially-varying pseudorandom numbers from earlier activations - is the preceding paragraph, not this span stochastic detail is supplied by the architecture: per-pixel noise is added after each convolution Quotation withheld because this record is not cleared for redistribution. mechanisms the division of labour between style and noise is enforced by the discriminator rather than by design: using noise for a global property would produce spatially inconsistent decisions and be penalised Quotation withheld because this record is not cleared for redistribution. mechanisms the intermediate latent space is defined by what it is NOT required to do - it need not follow the training data's density, which is the constraint the paper blames for entanglement in the input space Quotation withheld because this record is not cleared for redistribution. the second adjustment - dividing by epsilon squared because the perceptual distance is quadratic - is a separate sentence far down the section; a span reaching it crosses a footnote, Table 3 and a figure caption the path length is defined as a limit under infinitely fine subdivision and computed as an approximation with a small finite epsilon instead Quotation withheld because this record is not cleared for redistribution. the conditional entropy and its exponentiation into the final score are described in the sentences after this span separability is measured without an encoder: auxiliary attribute classifiers label generated images, the least confident half is discarded, and a linear SVM is fitted per attribute on the latent-space points Quotation withheld because this record is not cleared for redistribution. the input latent space, constrained to follow the training-data density, has some unavoidable entanglement what this limits: any disentanglement claim about the input latent space: THE INPUT LATENT SPACE MUST FOLLOW THE PROBABILITY DENSITY OF THE TRAINING DATA, AND WE ARGUE THAT THIS LEADS TO SOME DEGREE OF UNAVOIDABLE ENTANGLEMENT Quotation withheld because this record is not cleared for redistribution. The perceptual path length measurement is admitted to be biased in favour of the input latent space Z. what this limits: the perceptual path length comparison, biased against the paper's own conclusion: THIS MEASUREMENT IS IN FACT SLIGHTLY BIASED IN FAVOR OF THE INPUT LATENT SPACE Z Quotation withheld because this record is not cleared for redistribution. Continued FID improvement during training comes at the cost of a more entangled representation (rising path lengths). what this limits: any use of FID as the training objective: THE SLOWLY RISING PATH LENGTHS INDICATE THAT THE IMPROVEMENTS IN FID COME AT THE COST OF A MORE ENTANGLED REPRESENTATION Quotation withheld because this record is not cleared for redistribution. The new FFHQ face dataset was crawled from Flickr and therefore inherits that site's biases. what this limits: every result and every model trained on FFHQ: the images were crawled from Flickr, THUS INHERITING ALL THE BIASES OF THAT WEBSITE, then automatically aligned and cropped Quotation withheld because this record is not cleared for redistribution. The central term has no agreed definition and the paper proceeds on a shared goal instead. what this limits: any disentanglement claim made here: the paper states there are various definitions of disentanglement and adopts a common goal - a latent space of linear subspaces each controlling one factor of variation - rather than one of them Quotation withheld because this record is not cleared for redistribution. The two metrics reported are new because the existing ones cannot be run here, so the numbers compare to nobody. what this limits: the disentanglement figures given here: the authors state the previously proposed metrics require an encoder network mapping images to latent codes, which this architecture does not have, so the two metrics used are new ones of their own Quotation withheld because this record is not cleared for redistribution. The perceptual distance the headline metric is built on is another network's embedding, fitted to human ratings. what this limits: the perceptual path length reported here: it is built on a pairwise image distance computed as a weighted difference between two VGG16 embeddings, with the weights fitted so the metric agrees with human perceptual similarity judgments Quotation withheld because this record is not cleared for redistribution. The mechanism claimed for the improvement is posited rather than shown. what this limits: the explanation offered for why this generator disentangles: the authors posit that there is pressure on the generator because generating realistic images should be easier from a disentangled representation than from an entangled one Quotation withheld because this record is not cleared for redistribution. What the paper controls under the name 'style' includes the pose and the shape of the face. what this limits: any reading of this model's 'style' as a manner of depiction: copying the coarse-resolution styles transfers pose, general hair style, face shape and eyeglasses, which are properties of the subject rather than of the manner Quotation withheld because this record is not cleared for redistribution. the style-transfer idea of separating content from style, taken from that literature and built into the generator architecture so that style is injected at each convolution layer - the network is named after it Quotation withheld because this record is not cleared for redistribution. an ablation table beginning from a Progressive GAN baseline, with each row adding one change to the row above eliminates: attributing the improvement to the architecture as a whole - a table where every row differs from the previous one by a single change is what makes any single change attributable · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. style style Quotation withheld because this record is not cleared for redistribution. spatially invariant statistics spatially invariant statistics Quotation withheld because this record is not cleared for redistribution. disentanglement disentanglement Quotation withheld because this record is not cleared for redistribution. perceptual path length perceptual path length Quotation withheld because this record is not cleared for redistribution. linear separability linear separability Quotation withheld because this record is not cleared for redistribution. coarse spatial resolutions coarse spatial resolutions Quotation withheld because this record is not cleared for redistribution. 'style' - REUSED BY THE AUTHORS' OWN DECISION, FOR ARCHITECTURAL RESEMBLANCE: they say plainly that they choose to reuse the word for the vector y because similar network architectures are already used in feedforward style transfer and image-to-image translation, so the word marks a family of architectures rather than a property of a picture Quotation withheld because this record is not cleared for redistribution. style, once more removed - AS SPATIALLY INVARIANT STATISTICS: the paper's warrant is the style transfer literature's finding that spatially invariant statistics encode style while spatially varying features encode the instance, which is the same move gatys2015 made with a Gram matrix Quotation withheld because this record is not cleared for redistribution. disentanglement - A GOAL STANDING IN FOR A DEFINITION: the paper records that the literature holds various definitions and adopts the shared aim of a latent space of linear subspaces, then measures that aim with two metrics of its own devising Quotation withheld because this record is not cleared for redistribution. style AT COARSE RESOLUTION - POSE, FACE SHAPE AND EYEGLASSES: the styles copied from the coarse spatial resolutions carry high-level aspects of the subject itself, so at this level the word ranges over what the picture is OF and not only how it looks Quotation withheld because this record is not cleared for redistribution. style ON NON-FACE DATASETS - CAMERA VIEWPOINT AND CHOICE OF FURNITURE: in bedrooms the coarse styles control where the camera is and the middle styles select which furniture appears, which an art historian would call composition and subject matter rather than style Quotation withheld because this record is not cleared for redistribution. builds-on → style transfer literature Quotation withheld because this record is not cleared for redistribution. builds-on → Progressive GAN (Karras et al.) Quotation withheld because this record is not cleared for redistribution. builds-on → the Progressive GAN setup of Karras et al. [30] Quotation withheld because this record is not cleared for redistribution. extends → the parallel self-modulation work of Chen et al. [6] Quotation withheld because this record is not cleared for redistribution. style —operationalised-as→ spatially invariant statistics style spatially invariant statistics strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. 50,000 — images used to compute the FID metric Quotation withheld because this record is not cleared for redistribution. 1024×1024 — output resolution (progressive from 4x4 to 1024x1024) Quotation withheld because this record is not cleared for redistribution. The generator learns, without supervision, to separate high-level attributes such as pose and identity from stochastic variation such as freckles and hair, and allows intuitive, scale-specific control of synthesis. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. The new generator improves on the state of the art in traditional distribution-quality metrics, interpolates demonstrably better, and better disentangles the latent factors of variation. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. FID FID about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. adaptive instance normalization (AdaIN) adaptive instance normalization (AdaIN) about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. EVIDENCE CORRECTED (RE-GROUNDED) · FFHQ FFHQ EVIDENCE CORRECTED: RE-GROUNDED. The published string was not verbatim in EITHER copy of the paper. A quote field promises the author's words; this held 'a new, highly…' - which welds the abstract's phrasing 2,456 characters earlier to the introduction's parenthetical. The introduction states both in one sentence and that is what is quoted now. Quotation withheld because this record is not cleared for redistribution. mean mean about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. standard-deviation standard-deviation Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://arxiv.org/abs/1812.04948.