Auto-Encoding Variational Bayes

Diederik P. Kingma, Max Welling · ICLR / arXiv · 2014 · artificial-intelligence · record kingma2013-vae
tag layer · index · concepts · authors · עברית

Where this record's text came from. faceted 2026-08 by reading the locally held text (pdftext); every evidence string verified as an exact substring of that text before merging. The pdftext extraction badly scrambles Section 2's opening (a figure caption is spliced mid-sentence), so the sentence about deferring variational inference over global parameters to future work — a genuine self-stated limitation — could not be quoted verbatim and was left out; only two limitations are recorded as a result. Extensive appendices (marginal-likelihood estimator, MLP details, 2D manifold visualisations) are referenced in the body but are not present in this extracted text. The wake-sleep relation is labelled 'extends' because the paper claims AEVB serves the same model class without wake-sleep's two-objective drawback; the vocabulary offers no 'supersedes'/'compares-against' option. The concurrent-and-independent relation to [RMW14] was deliberately not encoded, since none of the four relation types describes independent co-discovery. 2026-08-14, 2026-08-15 and 2026-08-25 schema-0.3 passes: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node.

How to read this page. The article itself is not reproduced here — this paper is not cleared for redistribution, so what follows is the tag layer alone. The evidence sentences are withheld with the article. The table at the foot lists every tag the record carries; each row names the facet it belongs to, and the whose words column says whether the tag records the paper's own wording or the tagger's sentence about the paper. Every tag links to each other paper in the corpus carrying it. The machine-readable edition of the same thing is the TEI file beside this one, where the tags live in standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.

Other fields define these differently the same word, defined another way in another discipline
generative model 3 fields · likelihood 2 fields · variance 2 fields

Tags on this paper

Every tag the record carries, each linking to every other paper in the corpus that carries it. A tag appears here whether or not its words occur in the article — the spans below are only the ones a sentence in this paper licenses. Of the 9 concept tags on this paper, 5 are grounded in a quoted sentence.

variational autoencoder variational inference generative model latent variable AEVB recognition model approximate inference model reparameterization lower bound estimator the reparameterization trick: rewrite latent-variable sampling as a deterministic function of noise so the variational lower bound (ELBO) becomes differentiable and trainable by stochastic gradient descent variational latent posterior inference estimator gradient likelihood SGVB (Stochastic Gradient Variational Bayes) estimator continuous latent variables variational lower bound Bayesian chi-square Chi-Squared variance system-or-tool-building

Legend

relation to other work	`claim_relations`	how this paper stands to another: agrees, contradicts, extends, qualifies
claim type	`claim_type`	the shape of the claim: existence, mechanism, method, critique, prediction, null
concept edge	`concept_edges`	an asserted relation between two concepts, with the sentence asserting it
concept	`concepts`	an idea the paper works with, quoted where the paper uses it
content tag	`content_tags`	a finding, definition, quantity, statistic or variable named in the paper
control	`controls`	an experiment run to eliminate a rival explanation
limitation	`limitations`	what the paper says its evidence cannot support
mechanism	`mechanisms`	how the reported effect is said to come about
method	`method`	how the paper produced its evidence
phenomenon	`phenomenon`	the effect or regularity under study
sense	`senses`	a distinct use of a term, quoted where the paper fixes it
transfer	`transfers`	a method or concept carried in from another field

The paper

The full text is not reproduced here. This article is not cleared for redistribution, so the page carries the tag layer only. Its evidence sentences are withheld; the grounding status remains with each tag, and the article itself is at https://arxiv.org/abs/1312.6114. The reading view exists for the papers whose licence permits it.

Every tag on this page

claim_relations 3 · claim_type 1 · concept_edges 2 · concepts 5 · content_tags 9 · controls 1 · limitations 5 · mechanisms 1 · method 1 · phenomenon 1 · senses 3 · transfers 1

Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; 1 of 2 relations here carries that mark. The rest are the tagger’s reading of the sentence quoted beside them.

facet	tag	field	conf.	whose words
claim_type	method Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_type`		the paper's words
phenomenon	approximate posterior inference with continuous latent variables Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`phenomenon`		the paper's words
method	system-or-tool-building Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: system-or-tool-building	`method`		the paper's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: the reparameterization trick: rewrite latent-variable sampling as a deterministic function of noise so the variational lower bound (ELBO) becomes differentiable and trainable by stochastic gradient descent	`mechanisms[0]`		the tagger's words
limitations	The derivation assumes a fixed dataset for simplicity rather than the online or non-stationary case. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the derivation as presented: OUR METHOD CAN BE APPLIED TO ONLINE, NON-STATIONARY SETTINGS ... BUT HERE WE ASSUME A FIXED DATASET FOR SIMPLICITY	`limitations[0]`		the tagger's words
limitations	The marginal-likelihood evaluation only works for very low-dimensional latent spaces; estimates degrade beyond three latent variables. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the marginal-likelihood numbers reported: they use 3 LATENT VARIABLES, because FOR HIGHER DIMENSIONAL LATENT SPACE THE ESTIMATES BECAME UNRELIABLE	`limitations[1]`		the tagger's words
limitations	The approximate posterior is assumed Gaussian with diagonal covariance, and the paper argues in advance that this is a choice rather than a limitation. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: results obtained with this method: the approximate posterior is assumed to take an approximately Gaussian form with approximately diagonal covariance, which the authors call a simplifying choice and not a limitation of the method	`limitations[2]`	1.0	the tagger's words
limitations	The reconstruction criterion that gives the method its auto-encoding name is stated to be insufficient on its own for learning useful representations. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: an auto-encoding objective read as a representation-learning objective: the paper states it is well known that the	`limitations[3]`	1.0	the tagger's words
limitations	The guarantee is conditional on differentiability, which is assumed to hold almost everywhere rather than shown. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the claim that the estimator works in the intractable case: it holds under mild differentiability conditions, with the prior and likelihood assumed to have PDFs differentiable almost everywhere	`limitations[4]`	1.0	the tagger's words
transfers	variational Bayes / approximate posterior inference combined with neural networks (a recognition/encoder model) to learn deep generative models Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`transfers[0]`		the tagger's words
controls	comparison against the wake-sleep algorithm using the same encoder and the same parameters Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. eliminates: that AEVB's advantage over wake-sleep comes from the network architecture or parameter count rather than from the learning method · outcome: held	`controls[0]`	0.85	the tagger's words
concepts	AEVB Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: AEVB	`concepts[4]`		the paper's words
concepts	recognition model Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: recognition model	`concepts[5]`		the paper's words
concepts	approximate inference model Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: approximate inference model	`concepts[6]`		the paper's words
concepts	reparameterization Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: reparameterization	`concepts[7]`		the paper's words
concepts	lower bound estimator Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: lower bound estimator	`concepts[8]`		the paper's words
senses	the algorithm's name - AEVB IN THE PAPER, VAE IN THE FIELD: the authors name their algorithm Auto-Encoding VB and use that name throughout; the term the field kept appears once, as what you arrive at when the recognition model happens to be a neural network Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[0]`	1.0	the tagger's words
senses	the variational auto-encoder - AS A SPECIAL CASE, NOT THE SUBJECT: it is what AEVB becomes when a neural network is used for the recognition model, stated in a single consequence clause Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[1]`	1.0	the tagger's words
senses	the recognition model - TWO NAMES IN ONE PARENTHESIS: an approximate inference model, also called a recognition model, fitted to a posterior that cannot be computed Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[2]`	1.0	the tagger's words
claim_relations	extends → variational inference (variational Bayes) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[0]`		the tagger's words
claim_relations	builds-on → predictive sparse decomposition (PSD) [KRL08] Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[1]`		the tagger's words
claim_relations	extends → the wake-sleep algorithm [HDFN95] Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[2]`		the tagger's words
concept_edges	approximate inference model —same-as→ recognition model Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. strength of the relation: 1 also in: approximate inference model · recognition model	`concept_edges[0]`		the tagger's words
concept_edges	reparameterization —causes→ lower bound estimator Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. the relation itself is separately evidenced · strength of the relation: 1 · the relation stated: the reparameterisation YIELDS an estimator that ordinary SGD can optimise also in: reparameterization · lower bound estimator	`concept_edges[1]`		the tagger's words
content_tags	A reparameterization of the variational lower bound yields an estimator that can be optimized with standard stochastic gradient methods. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted.	`content_tags.findings[0]`		the tagger's words
content_tags	Posterior inference is made efficient by fitting an approximate inference (recognition) model to the intractable posterior using the proposed lower-bound estimator. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted.	`content_tags.findings[1]`		the tagger's words
content_tags	continuous latent variables Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: continuous latent variables	`content_tags.variables[0]`		the tagger's words
content_tags	variational lower bound Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: variational lower bound	`content_tags.variables[1]`		the tagger's words
content_tags	SGVB (Stochastic Gradient Variational Bayes) estimator Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. also in: SGVB (Stochastic Gradient Variational Bayes) estimator	`content_tags.definitions[0]`		the tagger's words
content_tags	recognition model Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. also in: recognition model	`content_tags.definitions[1]`		the tagger's words
content_tags	Bayesian Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: Bayesian	`content_tags.statistics[0]`		the tagger's words
content_tags	chi-square Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: chi-square	`content_tags.statistics[1]`		the tagger's words
content_tags	variance Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: variance	`content_tags.statistics[3]`		the tagger's words

Interpretive choices

Tags that could not be placed in the text:

none — every tag was located in the text

What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.