| phenomenon | variational inference for approximate posterior densities— “Abstract One of the core problems of modern statistics is to approximate difficult-to-compute probability densities.” |
| claim-type | method— “In this paper, we review variational inference (VI), a method from machine learning that approximates probability densities through optimization.” |
| method — evidence | — “In this paper, we review variational inference (VI), a method from machine learning for approximating probability densities (Jordan et al., 1999; Wainwright and Jordan, 2008).” |
| transfer | from machine learning: variational inference used to approximate difficult Bayesian posterior densities in statistics— “we review variational inference (VI), a method from machine learning that approximates probability densities through optimization” | transfers[0] |
| mechanism | recast posterior inference as optimization: choose the distribution in a simple (mean-field) family that is closest in KL divergence to the true posterior, via coordinate ascent— “is closest in KL divergence to the conditional of interest. The fitted variational density then serves as a proxy for the exact conditional density.” | mechanisms[0] |
| extends | MCMC sampling for approximate Bayesian inference— “Variational inference is widely used to approximate posterior densities for Bayesian models, an alternative strategy to Markov chain Monte Carlo (MCMC) sampling.” | claim_relations[0] |
| extends | Bishop (2006)— “The idea appears in Bishop (2006), but the argument there uses gradients, which we do not.” | claim_relations[1] |
| builds-on | Hoffman et al. (2013)— “We derive a CAVI algorithm for this model, based on Hoffman et al. (2013).” | claim_relations[2] |
| builds-on | Drugowitsch (2013)— “Here we present a conditionally conjugate Bayesian linear regression model with an ARD prior, based on Drugowitsch (2013).” | claim_relations[3] |
| sense | accuracy of variational inference - AS POSTERIOR VARIANCE: known to be underestimated, and stated to be a consequence of the objective function rather than of any implementation— “We do know that variational inference generally underestimates the variance of the posterior density; this is a consequence of its objective function.” | senses[0] |
| sense | accuracy of variational inference - AS POSTERIOR PREDICTIVE PERFORMANCE: reported empirically NOT to suffer, which is the opposite verdict on the same method— “Several lines of empirical research have shown that variational inference does not necessarily suffer in accuracy, e.g., in terms of posterior predictive densities” | senses[1] |
| sense | closeness - stipulated to be measured by Kullback-Leibler divergence, which is the choice that makes the whole method definite— “The idea behind VI is to first posit a family of densities and then to find the member of that family which is close to the target. Closeness is measured by Kullback-Leibler divergence.” | senses[2] |
| sense | local optimum - not treated as a defect throughout: the review states different initialisations reaching different optima is NOT ALWAYS A DISADVANTAGE— “Figure 2: Different initializations may lead CAVI to find different local optima of the ELBO. This is not always a disadvantage.” | senses[3] |
| edge | closeness —operationalised-as→ Kullback-Leibler divergence— “The idea behind VI is to first posit a family of densities and then to find the member of that family which is close to the target. Closeness is measured by Kullback-Leibler divergence.” | concept_edges[0] |
| audit | coordinate ascent variational inference —is-a→ variational inference — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'CAVI' = coordinate ascent variational inference, introduced in the previous sentence as an algorithm for the variational optimisation problem; is-a is loose.— “CAVI iteratively optimizes each factor of the mean-field variational density, while holding the others fixed. It climbs the ELBO to a local optimum.” | concept_edges[1] |
| edge | MCMC —distinct-from→ variational inference— “MCMC is a tool for simulating from densities and variational inference is a tool for approximating densities.” | concept_edges[2] |
| for which use | limitation — any statistical guarantee: VARIATIONAL INFERENCE HAS BEEN STUDIED LESS RIGOROUSLY THAN MCMC, AND ITS STATISTICAL PROPERTIES ARE LESS WELL UNDERSTOOD— “But variational inference has been studied less rigorously than MCMC, and its statistical properties are less well understood.” | limitations[0] |
| for which use | limitation — any guarantee about a variational solution: VI IS POWERFUL, BUT IT IS NOT YET WELL UNDERSTOOD— “VI is powerful, but it is not yet well understood.” | limitations[1] |
| for which use | limitation — the paper's own VI-versus-MCMC comparison: THIS IS NOT A DEFINITIVE COMPARISON BETWEEN VARIATIONAL INFERENCE AND MCMC— “This is not a definitive comparison between variational inference and MCMC.” | limitations[2] |
| for which use | limitation — the review's coverage of collapsed variational inference and related speed-ups: THESE IDEAS ARE OUTSIDE THE SCOPE OF OUR REVIEW— “These ideas are outside the scope of our review.” | limitations[3] |
| for which use | limitation — every result in the review, which assumes independence: IN THIS REVIEW WE FOCUS ON THE MEAN-FIELD VARIATIONAL FAMILY, WHERE THE LATENT VARIABLES ARE MUTUALLY INDEPENDENT AND EACH GOVERNED BY A DISTINCT FACTOR IN THE VARIATIONAL DENSITY— “In this review we focus on the mean-field variational family, where the latent variables are mutually independent and each governed by a distinct factor in the variational density.” | limitations[4] |
| duplicate of | limitation — limitations[3]— “These ideas are outside the scope of our review.” | limitations[5] |
| for which use | limitation — the same scope as limitations[3] - collapsed VI is excluded, so the review is not a survey of what is fastest in practice— “These ideas are outside the scope of our review.” | limitations[5] |
| for which use | limitation — the survey of applications: THIS DISCUSSION IS NOT EXHAUSTIVE; OUR INTENTION IS TO OUTLINE THE DIVERSITY OF APPLICATIONS OF VARIATIONAL INFERENCE— “This discussion is not exhaustive; our intention is to outline the diversity of applications of variational inference.” | limitations[6] |
| for which use | limitation — any accuracy claim against MCMC: THE RELATIVE ACCURACY OF VARIATIONAL INFERENCE AND MCMC IS STILL UNKNOWN— “The relative accuracy of variational inference and MCMC is still unknown.” | limitations[7] |
| for which use | limitation — any use of the posterior variance, with the direction and the cause: VARIATIONAL INFERENCE GENERALLY UNDERESTIMATES THE VARIANCE OF THE POSTERIOR DENSITY; THIS IS A CONSEQUENCE OF ITS OBJECTIVE FUNCTION— “We do know that variational inference generally underestimates the variance of the posterior density; this is a consequence of its objective function.” | limitations[8] |
| for which use | limitation — the empirical timing comparison: OTHER SAMPLERS, SUCH AS A COLLAPSED GIBBS SAMPLER, MAY PERFORM BETTER THAN HAMILTONIAN MONTE CARLO SAMPLING— “Other samplers, such as a collapsed Gibbs sampler, may perform better than Hamiltonian Monte Carlo sampling.” | limitations[9] |
| for which use | limitation — the same scope as limitations[0]: THE STATISTICAL PROPERTIES OF VARIATIONAL INFERENCE ARE NOT YET WELL UNDERSTOOD, ESPECIALLY IN CONTRAST TO THE WEALTH OF ANALYSIS OF MCMC TECHNIQUES— “Finally, the statistical properties of variational inference are not yet well understood, especially in contrast to the wealth of analysis of MCMC techniques.” | limitations[10] |
| for which use | limitation — choosing variational inference on the strength of its literature: the review states it has been studied less rigorously than MCMC and that its statistical properties are less well understood— “But variational inference has been studied less rigorously than MCMC, and its statistical properties are less well understood.” | limitations[11] |
| for which use | limitation — any posterior variance, credible interval or uncertainty estimate taken from a variational posterior: the review states VI generally underestimates posterior variance as a consequence of its objective function, and that whether this is acceptable depends on the task— “We do know that variational inference generally underestimates the variance of the posterior density; this is a consequence of its objective function. But, depending on the task at hand, underestimating the variance may be acceptable.” | limitations[12] |
| for which use | limitation — a claim that variational inference is or is not as accurate as MCMC for a given problem: the review states the relative accuracy is still unknown— “The relative accuracy of variational inference and MCMC is still unknown.” | limitations[13] |
| for which use | limitation — reading an ELBO value as a measure of divergence from the posterior: the review states the KL cannot be computed and that the ELBO is an alternative objective equal to it up to an added constant— “Because we cannot compute the KL, we optimize an alternative objective that is equivalent to the KL up to an added constant” | limitations[14] |
| for which use | limitation — a single variational fit as THE variational approximation: the review states the ELBO is generally non-convex, that CAVI guarantees only a local optimum sensitive to initialisation, and that ten random initialisations each reached a different value— “The ELBO is (generally) a non-convex objective function. CAVI only guarantees convergence to a local optimum, which can be sensitive to initialization.”— “Each initialization reaches a different value, indicating the presence of many local optima in the ELBO.” | limitations[15] |
| for which use | limitation — using held-out predictive probability to decide that CAVI has converged: the review recommends it as a proxy while stating it is not guaranteed to increase monotonically, unlike the full ELBO— “Monitoring changes here is a proxy to monitoring the ELBO of the full data. (Unlike the full ELBO, held-out predictive probability is not guaranteed to monotonically increase across iterations of CAVI.)” | limitations[16] |
| replication | untested |
| limitation | VI has been studied less rigorously than MCMC and its statistical properties are less well understood— “But variational inference has been studied less rigorously than MCMC, and its statistical properties are less well understood.” | limitations[0] |
| limitation | The authors state that variational inference is not yet theoretically well understood.— “VI is powerful, but it is not yet well understood.” | limitations[1] |
| limitation | The paper's comparison against MCMC is not definitive; other samplers might perform better.— “This is not a definitive comparison between variational inference and MCMC.” | limitations[2] |
| limitation | Collapsed variational inference is explicitly excluded from the review's scope.— “These ideas are outside the scope of our review.” | limitations[3] |
| limitation | The review deliberately restricts itself to the mean-field family, excluding richer variational families.— “In this review we focus on the mean-field variational family, where the latent variables are mutually independent and each governed by a distinct factor in the variational density.” | limitations[4] |
| limitation | Collapsed variational inference and related speed-ups are excluded from the review's coverage.— “These ideas are outside the scope of our review.” | limitations[5] |
| limitation | The survey of applications is not exhaustive.— “This discussion is not exhaustive; our intention is to outline the diversity of applications of variational inference.” | limitations[6] |
| limitation | The accuracy of variational inference relative to MCMC is not known, and VI is known to underestimate posterior variance.— “The relative accuracy of variational inference and MCMC is still unknown.” | limitations[7] |
| limitation | Variational inference systematically underestimates posterior variance because of its objective function.— “We do know that variational inference generally underestimates the variance of the posterior density; this is a consequence of its objective function.” | limitations[8] |
| limitation | The paper's empirical CAVI-versus-Hamiltonian-Monte-Carlo timing comparison is not a definitive comparison of the two paradigms.— “Other samplers, such as a collapsed Gibbs sampler, may perform better than Hamiltonian Monte Carlo sampling.” | limitations[9] |
| limitation | The statistical theory of variational inference remains undeveloped, which the paper flags as an open problem rather than something it settles.— “Finally, the statistical properties of variational inference are not yet well understood, especially in contrast to the wealth of analysis of MCMC techniques.” | limitations[10] |
| limitation | The review states that its own subject is not yet well understood, and that its statistical properties have been studied less rigorously than the alternative it is compared to.— “But variational inference has been studied less rigorously than MCMC, and its statistical properties are less well understood.” | limitations[11] |
| limitation | The defect is known, directional, and declared possibly acceptable — the acceptability depending on the task rather than on the method.— “We do know that variational inference generally underestimates the variance of the posterior density; this is a consequence of its objective function. But, depending on the task at hand, underestimating the variance may be acceptable.” | limitations[12] |
| limitation | How accurate the method is relative to the alternative is stated to be unknown.— “The relative accuracy of variational inference and MCMC is still unknown.” | limitations[13] |
| limitation | The objective actually optimised is a substitute for the one wanted, adopted because the wanted one cannot be computed.— “Because we cannot compute the KL, we optimize an alternative objective that is equivalent to the KL up to an added constant” | limitations[14] |
| limitation | The optimisation reaches a local optimum only, the objective is generally non-convex, and different initialisations were observed to reach different values.— “The ELBO is (generally) a non-convex objective function. CAVI only guarantees convergence to a local optimum, which can be sensitive to initialization.” | limitations[15] |
| limitation | The monitoring quantity the review recommends in practice lacks the guarantee that belongs to the quantity it stands in for.— “Monitoring changes here is a proxy to monitoring the ELBO of the full data. (Unlike the full ELBO, held-out predictive probability is not guaranteed to monotonically increase across iterations of CAVI.)” | limitations[16] |