Heavy-tailed prior distributions for sequence count data: removing the noise and preserving large differences — TEI-tagged view

Anqi Zhu, Joseph G. Ibrahim, Michael I. Love · 2018 · statistics · doi:10.1093/bioinformatics/bty895 · TEI xml · tagged edition · index · concepts · authors
Where this record's text came from. 2026-08-22 and 2026-08-26 schema-0.3 passes: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node.
Other fields define these differently the same word, defined another way in another discipline
library size 2 fields · variance 2 fields
Tags on this paper every tag the record carries, whether or not the words occur in the text below — a number is how many papers in the corpus carry that tag; a tag with no number is unique to this paper
logarithmic fold change2 pseudocounts shrinkage estimator Cauchy prior local false sign rate Laplace approximation empirical Bayes2 maximum a posteriori2 system-or-tool-building173
Method, as described: empirical Bayes with an adaptive Cauchy prior, fitted by approximate posterior estimation, evaluated against filtering, Normal priors and other shrinkage estimators

Our tag layer for this paper

Cross-disciplinary analysis
content-tags & facets: phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text
phenomenonmaximum-likelihood effect sizes that blow up when counts are low, producing large estimates that are not differences— “When the read counts are low or highly variable, the maximum likelihood estimates for the LFCs has high variance, leading to large estimates not representative of true differences, and poor ranking of genes by effect size.”
claim-typemethod— “The proposed method, Approximate Posterior Estimation for generalized linear model, apeglm , has lower bias than previously proposed shrinkage estimators, while still reducing variance for those genes with little information for statistical inference.”
method — evidence— “Here, we propose the use of a heavy-tailed Cauchy prior distribution for effect sizes, which avoids the use of filter thresholds or pseudocounts.”
object of study — evidence— “The difference is usually defined in terms of the logarithmic fold change (LFC) between average expression levels of different conditions.”
research question — evidence— “However, in our analysis, we found that filtering or application of Normal priors each can have drawbacks, either leading to loss of genes with sufficient signal, or overly aggressive shrinkage of true, large LFCs.”
contribution — evidence— “Here, we propose the use of a heavy-tailed Cauchy prior distribution for effect sizes, which avoids the use of filter thresholds or pseudocounts.”
mechanismthe estimate is unreliable exactly where the counts are small, so the maximum likelihood LFC has high variance and produces `large estimates not representative of true differences` - the noise appears as a big effect, not as a small one— “When the read counts are low or highly variable, the maximum likelihood estimates for the LFCs has high variance, leading to large estimates not representative of true differences, and poor ranking of genes by effect size.”mechanisms[0]
mechanisma heavy tail on the prior is what lets shrinkage be aggressive near zero and gentle far from it: the Cauchy pulls in the noise-driven estimates while leaving genuinely large fold changes where they are— “Here, we propose the use of a heavy-tailed Cauchy prior distribution for effect sizes, which avoids the use of filter thresholds or pseudocounts.”mechanisms[1]
mechanismswapping a Normal prior for a Cauchy gives heavy tails, so small effects are still shrunk while genuinely large ones survive - shrinkage without flattening— “We extend the basic framework of DESeq2 , a Negative Binomial (NB) generalized linear model (GLM) ( McCullagh and Nelder, 1989 ) with moderated dispersion parameter, by exchanging the Normal distribution as a prior on LFCs with a heavy-tailed Cauchy distribution (a t distribution with 1 degree of freedom).”mechanisms[2]
mechanismthe prior's scale is fitted by subtracting each estimate's own error variance from its squared value, so the prior learns how much of the observed spread is real— “This equation is motivated by assuming that the MLE β ^ g k follows a Normal distribution around the true value β gk with variance e g k 2 , and that the β gk themselves follow a Normal distribution with mean zero.”mechanisms[3]
mechanismthe scale is set from a Normal assumption even though the prior itself is Cauchy - the two roles come apart and the authors say the mismatch works in practice— “Although we use a Cauchy prior in apeglm in order to accommodate large effect sizes from potentially only a few genes, we found that setting the scale of the prior by assuming the β gk follow a zero-centered Normal distribution performed well in practice.”mechanisms[4]
critiques count filtering as practised in edgeR and limma-voom, which removes genes before analysis and can lose those with sufficient signal— “The authors of edgeR ( McCarthy et al. , 2012 ) and limma-voom ( Law et al. , 2014 ) suggest using a filtering rule that removes genes with low scaled counts before statistical analysis”claim_relations[0]
critiques DESeq2's adaptive Normal prior, which shrinks true large fold changes too aggressively— “DESeq2 applies an adaptive Normally distributed prior, to produce a shrinkage estimator for the LFC for each gene.”claim_relations[1]
extends the empirical Bayes shrinkage tradition itself - the method is offered as a replacement for the prior, not for the framework— “The proposed method, Approximate Posterior Estimation for generalized linear model, apeglm , has lower bias than previously proposed shrinkage estimators, while still reducing variance for those genes with little information for statistical inference.”claim_relations[2]
senselogarithmic fold change - AS ESTIMATED, NOT AS DEFINED. The definition is a ratio of average expression between conditions; the estimate at low counts is `large ... not representative of true differences`. THE SAME SYMBOL NAMES A QUANTITY AND A NUMBER THAT DOES NOT MEASURE IT— “When the read counts are low or highly variable, the maximum likelihood estimates for the LFCs has high variance, leading to large estimates not representative of true differences, and poor ranking of genes by effect size.”senses[0]
senseshrinkage - the family of corrections, and this paper's point is that the family members fail in OPPOSITE directions: a filter loses true signal, a Normal prior flattens true large effects. WHICH ERROR YOU GET IS A CHOICE, AND USUALLY AN UNEXAMINED ONE— “However, in our analysis, we found that filtering or application of Normal priors each can have drawbacks, either leading to loss of genes with sufficient signal, or overly aggressive shrinkage of true, large LFCs.”senses[1]
sensepseudocount - a correction that `must be adapted per dataset`. The paper's remedy is to remove the parameter rather than to tune it, which is the same move a field makes when it replaces an inferred name with a measured one— “One approach is to introduce filtering thresholds and pseudocounts to exclude or moderate estimated LFCs. Filtering may result in a loss of genes from the analysis with true differences in expression, while pseudocounts provide a limited solution that must be adapted per dataset.”senses[2]
controlthe method is compared against BOTH failure modes rather than against one baseline - filtering, which loses genes, and Normal priors, which over-shrink — held— “The proposed method, Approximate Posterior Estimation for generalized linear model, apeglm , has lower bias than previously proposed shrinkage estimators, while still reducing variance for those genes with little information for statistical inference.” — rules out a new estimator that beats the over-shrinking baseline by under-shrinking, or the reverse; the two-sided comparison is what makes 'lower bias while still reducing variance' a claim rather than a trade— “However, in our analysis, we found that filtering or application of Normal priors each can have drawbacks, either leading to loss of genes with sufficient signal, or overly aggressive shrinkage of true, large LFCs.”controls[0]
controlwhether the correction needs a per-dataset parameter — held— “Here, we propose the use of a heavy-tailed Cauchy prior distribution for effect sizes, which avoids the use of filter thresholds or pseudocounts.” — rules out a method whose performance is really the analyst's tuning - the proposed prior 'avoids the use of filter thresholds or pseudocounts' entirely— “One approach is to introduce filtering thresholds and pseudocounts to exclude or moderate estimated LFCs. Filtering may result in a loss of genes from the analysis with true differences in expression, while pseudocounts provide a limited solution that must be adapted per dataset.”controls[1]
controlwhether the approach is tied to the negative-binomial likelihood it was developed on — held— “The apeglm method accepts arbitrary likelihoods, as long as additional parameters are pre-specified, such as the dispersion. apeglm can therefore also be extended for use on… — rules out a method that works because of one distributional assumption - apeglm 'accepts arbitrary likelihoods, as long as additional parameters are pre-specified'controls[2]
controltesting robustness when the assumption of a unimodal symmetric distribution of effects is violated — held — rules out the possibility that the method only works when its own prior assumption happens to hold
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[3]
controlquantifying the effect of substituting estimated standard errors where the derivation requires known ones, specifically at very low counts — held
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
— rules out the possibility that the borrowed estimator breaks exactly where the data are weakest
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[4]
controltrying an alternative posterior approximation by integrating over a fine grid instead of the Laplace method — failed
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
— rules out the possibility that the chosen approximation was adopted without a competitor
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[5]
weightcount filtering as practised in edgeR and limma-voom, which removes genes before analysis and can lose those with sufficient signal — 0.9
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
claim_relations[0]
weightDESeq2's adaptive Normal prior, which shrinks true large fold changes too aggressively — 0.9
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
claim_relations[1]
weightthe empirical Bayes shrinkage tradition itself - the method is offered as a replacement for the prior, not for the framework — 0.8
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
claim_relations[2]
outcomecontrol — held
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[0]
outcomecontrol — held
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[1]
outcomecontrol — held
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[2]
outcomecontrol — held
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[3]
outcomecontrol — held
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[4]
outcomecontrol — failed
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
controls[5]
limitationEVERY STANDARD CORRECTION TRADES ONE ERROR FOR THE OPPOSITE ONE: `filtering or application of Normal priors each can have drawbacks, either leading to loss of genes with sufficient signal, or overly aggressive shrinkage of true, large LFCs`
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
limitations[0]
limitationand the second-order cost is stated: filtering loses genes with real differences, and `pseudocounts provide a limited solution that must be adapted per dataset` - a correction re-tuned per dataset is a free parameter wearing a method's name
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
limitations[1]
limitationchoosing the filter is named as an act of judgement rather than a setting: `Setting the threshold requires careful consideration of which genes may be lost`
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
limitations[2]
limitationif most genes move in the same direction the global normalisation breaks down - and the paper says this defeats EVERY method under comparison, not only theirs
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
limitations[3]
limitationthat failure is detectable, and the paper names the diagnostics rather than leaving it as a caveat
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
limitations[4]
limitationwhere the maximum likelihood estimates are not supplied the prior scale falls back to a fixed value rather than being fitted
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
limitations[5]
Content tags
content-observations: finding · quantity · variable · definition · key-term — grounded in the full text
finding
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.findings[0]
finding
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.findings[1]
finding
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.findings[2]
finding
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.findings[3]
finding
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.findings[4]
findingthe Laplace approximation was consistently more accurate than numerical integration over a grid
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.findings[5]
findingthe resulting scale estimate is only slightly biased even at very low counts
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.findings[6]
findingshrinkage improves the ranking of genes by effect size relative to methods that do not shrink at all
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.findings[7]
quantityruntimes for the pipeline on a ten-thousand-gene simulation - dispersion and MLE fitting, Normal-prior shrinkage, Cauchy-prior shrinkage, and MAP-only
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.quantities[0]
variablelibrary size
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.variables[0]
definitionlogarithmic fold change
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.definitions[0]
definitionlocal false sign rate
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.definitions[1]
Concepts
terms this paper carries, each with the sentence it was cut from — the layer that joins one paper to another. Terms with no quote yet are on CONCEPTS.html.
conceptlogarithmic fold change — not defined in this paper's own words; the difference between conditions, defined as a log ratio of average expression
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
concepts[0]
conceptpseudocounts — not defined in this paper's own words; constants added to counts to moderate the estimate - a limited solution that must be adapted per dataset
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
concepts[1]
conceptshrinkage estimator — not defined in this paper's own words; an estimator that pulls unreliable effect sizes toward zero; DESeq2's uses an adaptive Normal prior
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
concepts[2]
conceptCauchy prior — not defined in this paper's own words; the heavy-tailed prior proposed here, which shrinks noise without flattening large true effects and needs no threshold
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
concepts[3]
conceptlocal false sign rate — not defined in this paper's own words; the posterior probability that the estimate has the WRONG SIGN - a replacement for the P value that asks about direction rather than about zero
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
concepts[4]
conceptLaplace approximation — not defined in this paper's own words; estimating the posterior covariance from the negative inverse Hessian; chosen after a grid alternative was tried and found less accurate
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
concepts[5]
conceptempirical Bayes — not defined in this paper's own words; setting the prior's scale FROM THE DATA - the spread of the maximum likelihood estimates, corrected for their own standard errors
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
concepts[6]
conceptmaximum a posteriori — not defined in this paper's own words; the posterior mode used as the shrinkage estimator, found by numeric optimisation rather than sampling
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
concepts[7]
Statistics — cross-field join keys
field-invariant statistical terms, used in their statistical sense (gated on a number / symbol / test). A p-value or a standard deviation means the SAME thing in physics, biology, psychology and sociology — so these are the strongest cross-field join keys. Grounded verbatim.
maximum likelihood
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.statistics[0]
variance
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.statistics[1]
posterior SD
Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher.
content_tags.statistics[2]

When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.

Tag layer © the Meta-Tagging Project, CC BY 4.0. The paper text is © its authors/publisher and is linked, not reproduced. Excerpts shown are short quotations for identification and verification. · back to index · concepts · authors · עברית
© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824