The coefficient of determination R2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded

The coefficient of determination R2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded Shinichi Nakagawa P. Johnson Holger Schielzeth

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

The coefficient of determination R2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded 2017 Journal of the Royal Society Interface 10.1098/rsif.2017.0213 nakagawa2017-the-coefficient-of-determination-r nakagawa2017-the-coefficient-of-determination-r.txt, the project's staged copy 1a86d8c7ed8f9b8dc80765fe5e94e6a6796f06c2 cc-by statistics

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them WHAT A STATISTIC MEANS WHEN THE MODEL GETS COMPLICATED - R-squared and the intra-class correlation for generalized linear mixed models, revisited by the authors of the previous definition and extended to families their own method could not reach Quotation withheld because this record is not cleared for redistribution. how to define the residual variance term that R-squared needs, when for most distributional families it CANNOT BE SEPARATED into the components the earlier definition assumed Quotation withheld because this record is not cleared for redistribution. a general definition of the observation-level variance for non-Gaussian GLMMs via the delta method, the lognormal approximation or the trigamma function - plus a correction to two of their own earlier recommendations Quotation withheld because this record is not cleared for redistribution. method Quotation withheld because this record is not cleared for redistribution. one of the most-reported numbers in science, with no single definition once the model has random effects Quotation withheld because this record is not cleared for redistribution. formal-model formal-model Quotation withheld because this record is not cleared for redistribution. A NAMING RULE - WHEN A MODIFIER IN A NAME IS LICENSED: `In such a case, the ICC should not be called `ADJUSTED`. For an ICC value to be adjusted for a source of variance, that variance must be more than 0 and omitted from the ICC calculation.` A published condition on the use of one word inside a statistic's name - the same shape as buck2011's rule for when two fields may share an acronym, one level down Quotation withheld because this record is not cleared for redistribution. AND ONE NAME FOR TWO QUANTITIES, KEPT APART BY A SUBSCRIPT: `MARGINAL AND CONDITIONAL R2 differ only in whether the random effect variance is included in the numerator` - the marginal is the proportion explained by the fixed effects, the conditional by fixed AND random. Two numbers a reader will see reported as `R2` Quotation withheld because this record is not cleared for redistribution. AND A RENAMING WHERE THE NOTATION DELIBERATELY DOES NOT FOLLOW: `We refer to this generalized version ... as `THE OBSERVATION-LEVEL VARIANCE` here rather than the residual variance (but we keep the notation). Note that the observation-level variance should not be confused with the variance associated with OLRE, which ... can be considered to be A PART OF it.` The name changes, the symbol does not, and the sentence that renames it also warns which neighbouring term it is not Quotation withheld because this record is not cleared for redistribution. AND THE EARLIER DEFINITION WORKED ONLY FOR TWO SPECIAL CASES, SAID PLAINLY: `obtaining for specific distributions is not always possible, because in many families of GLMMs, (observation-level variance) cannot be clearly separated into (overdispersion variance) and (distribution-specific variance). It turns out that binomial and Poisson distributions are special cases where can be usefully calculated ... This is not the case for other families.` The general method was a special case and the authors are the ones who found out Quotation withheld because this record is not cleared for redistribution. AND A DISTINCTION THE AUTHORS WITHDRAW FOR ONE FAMILY AND KEEP FOR ANOTHER: `In the past, we distinguished between ICC on the latent scale and on the observation scale. Such a distinction turns out to be strictly appropriate only for binomial distributions but not for Poisson distributions (and probably also not for other non-Gaussian distributions).` A retraction with its scope stated, including the part they are only probably right about Quotation withheld because this record is not cleared for redistribution. AND THEY NAME WHAT THEY USED TO CALL IT: `note that we would have called 1/lambda THE DISTRIBUTION-SPECIFIC VARIANCE` - the same quantity, the old name, and the new one, in a parenthesis. The corpus has papers that rename things and papers that correct things; this one prints the substitution table Quotation withheld because this record is not cleared for redistribution. AND THE NAMING CHOICE MOVES EVERY REPORTED NUMBER, IN A KNOWN DIRECTION: `the MINIMUM VALUE of 1/(p(1-p)) IS 4, which is larger than pi^2/3 = 3.29, meaning that the use of 1/p(1-p) IN R2 and ICC for binary data will always produce larger values than those using pi^2/3.` Which term you call the denominator decides whether every R2 in a literature is larger or smaller - not a range, a DIRECTION, with a floor. And Browne et al. observed exactly that Quotation withheld because this record is not cleared for redistribution. AND TWO OF THEIR OWN EARLIER RECOMMENDATIONS CONTRADICT EACH OTHER: `In our earlier papers, we proposed to use the exponential of the intercept ... WE ALSO SUGGESTED that it is possible to use the mean of observed values. Unfortunately, these two recommendations are often inconsistent with each other.` Two ways to compute one input, from one paper, that disagree Quotation withheld because this record is not cleared for redistribution. AND THE REASON IS THAT ONE OF THEM ESTIMATES A DIFFERENT LOCATION ENTIRELY: `In fact, exp(beta0) gives the median value of y rather than the mean ... Thus, the use of exp(beta0) will often OVERESTIMATE, providing SMALLER ESTIMATES OF R2 AND ICC ... when lambda is small, the difference can be substantial so the choice of the method needs to be reported for reproducibility.` A methods choice promoted to a reporting requirement because the two answers differ Quotation withheld because this record is not cleared for redistribution. AND THREE ROUTES REACH ONE QUANTITY AND AGREE ONLY ABOVE A THRESHOLD: `ln(1 + 1/lambda) (LOGNORMAL APPROXIMATION), 1/lambda (DELTA METHOD APPROXIMATION) and (TRIGAMMA FUNCTION) GIVE SIMILAR RESULTS WHEN LAMBDA IS GREATER THAN 2. Our recommendation is to use the trigamma function.` Three names, one target, and a condition under which the choice stops mattering Quotation withheld because this record is not cleared for redistribution. THEIR OWN APPROXIMATION IS NOT THE BEST ONE, AND THEY SAY SO: `A comparison among equations (6.7)-(6.9) is also shown in electronic supplementary material, appendix S3 (it turns out equation (6.8) gives the best approximation). Simulation will give the most accurate conversions when no exact solutions are available.` Equation 6.7 is theirs; 6.8 is McCulloch et al.'s Quotation withheld because this record is not cleared for redistribution. AND A NEIGHBOURING STATISTIC IS ADMITTED INTO THE FAMILY WITH A HEDGE IN PARENTHESES: the trigamma function has been used for `heritability (which can be seen as a type of ICC although in a strict sense, it is not)`. A kindhood claim and its denial in eleven words - the corpus's `is-a` cannot carry that, so it is here in prose Quotation withheld because this record is not cleared for redistribution. and one field renames the statistic outright: `a type of ICC is often referred to as repeatability R, where the grouping factor is often individuals that have been phenotyped repeatedly` - ecology and evolution's name for what the rest of the literature calls an intra-class correlation Quotation withheld because this record is not cleared for redistribution. deriving the same observation-level variance three independent ways - delta method, lognormal approximation, trigamma function - and comparing them eliminates: a variance term that is an artefact of one derivation; they agree when lambda > 2 and the trigamma is recommended where it applies · outcome: held Quotation withheld because this record is not cleared for redistribution. comparing their own bias-correction approximation against McCulloch et al.'s and Zeger et al.'s, and against simulation eliminates: recommending their own formula without testing it - and the test says somebody else's is better · outcome: failed Quotation withheld because this record is not cleared for redistribution. requiring that the method used to obtain lambda be reported eliminates: two papers reporting 'R2' from the same data and the same model with different numbers, because they estimated lambda differently · outcome: partial Quotation withheld because this record is not cleared for redistribution. marginal R 2 marginal R 2 Quotation withheld because this record is not cleared for redistribution. conditional R 2 conditional R 2 Quotation withheld because this record is not cleared for redistribution. observation-level variance observation-level variance Quotation withheld because this record is not cleared for redistribution. distribution-specific variance distribution-specific variance Quotation withheld because this record is not cleared for redistribution. median median Quotation withheld because this record is not cleared for redistribution. mean mean Quotation withheld because this record is not cleared for redistribution. `marginal R2` - the proportion of total variance explained by the FIXED effects only; `conditional R2` - by fixed AND random. They `differ only in whether the random effect variance is included in the numerator` Quotation withheld because this record is not cleared for redistribution. `observation-level variance` (this paper) - the generalized residual term, explicitly renamed `rather than the residual variance` while KEEPING THE NOTATION, and not to be confused with the variance of an observation-level random effect Quotation withheld because this record is not cleared for redistribution. `distribution-specific variance` - what 1/lambda would have been called in the authors' earlier papers; kept as a separate concept for binomial distributions and declared not analogous Quotation withheld because this record is not cleared for redistribution. `repeatability R` (ECOLOGY AND EVOLUTION) - a type of intra-class correlation where the grouping factor is individuals measured repeatedly Quotation withheld because this record is not cleared for redistribution. `adjusted ICC` - and the condition on the word: an ICC may only be called adjusted if the variance it is adjusted for is `MORE THAN 0 AND OMITTED FROM THE CALCULATION` Quotation withheld because this record is not cleared for redistribution. extends → the authors' own earlier definitions of R2_GLMM and ICC_GLMM (Nakagawa & Schielzeth 2013; Nakagawa, Johnson & Schielzeth), which covered only binomial and Poisson families - 'It turns out that binomial and Poisson distributions are special cases' Quotation withheld because this record is not cleared for redistribution. contradicts → their own earlier distinction between ICC on the latent scale and on the observation scale, which 'turns out to be strictly appropriate only for binomial distributions but not for Poisson distributions' Quotation withheld because this record is not cleared for redistribution. marginal R 2 —distinct-from→ conditional R 2 marginal R 2 conditional R 2 the relation itself is separately evidenced · strength of the relation: 0.95 · note: 'As MARGINAL AND…' One name for two numbers, kept apart by a subscript a reader may not carry across papers · the relation stated: the two differ ONLY in whether the random-effect variance is in the numerator - and both are reported as 'R2' Quotation withheld because this record is not cleared for redistribution. observation-level variance —distinct-from→ distribution-specific variance observation-level variance distribution-specific variance the relation itself is separately evidenced · strength of the relation: 0.95 · note: 'ARE ANALOGOUS…' The authors previously called one of these by the other's name - 'note that we would have called 1/lambda THE DISTRIBUTION-SPECIFIC VARIANCE' - and the edge records the distinction they drew afterwards · the relation stated: the two observation-level variances are analogous to each other and the distribution-specific variance is not - which is why the term was renamed Quotation withheld because this record is not cleared for redistribution. median —distinct-from→ mean median mean the relation itself is separately evidenced · strength of the relation: 0.95 · note: 'In fact, exp(beta0) gives the median value of y rather than the mean of y, assuming a Poisson distribution. Thus, the use of exp(beta0) will often overestimate, providing smaller estimates of R2 AND ICC ... when lambda is small, the difference can be substantial so the choice of the method needs to be reported for reproducibility.' A textbook distinction with a reporting requirement attached to it · the relation stated: exp(beta0) estimates the MEDIAN where the formula requires the MEAN, so using it systematically shrinks R2 and ICC Quotation withheld because this record is not cleared for redistribution. A PUBLISHED CONDITION ON A WORD INSIDE A STATISTIC'S NAME: `the ICC should not be called `ADJUSTED` ... that variance must be more than 0 and omitted from the ICC calculation` A PUBLISHED CONDITION ON A WORD INSIDE A STATISTIC'S NAME: `the ICC should not be called `ADJUSTED` ... that variance must be more than 0 and omitted from the ICC calculation` Quotation withheld because this record is not cleared for redistribution. AND A RENAMING WHERE THE NOTATION DELIBERATELY DOES NOT FOLLOW: `we refer to this generalized version ... as `THE OBSERVATION-LEVEL VARIANCE` here rather than the residual variance (but we keep the notation)` AND A RENAMING WHERE THE NOTATION DELIBERATELY DOES NOT FOLLOW: `we refer to this generalized version ... as `THE OBSERVATION-LEVEL VARIANCE` here rather than the residual variance (but we keep the notation)` Quotation withheld because this record is not cleared for redistribution. AND THE AUTHORS NAME WHAT THEY USED TO CALL IT: `note that we would have called 1/lambda THE DISTRIBUTION-SPECIFIC VARIANCE` AND THE AUTHORS NAME WHAT THEY USED TO CALL IT: `note that we would have called 1/lambda THE DISTRIBUTION-SPECIFIC VARIANCE` Quotation withheld because this record is not cleared for redistribution. AND THE NAMING CHOICE MOVES EVERY NUMBER IN A KNOWN DIRECTION: `the minimum value of 1/(p(1-p)) IS 4, which is larger than pi^2/3 = 3.29, meaning that the use of 1/p(1-p) IN R2 and ICC for binary data will always produce larger values` AND THE NAMING CHOICE MOVES EVERY NUMBER IN A KNOWN DIRECTION: `the minimum value of 1/(p(1-p)) IS 4, which is larger than pi^2/3 = 3.29, meaning that the use of 1/p(1-p) IN R2 and ICC for binary data will always produce larger values` Quotation withheld because this record is not cleared for redistribution. the authors' two earlier recommendations for estimating lambda, exp(beta0) and the mean of observed values, are often inconsistent the authors' two earlier recommendations for estimating lambda, exp(beta0) and the mean of observed values, are often inconsistent Quotation withheld because this record is not cleared for redistribution. AND A DISTINCTION WITHDRAWN FOR ONE FAMILY AND KEPT FOR ANOTHER: `Such a distinction turns out to be strictly appropriate only for binomial distributions but not for Poisson distributions (and probably also not for other non-Gaussian distributions)` AND A DISTINCTION WITHDRAWN FOR ONE FAMILY AND KEPT FOR ANOTHER: `Such a distinction turns out to be strictly appropriate only for binomial distributions but not for Poisson distributions (and probably also not for other non-Gaussian distributions)` Quotation withheld because this record is not cleared for redistribution. `quantifies the proportion of variance explained by a statistical model` - and `ESTIMATING R2 FOR GENERALIZED LINEAR MIXED MODELS REMAINS CHALLENGING` coefficient of determination Quotation withheld because this record is not cleared for redistribution. `For an ICC value to be adjusted for a source of variance, that variance must be more than 0 and omitted from the ICC calculation` - otherwise `the ICC should not be called `ADJUSTED`` adjusted ICC Quotation withheld because this record is not cleared for redistribution. so the choice of term makes every R2 and ICC for binary data larger or smaller, in a fixed direction floor of the two candidate variance terms for binary data Quotation withheld because this record is not cleared for redistribution. below that the delta method, lognormal approximation and trigamma function diverge where the three derivations agree Quotation withheld because this record is not cleared for redistribution. `Simulation will give the most accurate conversions when no exact solutions are available` which approximation is best Quotation withheld because this record is not cleared for redistribution. the proportion of variance explained by a grouping factor - called `repeatability R` in ecology and evolution, and a statistic whose `adjusted` variant carries a published condition on the word intra-class correlation coefficient Quotation withheld because this record is not cleared for redistribution. the excess variation the residual term was supposed to separate out - and which `cannot be clearly separated` from the distribution-specific part in most families overdispersion Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1098/rsif.2017.0213.