Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests Stefan Wager Susan Athey

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests 2018 JASA / arXiv 10.1080/01621459.2017.1319839 1510.04342 wager2018-causal-forests wager2018-causal-forests.txt, the project's staged copy 1ffc1d5a85b61f3f18445a8fa3eaf3fc61958843 arxiv-open statistics

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them method Quotation withheld because this record is not cleared for redistribution. heterogeneous treatment effect estimation Quotation withheld because this record is not cleared for redistribution. mathematical-proof mathematical-proof Quotation withheld because this record is not cleared for redistribution. mechanisms treating a random forest as an adaptive nearest-neighbor estimator yields consistent, asymptotically Gaussian treatment-effect estimates, enabling confidence intervals Quotation withheld because this record is not cleared for redistribution. results require the unconfoundedness assumption what this limits: every causal estimate: the results require the POTENTIAL OUTCOMES FRAMEWORK WITH UNCONFOUNDEDNESS Quotation withheld because this record is not cleared for redistribution. The choice between double-sample forests and propensity forests was made by hand for each simulation setting rather than automatically. what this limits: the simulation comparison specifically: WE MANUALLY CHOSE WHETHER TO USE DOUBLE-SAMPLE FORESTS OR PROPENSITY FORESTS, DEPENDING ON WHICH PROCEDURE SEEMED MORE APPROPRIATE Quotation withheld because this record is not cleared for redistribution. The asymptotic normality proof does not cover classification forests built by voting. what this limits: the asymptotic normality result: EXTENDING OUR RESULTS TO VOTING TREES IS LEFT AS FURTHER WORK Quotation withheld because this record is not cleared for redistribution. The theory is not established for high-dimensional settings where variable selection matters. what this limits: the asymptotic theory in the regime most applications live in: WE LEAVE A STUDY OF ASYMPTOTIC NORMALITY OF RANDOM FORESTS IN HIGH DIMENSIONS TO FUTURE WORK Quotation withheld because this record is not cleared for redistribution. Residual bias degrades confidence-interval coverage, which the authors flag as the immediate open problem. what this limits: the confidence intervals as reported: AN IMMEDIATE CHALLENGE IS TO CONTROL THE BIAS OF CAUSAL FORESTS TO ACHIEVE BETTER COVERAGE Quotation withheld because this record is not cleared for redistribution. Empirically, nominal coverage of the intervals holds only up to moderate dimension. what this limits: any use of the intervals above ten covariates: OUR METHOD ACHIEVES NOMINAL COVERAGE UP TO d = 10, AFTER WHICH THE PERFORMANCE OF THE CONFIDENCE INTERVALS STARTS TO DECAY Quotation withheld because this record is not cleared for redistribution. Two asymptotic-normality results for random forests can look qualitatively the same and differ on whether the subsample grows fast enough for the squared bias to vanish faster than the variance. When it does not, the Gaussian limit is real and the confidence intervals built from it do not cover. A reader taking 'asymptotically normal' as the licence for an interval has no way to see this from the claim alone. Quotation withheld because this record is not cleared for redistribution. extending Breiman random forests to estimate heterogeneous treatment effects with valid statistical inference Quotation withheld because this record is not cleared for redistribution. treatment effect treatment effect Quotation withheld because this record is not cleared for redistribution. unconfoundedness unconfoundedness Quotation withheld because this record is not cleared for redistribution. honesty honesty Quotation withheld because this record is not cleared for redistribution. causal forest causal forest Quotation withheld because this record is not cleared for redistribution. double-sample trees double-sample trees Quotation withheld because this record is not cleared for redistribution. subsample size subsample size Quotation withheld because this record is not cleared for redistribution. confidence intervals confidence intervals Quotation withheld because this record is not cleared for redistribution. test set test set Quotation withheld because this record is not cleared for redistribution. double-sample tree double-sample tree Quotation withheld because this record is not cleared for redistribution. unconfoundedness - the assumption that makes nearby observations behave as if they came from a randomized experiment, which is what licenses matching Quotation withheld because this record is not cleared for redistribution. honesty - a condition the authors name and impose: the tree is grown on one subsample and its leaf predictions are estimated on a different one Quotation withheld because this record is not cleared for redistribution. treatment effect - a quantity not estimable from the observed data at all without further restrictions on the data generating distribution Quotation withheld because this record is not cleared for redistribution. evaluation - FOR A CAUSAL ESTIMAND THE STANDARD MACHINE-LEARNING SENSE IS UNAVAILABLE: only one potential outcome per unit is ever observed, so 'direct test-set evaluation is in general impossible' and asymptotic theory does the work that held-out prediction error does elsewhere Quotation withheld because this record is not cleared for redistribution. sample splitting - AS COSTED HERE: the usual objection is that it wastes half the training data, and the answer is that the waste is per TREE and recovered across the ENSEMBLE, since the I/J split is re-randomised over each subsample so every point serves both roles in some tree Quotation withheld because this record is not cleared for redistribution. 'honesty' - A MORAL WORD GIVEN A BICONDITIONAL DEFINITION, AND THE DEFINITION IS A PROHIBITION ON DOUBLE USE. 'A tree is honest if, for each training example i, it only uses the response Yi to estimate the within-leaf treatment effect τ ... or to decide where to place the splits, but not both.' Nothing about the definition is moral - it forbids using one observation twice for two purposes. The name imports an intuition (a tree that does not cheat) that the formal content neither needs nor supports, and it travels: a reader meeting 'honest trees' in a later paper will reconstruct the intuition, not the disjunction. This record is the second half of the typology's mechanism 25 - 'athey2016' -> 'wager2018' tightening this very term. STAGED-TEXT NOTE: LaTeX-adjacency fusion in the staged text, declared rather than hidden: 'responseYi' is 'response Y_i' and 'effectτ' is 'effect τ', the space lost where prose meets a math symbol. The span is kept because it is the paper's ONLY statement of the definition and the fusion is legible; re-slicing around the symbols would remove the quantifier the definition turns on. The schema has no place for this note on a sense, so the disclosure lives here instead. Quotation withheld because this record is not cleared for redistribution. `double-sample tree` - THE NAME DESCRIBES THE MECHANISM AND NOT THE PROPERTY IT BUYS. `Our first algorithm, which we call a double-sample tree, achieves honesty by dividing its training subsample into two halves.` Honesty is the requirement; two samples is one way to meet it. A name for the means, in a paper whose theorems are all about the end - which is why `honesty` and `double-sample` are not interchangeable even though this paper's only honest algorithm is a double-sample one. Quotation withheld because this record is not cleared for redistribution. extends → random forests (Breiman) for statistical inference Quotation withheld because this record is not cleared for redistribution. extends → Breiman's random forest algorithm [Breiman, 2001a] Quotation withheld because this record is not cleared for redistribution. builds-on → the honest causal trees of Athey and Imbens [2016] Quotation withheld because this record is not cleared for redistribution. extends → the asymptotic normality result of Mentch and Hooker [2016] Quotation withheld because this record is not cleared for redistribution. builds-on → the infinitesimal jackknife for random forests of Efron [2014] and Wager et al. [2014] Quotation withheld because this record is not cleared for redistribution. honesty —operationalised-as→ double-sample trees honesty double-sample trees strength of the relation: 0.9 · note: The condition is named first and realised second: '…1, can always be used…', and propensity trees give '…WITHOUT SAMPLE SPLITTING…' - so one condition has two constructions, only one of which splits the sample. Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the span says double-sample trees can be used to build honest trees; operationalised-as is loose. Quotation withheld because this record is not cleared for redistribution. subsample size —causes→ confidence intervals subsample size confidence intervals differs in: THE RATE AT WHICH THE SUBSAMPLE GROWS. 'the bias of the random forest decays as 1/sqrt(s_n), while its variance decays as s_n/n. IF s_n/sqrt(n) -> 0, the squared bias decays SLOWER than the variance, and so confidence intervals built using the resulting Gaussian limit distribution WILL NOT COVER' the target. Two asymptotic-normality results for random forests differ on this one exponent and on nothing a reader would notice from the statement of either theorem · strength of the relation: 0.9 · note: THE PAPER NAMES THE PARAMETER ITS DISAGREEMENT TURNS ON AND SAYS WHAT GOES WRONG: '…[2016] may appear qualitatively similar to ours, it cannot…'. A Gaussian limit is not by itself a licence to build an interval - the bias has to vanish faster than the standard error, which is a condition on a tuning parameter, not on the theorem. Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says the CIs fail to cover if the subsample size condition holds; causes is loose. Quotation withheld because this record is not cleared for redistribution. 2 — a treatment effect is a difference between two of them Quotation withheld because this record is not cleared for redistribution. 1 — survival threshold in a prediction example Quotation withheld because this record is not cleared for redistribution. causal forests are pointwise consistent and asymptotically Gaussian Quotation withheld because this record is not cleared for redistribution. first results letting any random forest be used for valid statistical inference Quotation withheld because this record is not cleared for redistribution. causal forests are more powerful than nearest-neighbor matching Quotation withheld because this record is not cleared for redistribution. the causal forest's advantage over kNN grows with the number of covariates Quotation withheld because this record is not cleared for redistribution. treatment effect treatment effect Quotation withheld because this record is not cleared for redistribution. number of covariates number of covariates Quotation withheld because this record is not cleared for redistribution. confidence intervals confidence intervals Quotation withheld because this record is not cleared for redistribution. honest trees honest trees Quotation withheld because this record is not cleared for redistribution. causal forest causal forest Quotation withheld because this record is not cleared for redistribution. potential outcomes framework with unconfoundedness potential outcomes framework with unconfoundedness Quotation withheld because this record is not cleared for redistribution. honesty honesty Quotation withheld because this record is not cleared for redistribution. heterogeneous treatment effects heterogeneous treatment effects Quotation withheld because this record is not cleared for redistribution. ANOVA ANOVA Quotation withheld because this record is not cleared for redistribution. Bayesian Bayesian Quotation withheld because this record is not cleared for redistribution. bootstrap bootstrap Quotation withheld because this record is not cleared for redistribution. confidence-interval confidence-interval about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. mean mean Quotation withheld because this record is not cleared for redistribution. sample-size sample-size about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1080/01621459.2017.1319839.