Recursive Partitioning for Heterogeneous Causal Effects

Recursive Partitioning for Heterogeneous Causal Effects Susan Athey Guido Imbens

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Recursive Partitioning for Heterogeneous Causal Effects 2016 PNAS / arXiv 10.1073/pnas.1510489113 1504.01132 athey2016-causal-trees athey2016-causal-trees.txt, the project's staged copy 85475cc600c207471c9cb7c2b05717aab7d2a550 arxiv-open economics

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them method Quotation withheld because this record is not cleared for redistribution. heterogeneous treatment effects Quotation withheld because this record is not cleared for redistribution. simulation simulation Quotation withheld because this record is not cleared for redistribution. mechanisms honest estimation: use one sample to construct the tree partition and a separate sample to estimate treatment effects, removing overfitting bias Quotation withheld because this record is not cleared for redistribution. requires the unconfoundedness (conditional randomization) assumption Quotation withheld because this record is not cleared for redistribution. Sample splitting for honest estimation costs precision, which is only partly offset by the removal of bias. what this limits: the precision of honest estimates: THERE IS A LOSS OF PRECISION DUE TO SAMPLE SPLITTING, offset in fit by the elimination of bias Quotation withheld because this record is not cleared for redistribution. In the simulations honest estimation is measurably worse than adaptive estimation on treatment-effect mean-squared error. what this limits: the recommendation itself, with the cost measured: THERE IS A COST TO HONEST ESTIMATION IN TERMS OF MSE-tau, VARYING BY DESIGN AND ESTIMATOR Quotation withheld because this record is not cleared for redistribution. The no-interference (SUTVA) assumption the methods rest on can fail when units are connected in a network. what this limits: any application where units interact: the no-interference assumption MAY BE VIOLATED IN SETTINGS WHERE SOME UNITS ARE CONNECTED THROUGH NETWORKS Quotation withheld because this record is not cleared for redistribution. Hybrid variants (adaptive tree construction combined with honest leaf estimation) are left unexamined. what this limits: the hybrid variants: WE DO NOT CONSIDER THOSE VARIATIONS IN THIS PAPER Quotation withheld because this record is not cleared for redistribution. The quantity the method estimates is never observed for any unit in the data. what this limits: any treatment-effect estimate produced by this method: unit-level causal effects are not observed for any unit, so the target is never available to check an estimate against Quotation withheld because this record is not cleared for redistribution. The paper's central adjective is a stipulated property of data use, not a claim about candour. what this limits: the word 'honest' as used throughout this paper: a model is honest if it does not use the same information for selecting the model structure and for estimation given that structure Quotation withheld because this record is not cleared for redistribution. The improvement claimed for honesty is measured in coverage, and the paper names what it costs. what this limits: the benefit reported for honest estimation: it is an improvement in confidence-interval coverage, attaining nominal rates, and the authors describe the cost as not much sacrifice in fitting treatment effects Quotation withheld because this record is not cleared for redistribution. The comparison class is conventional tree estimation under a name the paper assigns it. what this limits: the contrast drawn here between honest and adaptive estimation: adaptive is the label the authors give conventional CART, in which the same data builds the partition and estimates leaf effects Quotation withheld because this record is not cleared for redistribution. regression-tree (recursive partitioning) methods adapted to estimate heterogeneous treatment effects Quotation withheld because this record is not cleared for redistribution. three simulation designs differing in whether noise covariates are present and in how covariates enter the treatment effect eliminates: that the advantage of honest estimation depends on one particular data-generating design, in particular on the absence of noise covariates · outcome: held Quotation withheld because this record is not cleared for redistribution. honest estimation honest estimation Quotation withheld because this record is not cleared for redistribution. adaptive adaptive Quotation withheld because this record is not cleared for redistribution. unconfoundedness unconfoundedness Quotation withheld because this record is not cleared for redistribution. mean-squared error mean-squared error Quotation withheld because this record is not cleared for redistribution. coverage coverage Quotation withheld because this record is not cleared for redistribution. 'honest' - A STIPULATED PROPERTY OF DATA USE: a model is honest when the information selecting its structure is not the information estimating its parameters, which is sample splitting and nothing about truthfulness, candour or good faith - all of which the word carries anyway Quotation withheld because this record is not cleared for redistribution. 'adaptive' - THE NAME GIVEN TO THE ALTERNATIVE: conventional CART, where one sample does both jobs, is labelled adaptive here so that the two can be compared; the word is the paper's, applied to a practice that did not call itself that Quotation withheld because this record is not cleared for redistribution. the estimand - UNOBSERVABLE FOR EVERY UNIT: unit-level causal effects are not observed for any unit at all, so unlike an ordinary prediction problem there is no held-out truth anywhere in the data against which a per-unit estimate could be scored Quotation withheld because this record is not cleared for redistribution. builds-on → regression tree methods (machine learning) Quotation withheld because this record is not cleared for redistribution. builds-on → standard CART regression trees ([4], [3]) Quotation withheld because this record is not cleared for redistribution. builds-on → the Rubin Causal Model / potential outcome framework ([19], [11], [14]) Quotation withheld because this record is not cleared for redistribution. EVIDENCE CORRECTED (RE-GROUNDED) · honest estimation —contrasted-with→ adaptive honest estimation adaptive EVIDENCE CORRECTED: RE-GROUNDED. The old span named only one endpoint. This sentence names both and states the contrast WITH ITS DIRECTION - honest estimation improves on adaptive, not merely differs. — strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. honest estimation —contrasted-with→ adaptive honest estimation adaptive strength of the relation: 1 · note: RE-GROUNDS another row of this record: the old span named only ONE endpoint. Quotation withheld because this record is not cleared for redistribution. Honest estimation splits the sample: one half builds the partition, the other estimates each subpopulation's treatment effect. Quotation withheld because this record is not cleared for redistribution. covariates covariates Quotation withheld because this record is not cleared for redistribution. treatment effects treatment effects Quotation withheld because this record is not cleared for redistribution. honest estimation honest estimation Quotation withheld because this record is not cleared for redistribution. heterogeneous treatment effects heterogeneous treatment effects about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. Bayesian Bayesian Quotation withheld because this record is not cleared for redistribution. confidence-interval confidence-interval Quotation withheld because this record is not cleared for redistribution. false-discovery-rate false-discovery-rate Quotation withheld because this record is not cleared for redistribution. hypothesis-testing hypothesis-testing about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. mean mean Quotation withheld because this record is not cleared for redistribution. null-hypothesis null-hypothesis about this quote: THE QUOTE STOPS WHERE THE EXTRACTED TEXT GOES OUT OF ORDER: the extraction splices another line of the page ('If the two leaves are denoted L (Left) and') into this sentence, so the quote ends before that line, at 'the two potential', and the end of the sentence is not quoted. Quotation withheld because this record is not cleared for redistribution. p-value p-value Quotation withheld because this record is not cleared for redistribution. regression regression about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. sample-size sample-size Quotation withheld because this record is not cleared for redistribution. t-test t-test Quotation withheld because this record is not cleared for redistribution. variance variance Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1073/pnas.1510489113.