Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on Voting Behavior and Political Opinions

Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on Voting Behavior and Political Opinions Alan S Gerber Dean Karlan Daniel E. Bergan

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on Voting Behavior and Political Opinions 2009 American Economic Journal Applied Economics 10.1257/app.1.2.35 gerber2009-does-the-media-matter-a-field-expe gerber2009-does-the-media-matter-a-field-expe.txt, the project's staged copy 575519602d06dbde0782560821e6c8f481cc82e1 open-access political-economy

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them existence Quotation withheld because this record is not cleared for redistribution. media effects on voting behavior Quotation withheld because this record is not cleared for redistribution. behavioural-experiment behavioural-experiment Quotation withheld because this record is not cleared for redistribution. mechanisms Online processing: readers update their candidate evaluations as information arrives but do not retain the facts that moved them, which is why opinions shift while measured knowledge does not Quotation withheld because this record is not cleared for redistribution. mechanisms Shared content beats slant: both papers covered war casualties and administration scandals during a period bad for Republicans, so what the two papers had in common moved readers in the same direction Quotation withheld because this record is not cleared for redistribution. mechanisms Continued exposure after the election — leftover subscription weeks plus voluntary renewals — carries the treatment forward and explains why turnout moved in 2006 but not 2005 Quotation withheld because this record is not cleared for redistribution. Small sample, so treatment effects are estimated imprecisely. what this limits: the precision of every treatment effect: THE MOST IMPORTANT [limitation] OF WHICH IS THE RELATIVELY SMALL SAMPLE SIZE Quotation withheld because this record is not cleared for redistribution. Delivery was randomised but actual reading could not be verified. what this limits: the interpretation of the treatment itself: WE KNOW WHICH HOUSEHOLDS RECEIVED NEWSPAPERS, [but] WE CANNOT BE SURE THAT THE NEWSPAPERS WERE READ Quotation withheld because this record is not cleared for redistribution. Attrition on unobservables could bias the estimates. what this limits: the unbiasedness of the estimates: IF THERE IS ATTRITION BASED ON UNOBSERVED VARIABLES THAT ARE CORRELATED WITH THE OUTCOME MEASURES BUT NOT PREDICTED BY THE OBSERVABLES, OUR RESULTS MAY BE BIASED Quotation withheld because this record is not cleared for redistribution. The sample frame excludes existing subscribers, so the estimated effect applies to an unusually low-media-exposure population. what this limits: the population the effect applies to: WE ARE STUDYING INDIVIDUALS WHO DO NOT ALREADY SUBSCRIBE TO A NEWSPAPER, HENCE ARE EXAMINING THE EFFECT OF EXPOSING INDIVIDUALS WHO, ON AVERAGE, ARE LESS EXPOSED TO THE MEDIA THAN THE A[verage] Quotation withheld because this record is not cleared for redistribution. The turnout effect appeared only in 2006 and was of borderline significance, so both the effect and its mechanism are held provisional. what this limits: the turnout result specifically: BOTH THE EXISTENCE OF AN EFFECT AND ITS POTENTIAL MECHANISM SHOULD BE TREATED GUARDEDLY UNTIL FURTHER STUDY Quotation withheld because this record is not cleared for redistribution. What was randomised is delivery, not reading: the authors state they cannot be sure the newspapers were read. what this limits: reading the estimates as effects of newspaper EXPOSURE: the randomisation assigned delivery, and the authors state they cannot be sure the papers were read Quotation withheld because this record is not cleared for redistribution. The authors name small sample size as their most important limitation, with consequences for the standard errors. what this limits: treating a null result here as evidence of no effect: the authors name the relatively small sample size as their most important limitation, with 1,081 survey interviews and 2,571 merged records Quotation withheld because this record is not cleared for redistribution. The orthogonality claim is conditional: unobservables correlated with the outcome but unpredicted by the observables would still bias the result. what this limits: the randomisation as guaranteeing unbiasedness of these particular estimates: the authors state results may still be biased through unobservables not predicted by the observed covariates Quotation withheld because this record is not cleared for redistribution. Just under a third of the assigned sample supplied the survey outcomes. what this limits: the follow-up survey as covering the assigned sample: 1,081 of the 3,347 individuals were reinterviewed for it Quotation withheld because this record is not cleared for redistribution. WHAT WAS RANDOMISED IS DELIVERY, NOT READING, and the authors say so plainly: they know which households received newspapers but cannot be sure the newspapers were read. The treatment the design controls is not the treatment the theory is about, and the gap is bridged by three self-reported survey measures. Quotation withheld because this record is not cleared for redistribution. The online-processing model of attitude formation, taken from psychological work on impression formation and persuasion, is used to explain why candidate preference moved without any measurable gain in factual knowledge Quotation withheld because this record is not cleared for redistribution. households randomly assigned to a third arm that was sent NEITHER newspaper, alongside the Post and Times subscription arms eliminates: prior political interest and everything else that differs between people who read a paper and people who do not - random assignment makes the untreated arm comparable in expectation, which observational media studies cannot do · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. media slant media slant Quotation withheld because this record is not cleared for redistribution. field experiment field experiment Quotation withheld because this record is not cleared for redistribution. newspaper provision newspaper provision Quotation withheld because this record is not cleared for redistribution. media exposure media exposure Quotation withheld because this record is not cleared for redistribution. turnout turnout Quotation withheld because this record is not cleared for redistribution. treatment - AS ASSIGNMENT: which households were sent a free subscription, which is the thing actually randomised Quotation withheld because this record is not cleared for redistribution. treatment - AS EXPOSURE: whether the newspaper was read, which the paper states it cannot be sure of and can only proxy through three survey measures Quotation withheld because this record is not cleared for redistribution. media effect - separated into SLANT and EXPOSURE, with the paper reporting that slant mattered less than exposure in this case Quotation withheld because this record is not cleared for redistribution. effect of newspapers - measured on four outcomes that came apart: NO effect on political knowledge, stated opinions, or 2005 turnout; an effect on candidate support; and weaker evidence of an effect on 2006 turnout Quotation withheld because this record is not cleared for redistribution. replicates → Gentzkow (2006) Quotation withheld because this record is not cleared for redistribution. media slant —distinct-from→ media exposure media slant media exposure differs in: WHICH ONE THE EXPERIMENT COULD MOVE THE OUTCOME WITH. Both papers - the conservative Times and the liberal Post - produced more support for the DEMOCRATIC candidate, so the effect cannot be attributed to slant. The design separates the two by giving opposite slants and finding the same direction · strength of the relation: 0.9 · note: THE FINDING IS A NULL ON THE VARIABLE THE STUDY WAS DESIGNED AROUND. Slant is what the two-newspaper design exists to test, and it is the one thing that did not matter - 'media slant mattered less in this case than media exposure'. Quotation withheld because this record is not cleared for redistribution. newspaper provision —causes→ turnout newspaper provision turnout strength of the relation: 0.6 — the record marks this weak · note: Reported at the weakest grade the paper uses - 'SOME EVIDENCE from voting records ALSO SUGGESTS' - and the record already carries a limitation saying the turnout effect appeared only in 2006 and was of borderline significance. Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but 'Some evidence ... suggests' increased 2006 turnout; hedged. Quotation withheld because this record is not cleared for redistribution. ordinary least squares regression and probit ordinary least squares regression and probit Quotation withheld because this record is not cleared for redistribution. chi-squared test chi-squared test Quotation withheld because this record is not cleared for redistribution. multinomial logit regression multinomial logit regression Quotation withheld because this record is not cleared for redistribution. standard error standard error Quotation withheld because this record is not cleared for redistribution. null hypothesis significance test / minimum detectable effect null hypothesis significance test / minimum detectable effect Quotation withheld because this record is not cleared for redistribution. standardization by standard deviations from the mean standardization by standard deviations from the mean Quotation withheld because this record is not cleared for redistribution. p-value p-value Quotation withheld because this record is not cleared for redistribution. Ten weeks of a free newspaper subscription had no detectable effect on political knowledge, stated opinions, or turnout in the 2005 election. Quotation withheld because this record is not cleared for redistribution. Receiving either newspaper increased support for the Democratic candidate, so exposure to news rather than the paper's slant drove the effect. Quotation withheld because this record is not cleared for redistribution. Receiving the Post raised the probability of voting for the Democrat by 11.2 percentage points among those who voted and by 7.2 percentage points across all respondents. Quotation withheld because this record is not cleared for redistribution. The conservative Washington Times also raised the probability of a Democratic vote, contrary to what a slant-based account predicts. Quotation withheld because this record is not cleared for redistribution. The estimated effects of the two papers cannot be statistically distinguished from one another. Quotation withheld because this record is not cleared for redistribution. Administrative records show turnout in the 2006 election 2.8 percentage points higher in the treatment groups, an effect of only borderline significance. Quotation withheld because this record is not cleared for redistribution. Neither newspaper improved respondents' ability to answer factual questions about recent news, individually or in aggregate. Quotation withheld because this record is not cleared for redistribution. About 17 percent of the Post treatment group had subscribed three months after the free subscription ended. Quotation withheld because this record is not cleared for redistribution. Randomization succeeded: no baseline characteristic differed significantly across the three groups, the weakest balance being on gender. Quotation withheld because this record is not cleared for redistribution. The study is underpowered for small effects: turnout effects below 3.6 points, Democratic-vote effects below 7 points, and Post-versus-Times differences below 8.6 points could not have been detected. Quotation withheld because this record is not cleared for redistribution. intent to treat effects intent to treat effects Quotation withheld because this record is not cleared for redistribution. Broad Policy index Broad Policy index Quotation withheld because this record is not cleared for redistribution. Specific Issue Index Specific Issue Index Quotation withheld because this record is not cleared for redistribution. free subscription treatment free subscription treatment Quotation withheld because this record is not cleared for redistribution. voter turnout voter turnout Quotation withheld because this record is not cleared for redistribution. strata strata Quotation withheld because this record is not cleared for redistribution. survey date survey date Quotation withheld because this record is not cleared for redistribution. surveyor surveyor Quotation withheld because this record is not cleared for redistribution. 3,347 — Households that reported taking neither newspaper at baseline and were randomized into the three arms Quotation withheld because this record is not cleared for redistribution. 1,081 / 2,571 — Post-election survey interviews, and the number of subjects matched to administrative voting records Quotation withheld because this record is not cleared for redistribution. 11.2 percentage points ( p , 0.014) — Estimated increase in the probability of voting for the Democratic candidate caused by receiving the Washington Post, among those who reported voting Quotation withheld because this record is not cleared for redistribution. 2.8 percentage point — Higher 2006 turnout in the treatment groups in administrative records, standard error 1.6 points Quotation withheld because this record is not cleared for redistribution. approximately 17 percent — Share of the Post treatment group that had bought its own subscription three months after the free trial ended Quotation withheld because this record is not cleared for redistribution. 6 percent — Share of treatment-group households that cancelled the free subscription, kept in the sample under intent to treat Quotation withheld because this record is not cleared for redistribution. approximately 92 percentage points — Actual increase in Post exposure produced by the treatment, rather than 100 points, because some control and Times households already received the Post on Sundays Quotation withheld because this record is not cleared for redistribution. 1,081 of 3,347 — individuals reinterviewed for the post-election follow-up survey Quotation withheld because this record is not cleared for redistribution. 2,571 — sample size for the data merged with administrative voting records, named as a limitation Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1257/app.1.2.35.