Practical guidelines for rigor and reproducibility in preclinical and clinical studies on cardioprotection

Practical guidelines for rigor and reproducibility in preclinical and clinical studies on cardioprotection Hans Erik Bøtker Derek J. Hausenloy Ioanna Andreadou Salvatore Antonucci Kerstin Boengler Sean M. Davidson Soni Deshwal Yvan Devaux Fabio Di Lisa Moises Di Sante Panagiotis Efentakis Saveria Femminò David García‐Dorado Zoltán Giricz Borja Ibáñez Efstathios K. Iliodromitis Nina Kaludercic Petra Kleinbongard Markus Neuhäuser Michel Ovize Pasquale Pagliaro Michael Rahbek-Schmidt Marisol Ruiz‐Meana Klaus‐Dieter Schlüter Rainer Schulz Andreas Skyschally Catherine D. E. Wilder Derek M. Yellon Péter Ferdinandy Gerd Heusch

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Practical guidelines for rigor and reproducibility in preclinical and clinical studies on cardioprotection 2018 Basic Research in Cardiology 10.1007/s00395-018-0696-8 botker2018-practical-guidelines-for-rigor-and botker2018-practical-guidelines-for-rigor-and.txt, the project's staged copy 57b193ee4e1ccb9450142d5a02679648a7816c21 cc-by medicine

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them A THIRTY-AUTHOR CONSENSUS DEFINING WHAT COUNTS AS RIGOROUS in a field where thousands of preclinical studies produced exactly one unequivocal clinical translation - guidelines whose subject is the reliability of an evidence base, written by the people who produced it Quotation withheld because this record is not cleared for redistribution. what design, conduct, analysis and reporting practices would make cardioprotection studies reproducible enough to be worth translating Quotation withheld because this record is not cleared for redistribution. practical guidelines covering study design, randomisation, blinding, power, statistics, each experimental model with its own limitations, the omics technologies, and the design of clinical proof-of-concept and outcome trials Quotation withheld because this record is not cleared for redistribution. method Quotation withheld because this record is not cleared for redistribution. a field that cannot replicate itself, writing down what it will count as rigour Quotation withheld because this record is not cleared for redistribution. expert-consensus expert-consensus Quotation withheld because this record is not cleared for redistribution. THE PAPER DEFINES ITS OWN TWO WORDS AND MAKES ONE DEPEND ON THE OTHER: `we define rigor as standardized state-of-the-art design, conduct and reporting of a study, which is then a prerequisite for reproducibility, i.e. replication of results by another laboratory when performing exactly the same experiment.` Rigour is a property of the doing; reproducibility is a property of what happens next, somewhere else Quotation withheld because this record is not cleared for redistribution. AND THE FAILURE THAT MOTIVATES IT IS COUNTED: `literally thousands of experimental studies in various species and models ... so far there is only a single and very recent study, which has unequivocally translated cardioprotection to improved clinical outcome as the primary endpoint in patients` Quotation withheld because this record is not cleared for redistribution. AND A STANDARDISED MULTI-CENTRE REPLICATION FOUND ONE EFFECT OF THREE: `the CAESAR consortium ... in a highly standardized multi-center approach of preclinical studies identified only ischemic preconditioning, but not nitrite or sildenafil, when given as adjunct to reperfusion, to reduce infarct size` Quotation withheld because this record is not cleared for redistribution. AND TWO IMPLEMENTATIONS OF ONE NAMED TEST GIVE DIFFERENT ANSWERS: `Caveat - different statistical software packages work with slightly different assumptions and can come to different outcomes, as demonstrated for the Wilcoxon-Mann-Whitney`. Third consecutive record in this corpus where one construct measured two ways gives two answers - zook2016's aligners, ainsworth2020's five assays, and now a statistical test that has a name and a citation Quotation withheld because this record is not cleared for redistribution. AND THE SURROGATES DO NOT PREDICT THE THING THEY STAND IN FOR, with the real definition arriving in a parenthesis immediately afterwards: `None of the above-mentioned variable/surrogate endpoints is indeed predictive of clinical outcomes in patients with acute myocardial infarction, i.e. truly indicative of `cardioprotection` (which is primarily infarct size reduction)` Quotation withheld because this record is not cleared for redistribution. AND ONE DRUG IS THE COMMON DENOMINATOR OF THE FAILURES: `propofol specifically abrogates the protection by remote ischemic conditioning in patients undergoing elective coronary artery bypass graft surgery ... and seems to be a common denominator of all studies that failed to see protection`. A confounder that explains a literature's negative results, named Quotation withheld because this record is not cleared for redistribution. the choice of statistical test follows the shape of the data, not the field: one endpoint at one time point takes a t test or one-way ANOVA, sequential measurements take a two-way repeated-measures ANOVA Quotation withheld because this record is not cleared for redistribution. an improving standard of care raises the sample size a trial needs, because the event rate the intervention must beat keeps falling Quotation withheld because this record is not cleared for redistribution. the assumed effect size is imported from proof-of-concept studies - a 25% reduction in events is rationalised by 30% reductions in infarct size reported earlier Quotation withheld because this record is not cleared for redistribution. THE INSTRUMENT DECIDES WHAT IS FOUND, STATED THREE TIMES IN THREE TECHNOLOGIES. RNA: `it is not recommended to use different kits in the same study, since some will favor the isolation of short RNAs while others will enrich the RNA fraction with long molecules` Quotation withheld because this record is not cleared for redistribution. proteomics: `the large abundance proteins mask the changes of low-abundance proteins which also make a significant contribution to cellular signalling` - and the targeted alternative `loses the unbiased feature of proteomics`. Coverage and accuracy traded against each other, explicitly Quotation withheld because this record is not cleared for redistribution. and multiomics software: `their diversity in mathematical models and source data do not yet allow accurate comparison of different results` Quotation withheld because this record is not cleared for redistribution. AND THERE IS NO ACCEPTED METHOD FOR MISSING VALUES, QUOTED FROM SOMEBODY ELSE RATHER THAN ASSERTED: `Unfortunately, there is `NO METHODOLOGICAL APPROACH FOR HANDLING MISSING VALUES THAT IS UNIVERSALLY ACCEPTED IN ALL SITUATIONS`` Quotation withheld because this record is not cleared for redistribution. AND A COMMON PRACTICE IS REFUSED FOR A REASON THAT IS A ZERO-CLAIM ABOUT AN INSTRUMENT: `pretests are performed to test data for normality ... Those pretests cannot be recommended, since the failure to reject the null hypothesis does not imply that the null hypothesis holds, in particular since the power of the pretests is often low` Quotation withheld because this record is not cleared for redistribution. AND THE CHOICE OF DESIGN IS ADMITTED TO BE A COMPROMISE RATHER THAN A STANDARD: `The selection of the design ... always represents a compromise of a most rigorous approach vs. limited resources and feasibility. Clearly, an exploratory study, while aiming for novelty, is associated with a greater risk of false-positive findings ... whereas a prospective study aims for an existing specific hypothesis but provides more robust data` Quotation withheld because this record is not cleared for redistribution. and two authors of a rigour guideline hold commercial interests in the field, declared: shareholders in a company and a founder-CEO of a group of R&D companies. Not a defect - a disclosure, and it belongs next to the paper's subject the way hoste2006's `SOME MEMBERS OF OUR RESEARCH GROUP HAVE CONTRIBUTED TO THE CONSENSUS PROCESS BY WHICH RIFLE WAS DEVELOPED` does Quotation withheld because this record is not cleared for redistribution. cardiac mortality has fallen steadily for 25 years and is now very low, so the outcome a cardioprotection trial must improve on is already excellent Quotation withheld because this record is not cleared for redistribution. a clinical outcome trial on the composite endpoint needs patients in the thousands per arm, which the paper spells out rather than leaving implicit Quotation withheld because this record is not cleared for redistribution. reduction of the biochemical marker by the intervention has ranged from no effect at all to a large reduction across previous studies Quotation withheld because this record is not cleared for redistribution. stratified analyses must carry their own power calculations, or the stratification is not valid Quotation withheld because this record is not cleared for redistribution. pre-specified inclusion and exclusion criteria, blinding, randomisation and a power calculation before the experiment starts eliminates: post-hoc adjustment of the study to its own result - stated as a prohibition: 'it is not permitted to change the sample size post hoc, to add a new randomized group, or to supplement additional experiments' · outcome: held Quotation withheld because this record is not cleared for redistribution. pretests for normality or homogeneity of variance before choosing a test eliminates: nothing, and the paper refuses the practice: 'Those pretests cannot be recommended, since the failure to reject the null hypothesis does not imply that the null hypothesis holds' · outcome: failed Quotation withheld because this record is not cleared for redistribution. a consensus re-read when two human readers disagree about infarct size by more than 3% eliminates: reader-dependent infarct size on the primary endpoint - a disagreement threshold that triggers a procedure rather than an argument · outcome: held Quotation withheld because this record is not cleared for redistribution. a surrogate endpoint that predicts clinical outcome eliminates: nothing - the paper says none exists: 'None of the above-mentioned variable/surrogate endpoints is indeed predictive of clinical outcomes' · outcome: absent-not-in-staged-text Quotation withheld because this record is not cleared for redistribution. rigor rigor Quotation withheld because this record is not cleared for redistribution. reproducibility reproducibility Quotation withheld because this record is not cleared for redistribution. exploratory exploratory Quotation withheld because this record is not cleared for redistribution. prospective prospective Quotation withheld because this record is not cleared for redistribution. propofol propofol Quotation withheld because this record is not cleared for redistribution. isoflurane isoflurane Quotation withheld because this record is not cleared for redistribution. ischemic preconditioning ischemic preconditioning Quotation withheld because this record is not cleared for redistribution. nitrite nitrite Quotation withheld because this record is not cleared for redistribution. salvage index salvage index Quotation withheld because this record is not cleared for redistribution. area at risk area at risk Quotation withheld because this record is not cleared for redistribution. hazard ratio hazard ratio Quotation withheld because this record is not cleared for redistribution. multiplicity multiplicity Quotation withheld because this record is not cleared for redistribution. effect size effect size Quotation withheld because this record is not cleared for redistribution. transcriptomics transcriptomics Quotation withheld because this record is not cleared for redistribution. epitranscriptomics epitranscriptomics Quotation withheld because this record is not cleared for redistribution. pseudouridine pseudouridine Quotation withheld because this record is not cleared for redistribution. fifth nucleoside fifth nucleoside Quotation withheld because this record is not cleared for redistribution. HL-1 cells HL-1 cells Quotation withheld because this record is not cleared for redistribution. adult cardiomyocytes adult cardiomyocytes Quotation withheld because this record is not cleared for redistribution. `rigor` - DEFINED HERE AS A PROPERTY OF THE DOING: `standardized state-of-the-art design, conduct and reporting of a study` Quotation withheld because this record is not cleared for redistribution. `reproducibility` - DEFINED AS A PROPERTY OF WHAT HAPPENS ELSEWHERE: `replication of results by another laboratory when performing exactly the same experiment`, and rigour is named as its PREREQUISITE Quotation withheld because this record is not cleared for redistribution. `cardioprotection` - the operative sense arrives in a parenthesis at the moment the surrogates are disowned: `truly indicative of `cardioprotection` (which is primarily infarct size reduction)` Quotation withheld because this record is not cleared for redistribution. `outlier` - operationalised by Tukey's fences rather than by judgement: data outside `first quartile - 1.5 inter-quartile range and third quartile + 1.5 inter-quartile range` Quotation withheld because this record is not cleared for redistribution. `primary variable` - `the variable capable of providing the most relevant and convincing evidence directly related to the primary objective of the trial`, quoted from ICH E9, with `ideally there is only one primary endpoint` Quotation withheld because this record is not cleared for redistribution. `epitranscriptomics` - DEFINED ENTIRELY BY ITS PREFIX AND ITS SIBLING, IN ONE SENTENCE THAT DEFINES BOTH: `While transcriptomics refers to the characterization of gene expression at the ribonucleic acid (RNA) level, epitranscriptomics refers to post-transcriptional RNA modifications`. Neither term is given independently; the pair is introduced as a division of one territory. Quotation withheld because this record is not cleared for redistribution. `pseudouridine` / `the fifth nucleoside` - AN ALIAS THAT NAMES A POSITION IN A LIST, NOT A STRUCTURE. `pseudouridine (also called the “fifth nucleoside”)`. The paper reaches for the older name in passing; it is the kind of name that stops being true when the list it counts is extended. Quotation withheld because this record is not cleared for redistribution. extends → the ARRIVE guidelines for reporting animal research and the ICH E9 and E6 guidelines for clinical trials, which this document adapts to cardioprotection Quotation withheld because this record is not cleared for redistribution. builds-on → the CAESAR consortium's standardised multi-centre preclinical replication, whose result is the evidence that the field's preclinical base does not replicate Quotation withheld because this record is not cleared for redistribution. exploratory —contrasted-with→ prospective exploratory prospective strength of the relation: 0.95 · note: 'an EXPLORATORY…' - and the choice between them 'VS. LIMITED RESOURCES…' Quotation withheld because this record is not cleared for redistribution. propofol —distinct-from→ isoflurane propofol isoflurane strength of the relation: 0.95 · note: 'Propofol seems…'. Equal on one axis, opposite on the one that decides the result Quotation withheld because this record is not cleared for redistribution. ischemic preconditioning —distinct-from→ nitrite ischemic preconditioning nitrite strength of the relation: 0.95 · note: 'identified ONLY…'. A standardised multi-centre replication separating one claimed effect from two that did not survive it Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the span reports nitrite failing where preconditioning worked; distinct-from is loose. Quotation withheld because this record is not cleared for redistribution. transcriptomics —distinct-from→ epitranscriptomics transcriptomics epitranscriptomics strength of the relation: 1 · note: 'While X refers to ..., Y refers to ...' - the paper asserts the division outright, so weight 1.0. Quotation withheld because this record is not cleared for redistribution. pseudouridine —same-as→ fifth nucleoside pseudouridine fifth nucleoside strength of the relation: 1 · note: 'also called' - an explicit alias, both names inside the one span. Quotation withheld because this record is not cleared for redistribution. HL-1 cells —distinct-from→ adult cardiomyocytes HL-1 cells adult cardiomyocytes strength of the relation: 1 · note: 'they do not represent true adult cardiomyocytes' - a flat denial of the model relation, in a paper whose subject is whether preclinical models support their conclusions. Not hedged: weight 1.0. Quotation withheld because this record is not cleared for redistribution. THE PAPER DEFINES ITS OWN TWO WORDS AND MAKES ONE DEPEND ON THE OTHER: rigor is `standardized state-of-the-art design, conduct and reporting`, and it is `a prerequisite for reproducibility, i.e. replication of results by another laboratory when performing exactly the same experiment` THE PAPER DEFINES ITS OWN TWO WORDS AND MAKES ONE DEPEND ON THE OTHER: rigor is `standardized state-of-the-art design, conduct and reporting`, and it is `a prerequisite for reproducibility, i.e. replication of results by another laboratory when performing exactly the same experiment` Quotation withheld because this record is not cleared for redistribution. AND THE FAILURE IS COUNTED: `thousands of experimental studies ... so far there is only a single and very recent study, which has unequivocally translated cardioprotection to improved clinical outcome` AND THE FAILURE IS COUNTED: `thousands of experimental studies ... so far there is only a single and very recent study, which has unequivocally translated cardioprotection to improved clinical outcome` Quotation withheld because this record is not cleared for redistribution. A STANDARDISED MULTI-CENTRE REPLICATION CONFIRMED ONE EFFECT OF THREE: CAESAR `identified only ischemic preconditioning, but not nitrite or sildenafil ... to reduce infarct size` A STANDARDISED MULTI-CENTRE REPLICATION CONFIRMED ONE EFFECT OF THREE: CAESAR `identified only ischemic preconditioning, but not nitrite or sildenafil ... to reduce infarct size` Quotation withheld because this record is not cleared for redistribution. TWO IMPLEMENTATIONS OF ONE NAMED TEST GIVE DIFFERENT ANSWERS: `different statistical software packages work with slightly different assumptions and can come to different outcomes, as demonstrated for the Wilcoxon-Mann-Whitney` TWO IMPLEMENTATIONS OF ONE NAMED TEST GIVE DIFFERENT ANSWERS: `different statistical software packages work with slightly different assumptions and can come to different outcomes, as demonstrated for the Wilcoxon-Mann-Whitney` Quotation withheld because this record is not cleared for redistribution. THE SURROGATES DO NOT PREDICT WHAT THEY STAND IN FOR, AND THE REAL DEFINITION ARRIVES IN THE NEXT PARENTHESIS: `NONE ... is indeed predictive of clinical outcomes ... i.e. truly indicative of cardioprotection (which is primarily infarct size reduction)` THE SURROGATES DO NOT PREDICT WHAT THEY STAND IN FOR, AND THE REAL DEFINITION ARRIVES IN THE NEXT PARENTHESIS: `NONE ... is indeed predictive of clinical outcomes ... i.e. truly indicative of cardioprotection (which is primarily infarct size reduction)` Quotation withheld because this record is not cleared for redistribution. ONE DRUG IS THE COMMON DENOMINATOR OF THE FAILURES: `propofol specifically abrogates the protection by remote ischemic conditioning ... and seems to be a common denominator of all studies that failed to see protection` ONE DRUG IS THE COMMON DENOMINATOR OF THE FAILURES: `propofol specifically abrogates the protection by remote ischemic conditioning ... and seems to be a common denominator of all studies that failed to see protection` Quotation withheld because this record is not cleared for redistribution. THE INSTRUMENT DECIDES WHAT IS FOUND: `some will favor the isolation of short RNAs while others will enrich the RNA fraction with long molecules` - hence `not recommended to use different kits in the same study` THE INSTRUMENT DECIDES WHAT IS FOUND: `some will favor the isolation of short RNAs while others will enrich the RNA fraction with long molecules` - hence `not recommended to use different kits in the same study` Quotation withheld because this record is not cleared for redistribution. AND ABUNDANCE HIDES ABUNDANCE: `the large abundance proteins mask the changes of low-abundance proteins which also make a significant contribution to cellular signalling` AND ABUNDANCE HIDES ABUNDANCE: `the large abundance proteins mask the changes of low-abundance proteins which also make a significant contribution to cellular signalling` Quotation withheld because this record is not cleared for redistribution. AND THE ANALYSIS TOOLS CANNOT BE COMPARED WITH EACH OTHER: `their diversity in mathematical models and source data do not yet allow accurate comparison of different results` AND THE ANALYSIS TOOLS CANNOT BE COMPARED WITH EACH OTHER: `their diversity in mathematical models and source data do not yet allow accurate comparison of different results` Quotation withheld because this record is not cleared for redistribution. NO ACCEPTED METHOD FOR MISSING VALUES, QUOTED RATHER THAN ASSERTED: `there is no methodological approach for handling missing values that is universally accepted in all situations` NO ACCEPTED METHOD FOR MISSING VALUES, QUOTED RATHER THAN ASSERTED: `there is no methodological approach for handling missing values that is universally accepted in all situations` Quotation withheld because this record is not cleared for redistribution. AND A COMMON PRACTICE IS REFUSED BECAUSE A NON-SIGNIFICANT PRETEST PROVES NOTHING: `Those pretests cannot be recommended, since the failure to reject the null hypothesis does not imply that the null hypothesis holds ... the power of the pretests is often low` AND A COMMON PRACTICE IS REFUSED BECAUSE A NON-SIGNIFICANT PRETEST PROVES NOTHING: `Those pretests cannot be recommended, since the failure to reject the null hypothesis does not imply that the null hypothesis holds ... the power of the pretests is often low` Quotation withheld because this record is not cleared for redistribution. AND A HUMAN DISAGREEMENT THRESHOLD TRIGGERS A PROCEDURE: `If the difference in infarct size between the readers exceeds 3%, a consensus reading is recommended` AND A HUMAN DISAGREEMENT THRESHOLD TRIGGERS A PROCEDURE: `If the difference in infarct size between the readers exceeds 3%, a consensus reading is recommended` Quotation withheld because this record is not cleared for redistribution. with the outlier included, the test gives t = 4.82 (36 df) and P < 0.001 Quotation withheld because this record is not cleared for redistribution. without the outlier, t = 6.08 (35 df), P < 0.001, and the group-mean difference is 20.06 (SD 10.03) Quotation withheld because this record is not cleared for redistribution. detecting a 0.1 increase in salvage index needs 83 patients per group at 80% power and two-sided alpha 0.05 Quotation withheld because this record is not cleared for redistribution. a biomarker reduction that was not powered for outcome nonetheless tracked increased survival and fewer events in an earlier study Quotation withheld because this record is not cleared for redistribution. `standardized state-of-the-art design, conduct and reporting of a study, which is then a prerequisite for reproducibility` rigor Quotation withheld because this record is not cleared for redistribution. `replication of results by another laboratory when performing exactly the same experiment` reproducibility Quotation withheld because this record is not cleared for redistribution. data outside Tukey's inner fences - `first quartile - 1.5 inter-quartile range and third quartile + 1.5 inter-quartile range` outlier Quotation withheld because this record is not cleared for redistribution. quoted from ICH E9: `the variable capable of providing the most relevant and convincing evidence directly related to the primary objective of the trial` primary variable Quotation withheld because this record is not cleared for redistribution. 3 — the disagreement between two human readers that triggers a consensus re-read Quotation withheld because this record is not cleared for redistribution. 1.5 — Tukey's inner fences - the operational boundary of 'outlier' Quotation withheld because this record is not cleared for redistribution. 1 — the number of studies that had unequivocally translated cardioprotection to improved clinical outcome, against thousands of experimental ones Quotation withheld because this record is not cleared for redistribution. the infarct size target and the salvage index change it corresponds to Quotation withheld because this record is not cleared for redistribution. current cardiac mortality, heart-failure rehospitalisation and composite endpoint incidence in the most recent cohort Quotation withheld because this record is not cleared for redistribution. `the gold standard primary endpoint` for cardioprotection, and the thing the surrogate endpoints fail to predict Infarct size Quotation withheld because this record is not cleared for redistribution. the same comparison run with and without the outlier, both t statistics, both P values, both mean differences and both confidence intervals Quotation withheld because this record is not cleared for redistribution. the worked survival-analysis power calculation, from assumed survival percentages to a hazard ratio to a per-group sample size Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1007/s00395-018-0696-8.