The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement — TEI-tagged view

Eric I. BenchimolpersName, Liam SmeethpersName, Astrid GuttmannpersName, Katie HarronpersName, David MoherpersName, Irene PetersenpersName, Henrik T. SørensenpersName, Erik von ElmpersName, Sinéad M. LanganpersName (2015date). “The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement.” PLOS Medicinetitle·j.

Object of study — reporting of observational research conducted with routinely collected health dataterm

Research question — which routine-data-specific information must be reported so readers can assess validity, biasterm, reproducibility, and applicability

Contribution — the 13-item RECORDterm·mth extension to STROBE with explanations, examples, implementation infrastructure, and future revision process

Full text — tagged

The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.

Routinely collected health dataterm, obtained for administrative and clinical purposes without specific a priori research goals, are increasingly used for research. The rapid evolution and availability of these data have revealed issues not addressed by existing reporting guidelines, such as Strengthening the Reporting of Observational Studies in Epidemiology (STROBE). The REporting of studies Conducted using Observational Routinely collected health Dataterm (RECORDterm·mth) statement was created to fill these gaps. RECORDterm·mth was created as an extension to the STROBE statementterm·mth to address reporting items specific to observational studies using routinely collected health dataterm. RECORDterm·mth consists of a checklist of 13 itemsterm related to the title, abstract, introduction, methods, results, and discussion section of articles, and other information required for inclusion in such research reports. This document contains the checklist and explanatory and elaboration informationterm to enhance the use of the checklist. Examples of good reporting for each RECORD checklistterm item are also included herein. This document, as well as the accompanying website and message board ( http://www.record-statement.org ), will enhance the implementation and understanding of RECORDterm·mth. Through implementation of RECORDterm·mth, authors, journals editors, and peer reviewers can encourage transparency of research reportingterm.

Introduction The growing availability of data generated during health care delivery, and through monitoring of disease incidence and outcomes, has transformed the research landscape. Routinely collected health dataterm are defined as data collected without specific a priori research questions developed prior to utilization for research [ 1 ]. These data sources could include provision of broad resources for research (e.g., disease registriesterm), clinical management (e.g., primary care databasesterm), health system planning (e.g., health administrative dataterm), documentation of clinical care (e.g., electronic health record data repositoriesterm), or epidemiological surveillance (e.g., cancer registriesterm and public health reporting data). These data, generated in various health care settings and geographic locations, present opportunities for innovative, efficient, and cost-effective research to inform decisions in clinical medicine, health services planning, and public health [ 2 ]. Internationally, governments and funding agencies have prioritized use of routinely collected health dataterm as tools to improve patient care, transform health research, and improve health care efficiency [ 3 ].

While the explosion in data availability presents significant opportunities to answer pressing research questions, it also poses challenges for those undertaking and evaluating the research and implementing its findings. The broad spectrum of routinely collected health dataterm sources and the rapid expansion of the field makes it challenging to identify the strengths and limitations and associated biasesterm of individual data sources. Incomplete or inadequate reporting of research based on routinely collected data exacerbates these challenges. A systematic analysis of a sample of studies utilizing routine data sources has identified a variety of areas of incomplete or unclear reportingterm [ 4 ]. Reporting deficiencies include inadequate or missing information concerning coding of exposures and outcomesterm as well as details of linkage ratesterm of different data sources. Two recent systematic reviews also document poor reporting of studies undertaken to validate data from routine data sources [ 5 , 6 ], which can obscure sources of biasterm, hamper efforts to undertake meta-analysesterm, and lead to erroneous conclusions.

Population hierarchy in studies using routinely collected data sources.

Reporting guidelines have been developed to guide reporting for a range of study designs and contexts and are associated with improved quality of reporting [ 8 , 9 ]. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement was developed to enhance the transparency of reporting of observational research [ 10 , 11 ] and has been widely adopted and endorsed by leading medical journals. It has been demonstrated to improve the quality of research reporting when implemented in the editorial process [ 12 , 13 ]. Most research conducted using routinely collected data is observational in design, and therefore, the STROBE guidelines are relevant and applicable. However, as the STROBE statementterm·mth is designed to apply to all observational studies, specific issues related to reporting research using routinely collected data are not addressed. An international group of scientists with a specific interest in using routinely collected health dataterm and representatives of the STROBE group met following the Primary Care Databaseterm Symposium in London in 2012 to discuss STROBE in the context of studies using routinely collected health dataterm [ 14 , 15 ]. Important gaps in STROBE specific to research using these data sources were identified, and agreement was reached that an extension to STROBE was warranted. Thus, the REporting of studies Conducted using Observational Routinely collected Data (RECORDterm·mth) initiative was established as an international collaborative process and an expansion of STROBE to explore and address specific reporting issues relevant to research using routinely collected health dataterm. The RECORDterm·mth initiative has involved more than 100 international stakeholders comprising researchers, journal editors, and consumers of data, including those utilizing research findings from routine data to inform decision making. The methodology used to develop the RECORDterm·mth guidelines is detailed elsewhere [ 16 ] and based on the established methods to develop reporting guidelines [ 17 ]. In brief, stakeholders were surveyed twice to establish and prioritize themes for inclusion in the RECORDterm·mth statement. A working committee then met in person to establish wording of the statements. Stakeholders reviewed the statements and provided feedback. The final checklist and this explanatory document were drafted by members of the steering committee, with review and approval by the working committee. Members of the STROBE steering committee were involved in the creation of RECORDterm·mth.

Consistent with the STROBE approach, the RECORDterm·mth guidelines are not designed to recommend the methods used to conduct research, but rather to improve its reporting to ensure that readers, peer reviewers, journal editors, and other consumers of research can assess its internal and external validityterm. By improving the quality of reporting of research using routinely collected health dataterm, we seek to reduce unclear research reports and achieve the tenets of the scientific process: discovery, transparency, and replicabilityterm [ 18 ].

Items in the RECORD Checklistterm The complete RECORD checklistterm is provided in Table 1 . Since RECORDterm·mth is an extension to available STROBE items, statements are presented next to corresponding STROBE checklist items, organized by manuscript section. We advise authors to adequately address each item in the checklist but do not prescribe a precise order or location in the manuscript. Below we have provided explanatory text for each RECORD checklistterm item, organized by manuscript section. When no additional checklist items were required to expand STROBE to studies using routinely collected health dataterm, any needed explanation is provided under the respective STROBE item.

The RECORDterm·mth statement: Checklist of items, extended from the STROBE statementterm·mth, that should be reported in observational studies using routinely collected health dataterm.

Title and Abstract RECORDterm·mth ITEM 1.1: The type of dataterm used should be named in the title or abstract. Where possible, the names of the databases used should be included.

RECORDterm·mth ITEM 1.2: If applicable, the geographic regionterm and time frameterm within which the study took place should be reported in the title or abstract.

RECORDterm·mth ITEM 1.3: If linkage between databasesterm was conducted for the study, this should be clearly stated in the title or abstract.

Examples Two examples of good reporting of this section are contained in the articles listed below: “Perforations and Haemorrhages after Colonoscopy in 2010: A Study Based on Comprehensive French Health Insurance Data (SNIIRAM)” [ 19 ].

“The Dutch Hospital Standardised Mortality Ratio (HSMR) Method and Cardiac Surgery: Benchmarking in a National Cohort Using Hospital Administration Data versus a Clinical Database” [ 20 ].

Explanation As there are no accepted Medical Subject Heading (MeSH) subject headings for identifying studies that use routinely collected health dataterm, it is important to be able to identify a study as one conducted using such data. However, considering the wide variety of data types, simply stating that routine data were used is insufficient. Instead, the type of routine data should be specified in the title and/or abstract. Examples of data types include health administrative dataterm, other administrative data (e.g., insurance, birth/death registries, and employment), disease registriesterm, primary care databasesterm, electronic health recordterm·mth data, and population registries. Naming the database(s) used is important but does not replace providing the type of dataterm sources in the title or abstract.

The geographic regionterm and time frameterm are items included in the STROBE checklist. We suggest that this information is also a necessary item in the title or abstract sections of manuscripts using the RECORD checklistterm. Clearly, the extent of reporting of region and time frameterm needs to adhere to word count limitations and take into account confidentiality issues. However, region should be reported at least at the largest geographical level used to define the study populationterm (e.g., nation, state, province, and region).

In addition, linkage between databasesterm (if it was conducted) should be reported in the title or abstract. Examples of acceptable wording include “using multiple linked health administrative databases” or “(database name) linked to (database name).” Using the words “linked” or “linkage” provides sufficient information in the title or abstract; further detail on linkage methodology should be provided in the Methods section of the manuscript.

Introduction No items specific to the RECORDterm·mth guidelines are needed in addition to the STROBE items. The STROBE guidelines advise that “specific objectives, including any pre-specified hypotheses,” be stated in the Introduction section. Stating the specific research objectives is essential for replication and translation of any observational research. For studies using routinely collected data, authors should further clarify whether the analyses were exploratoryterm·mth with the purpose of finding new relationships in the data (examples are data mining or hypothesis-generating studies [ 21 , 22 ]) or confirmatoryterm·mth with the purpose of testing one or more hypotheses [ 23 ]. Authors also should indicate whether their hypotheses were generated before or after data analysis. They should clearly state whether there is a study protocolterm and how this can be accessed and if the study was registered in a publicly accessible study registry. Because the strengths and limitations of methods used to conduct research with routinely collected data may be contentious, a clear description of a study’s objectives is essential [ 23 , 24 ]. It is insufficient to simply label a study as descriptive without clarifying whether it aims to generate or examine a hypothesis.

Methods (Setting) No additional RECORDterm·mth items are needed to expand the STROBE requirement to “describe the setting, locations, and relevant dates, including periods of recruitment, exposure, follow-up, and data collection.” Authors should note that beyond the type of database already alluded to in the title and/or abstract, information should be provided to allow the reader to understand the content and validity of the database and the original reasons why the data were collected. For instance, an electronic health recordterm·mth can be used by specialists or primary care physicians, for ambulatory or inpatient care, or by senior physicians or medical students. Users can be specifically trained for exhaustive and reproducible data entry, or no training may be provided [ 25 ]. Authors should also describe how the database populationterm relates to the source populationterm, including selection criteria, in order for readers to determine whether findings can be applied to the source populationterm.

Methods (Participants) RECORDterm·mth ITEM 6.1: The methods of study populationterm selection (such as codes or algorithmsterm used to identify subjects) should be listed in detail. If this is not possible, an explanation should be provided.

RECORDterm·mth ITEM 6.2: Any validation studiesterm of the codes or algorithmsterm used to select the population should be referenced. If validation was conducted for this study and not published elsewhere, detailed methods and results should be provided.

RECORDterm·mth ITEM 6.3: If the study involved linkage of databases, consider use of a flow diagramterm or other graphical display to demonstrate the data linkage process, including the number of individuals with linked data at each stage.

Examples RECORDterm·mth ITEM 6.1. An example of good reporting is provided in the following excerpt: The OCCC [Ontario Crohn’s and Colitis Cohort] uses validated algorithms to identify patients with IBD based on age group. Each of these algorithms was validated in Ontario, in the specific age group to which it was applied, in multiple cohorts, medical practice types, and regions. For children younger than 18 years, the algorithm was defined by whether they underwent colonoscopy or sigmoidoscopy. If they had undergone endoscopy, children required 4 outpatient physician contacts or 2 hospitalizations for IBD within 3 years. If they had not undergone endoscopy, children required 7 outpatient physician contacts or 3 hospitalizations for IBD within 3 years.… This algorithm correctly identified children with IBD with a sensitivityterm of… [ 26 ].

This article referenced two previous validation studiesterm of algorithms to identify patients with inflammatory bowel disease of different ages, including measures of diagnostic accuracy.

RECORDterm·mth ITEM 6.2: 1. In their article, Ducharme and colleagues described in detail the validation of codes to identify children with intussusception and then used the validated codes to describe epidemiology. The codes involved in the validation study were listed in figure 2 of the article [ 27 ]. 2. In their article, BenchimolpersName and colleagues did not conduct validation work; however, the validation work previously conducted was referenced. Details of diagnostic accuracy of identification algorithm codes were described [ 26 ].

RECORDterm·mth ITEM 6.3: Some possible ways to illustrate the linkage process are demonstrated in the example figure 2, figure 3, and figure 4 on the RECORDterm·mth website: Figure 2. Venn diagram to illustrate linkage process (reproduced with permission from Herrett et al. [ 28 ] on our website: http://record-statement.org/images/figure2.jpg ).

Figure 3. Mixed flow diagramterm and Venn diagram illustrating linkage process (reproduced with permission from van Herk-Sukel et al. [ 29 ] on our website: http://record-statement.org/images/figure3.jpg ).

Figure 4. Linkage diagram combined with participant flow diagramterm (reproduced with permission from Fosbøl et al. [ 30 ] on our website: http://record-statement.org/images/figure4.jpg ).

Explanation RECORDterm·mth ITEMS S 6.1 and 6.2: Reporting the validity of identification codes/algorithms used to derive the study populationterm is essential to the transparency of reporting of observational research using routinely collected health dataterm. In addition, reporting of codes/algorithmsterm allows other investigators to engage in external or internal validationterm.

The methods used to identify study subjects should be explicitly and clearly stated, including whether identification is based on single codes, algorithms (combinations of recordsterm·mth or codes), linkage between databasesterm, or free-text fields.

The risk of misclassification biasterm in studies using routine health data, as in many other epidemiological studies, may threaten the validity of study findings [ 31 ]. Although the risk of misclassification is amplified in studies using databases containing large populations, such studies offer an opportunity to study rare or uncommon diseases [ 32 ]. Validation of identification methods has been increasingly emphasized as essential for studies using routinely collected health dataterm, particularly for disease codes in studies using administrative data collected for billing purposes [ 33 ]. External validation studiesterm typically entail comparing the codes or algorithmsterm used to identify study populationsterm to a reference standardterm·mth. The most common reference standardsterm·mth are medical recordsterm·mth, surveys of patients or practitioners, and clinical registries [ 5 , 34 ]. In addition, internal validationterm of databases may be undertaken to compare overlapping data sources within a single database [ 35 ]. Measures of accuracy are similar to those reported in diagnostic test studies, including sensitivityterm, specificityterm, positive and negative predictive valuesterm, or kappa coefficientsterm [ 5 , 34 ].

Thus, for observational studies using routinely collected health dataterm, we recommend that details of external or internal validationterm of identification codes/algorithms be presented in the Methods section of the manuscript. If one or more validation studiesterm were previously undertaken, these should be referenced. If such validation studiesterm were not conducted, this should be explicitly stated. In addition, a brief discussion of the accuracy of the identification methods (using common diagnostic accuracy terms) and their functioning in the subpopulations under study should be included. If validation work was conducted as part of the observational study in question, we suggest that authors use the published reporting guidelines for validation studiesterm [ 5 ]. It is important to state whether the validation occurred in a source or database populationterm different from that selected for the present study, as codes may function differently in different populations or databases [ 36 ]. In addition, if there are known problems with the reference standardterm·mth to which the data were compared, for example, incompleteness or inaccuracy, these problems should be reported and additionally be discussed as a limitation. Authors should discuss the implications of using the selected codes/algorithms to identify study populationsterm and outcomes, the risk of misclassification, and the potential impacts on study findings. It is particularly important to discuss the implications of relying on a validation study conducted in a population different from the one being examined.

RECORDterm·mth ITEM 6.3: A flow diagramterm or other graphical display can convey useful information about the linkage process and can simplify a potentially lengthy description. Such illustrations can provide key data such as information on the proportion and characteristics of the linked and unlinked individuals. Readers should be able to establish the proportion of the database populationsterm that were successfully linked and the representativenessterm·var of the resulting study populationterm. Linkage flow diagramsterm can either be stand-alone diagrams (e.g., Venn or flow diagramsterm) or can be combined with the participant flow diagramterm as recommended by STROBE. As graphical displays can be provided in many formats, we do not recommend a specific one.

Methods (Variables) RECORDterm·mth ITEM 7.1: A complete list of codes and algorithms used to classify exposures, outcomes, confounders, and effect modifiersterm should be provided. If these codes or algorithmsterm cannot be reported, an explanation should be provided.

Examples Hardelid and colleagues provided all codes in their S1 Table in Data Supplement 2 [ 37 ].

Murray and colleagues provided all codes for at-risk groups in their Appendix S1 [ 38 ].

Explanation Just as with codes/algorithms used to identify the study populationterm, codes/algorithms to classify exposures, outcomes, confounders, or effect modifiers subject the research to potential misclassification biasterm. In order to allow for replication, evaluation, and comparisons to other studies, we recommend that a list of all the diagnostic, procedural, medication, or other codes used to conduct the study be provided in the manuscript, an online appendix, and/or an external website. For routine data consisting of survey results, the survey questions should be provided with the precise wording given to study subjects. Considering the risk of misclassification biasterm in all research, including research conducted using routine health data [ 31 ], authors should provide sufficient detail to make their research reproducible and to make the risk of biasterm apparent. Validation studiesterm may be described in the article manuscript or provided as references to other published or online material. As noted above, authors should state whether the validation study was conducted in a source or database populationterm different from that examined in the present study.

We recognize that in some situations, researchers may be prevented from providing code liststerm and algorithms used in a publication, as this information is considered proprietaryterm or protected by copyright, intellectual property, or other laws. For example, some comorbidity adjustment indices have been created by for-profit companies and sold to researchers for use in academic research settings [ 39 , 40 ]. In these situations, authors may have relied on data providers or trusted third parties to collect, clean, and/or link the data. Authors should provide a detailed explanation regarding their inability to provide code liststerm or other details on how individuals or conditions are identified and should endeavour to include contact information for the group holding proprietaryterm rights to these lists. In addition, authors should address how their inability to provide this information may impact consumers of the research in terms of research replication and evaluation. Optimally, the third parties should provide detailed information on how the data were collected, cleaned, or linked. Improved communication between data providers and data users could be mutually beneficial.

Some have argued that code liststerm represent the researchers’ intellectual property. Publication of these lists could allow other researchers to use them for their own research, thereby depriving the authors of their intellectual property and credit for creating the code listterm. We felt that this view is inconsistent with the scientific standard of transparency to allow for replication of research. Therefore, apart from those protected by law or contract, we recommend that the full code liststerm be published.

Considering word count and space restrictions in many journals and the potential length of code liststerm/algorithms, we recognize that publication in a paper-format journal article may not be possible. Instead, detailed information could be reported in the text, published tables, online supplements on the journal website as appendices, hosted online permanently by the authors or other individuals, or deposited in a third party data repository (e.g., Dryad or Figshare). The text and reference sections of the manuscript should provide detailed information on how to access code liststerm. Code repositories such as ClinicalCodes.org hold great promise for the documentation and transparency of codes used in research based on health data [ 41 ]. If the code liststerm are published in online supplements on the journal website or on an external website provided by the authors, the link should be published in the main journal article. Publication on a journal website or on PubMed Central ( http://www.ncbi.nlm.nih.gov/pmc/ ) increases the probability that the supplement will be available as long as the journal is operational. If publication on an external private or institutional website is the only option, we recommend that these lists continue to be available for at least 10 years following publication of the journal article. If the URL address is changed, automatic redirection from the old web address is required. These measures will allow for future readers of the article to have access to the complete code liststerm.

In addition to code liststerm provided in the article (or an online appendix), the authors should include a reflection on whether the choice of codes/algorithms used in the study might lead to biasterm. Such biasterm could include misclassification biasterm, ascertainment biasterm, and biasterm due to missing dataterm. If sensitivity analysesterm were conducted based on different sets of codes/algorithms, these should also be described and evaluated. Discussion of potential biasterm could also be linked to other parts of the RECORDterm·mth and STROBE checklists, such as study subject selection, and validation of codes (or lack thereof).

Methods (Statistical Methods) Data access and cleaning methods RECORDterm·mth ITEM 12.1: Authors should describe the extent to which the investigators had access to the database used to create the study populationterm. RECORDterm·mth ITEM 12.2: Authors should provide information on the data cleaning methodsterm used in the study.

Linkage RECORDterm·mth ITEM 12.3: State whether the study included person-level, institutional-level, or other data linkage across two or more databases. Linkage techniques and methods used to evaluate linkage qualityterm·var should be provided.

Examples RECORDterm·mth ITEM 12.1: The following articles describe access to a subset of the UK General Practice Research Database (GPRD).

“The GPRD restricts its data sets to 100,000 individuals for projects funded through the Medical Research Council licence agreement. This restriction mandated a case-control rather than cohort design to ensure we identified sufficient cases of cancer for each particular symptom…” [ 42 ].

“A random sample from the General Practice Research Database…was obtained under a Medical Research Council licence for academic institutions [ 43 ].

RECORDterm·mth ITEM 12.2: The following is an example of a data cleaning methodsterm description [ 44 ]: Completeness of common identifiers for linking varied between datasets and by time (identifiers were more complete in recent years). For LabBase2, completeness of identifiers varied by unit (figure 2). For PICANet [Paediatric Intensive Care Audit Network], date of birth and hospital number were 100% complete, and the majority of other identifiers were >98% complete, with the exception of NHS [National Health Service] number (85% complete). For both datasets, cleaning and data preparation were undertaken: NHS or hospital numbers such as “Unknown” or “9999999999” were set to null; generic names (e.g., “Baby,” “Twin 1,” “Infant Of”) were set to null; multiple variables were created for multiple surname and first names; postcodes beginning “ZZ” (indicating no UK postcode) were set to null.

RECORDterm·mth ITEM 12.3: The following excerpts from articles are good examples of good reporting of the level of data linkage, the linkage techniques and methods used, and the methods used to evaluate linkage qualityterm·var: “We linked live birth and fetal death certificates into chronological chains of events that, excluding induced abortions and ectopic pregnancies, constituted the reproductive experience of individual women” [ 45 ].

Two articles contain excellent descriptions of linkage undertaken specifically for the study being reported [ 44 , 45 ]. In the article by HarronpersName and colleagues [ 44 ], a detailed explanation on linkage is provided with graphical demonstration of the match process. In addition, the methods to calculate probability of linkage are described: “Match probabilities P(M|agreement pattern) were calculated to estimate the probability of a match given agreement on a joint set of identifiers. This avoided the assumption of independence between identifiers. Probabilities were derived as the number of links divided by the total number of pairs for each agreement pattern (based on probable links identified in the training datasets). For example, if 378 comparison pairs agreed on date of birth and Soundex but disagreed on sex, and 312 of these were probable links, the match probability for the agreement pattern [1,1,0] was 312/378 = 0.825” [ 44 ]. The article by Adams and colleagues also provided a detailed explanation of the linkage process: “The deterministic linkageterm consisted of phase I, which entailed six processing steps during which chains were formed and individual (previously unlinked) recordsterm·mth were added to chains. Next followed phase n, which entailed multiple passes through the file to combine chains belonging to the same mother” [ 45 ].

By contrast, if a study refers to prior linked data, referring to a prior paper may be adequate as follows: “Recordsterm·mth from both databases were linked to the municipal registries based on date of birth, gender and zip code, and were subsequently linked to each other. The linkage was performed by Statistics Netherlands and is described in previous publications” [ 20 ].

The following is an example of good reporting of characteristics of linked and unlinked individuals: “For the purposes of this paper unmatched ISC [Inpatient Statistics Collection] recordsterm·mth will be referred to as ISC residuals, unmatched MDC [Midwives Data Collection] recordsterm·mth as MDC residuals and linked pairs as matched recordsterm·mth….Selected variables that were available on both data sets were compared across three groups—ISC residuals, MDC residuals and matched recordsterm·mth” [ 46 ].

Explanation RECORDterm·mth ITEMS 12.1 and 12.2: Errors can occur if data analysts unfamiliar with the nuances of cohort creation or study aims create the study cohorts. Consequently, the extent authors had access to the database should be reported. The description of data cleaning methodsterm at different stages of the study should include those used to screen for erroneous and missing dataterm, including range checks, checks for duplicate recordsterm·mth, and handling of repeated measures [ 47 , 48 ]. Other methods to be reported could include assessment of frequency distributions and data cross tabulations and graphical exploration or use of statistical methods for outlier detection [ 49 ]. Further detail could be provided on error diagnosis, including definitions of plausibility, and error handling in the analysis. A clear and transparent description of data cleaning methodsterm is important, as choice of methods could affect study findings, repeatability of the study, and reproducibility of study findings [ 50 ].

RECORDterm·mth ITEM 12.3: For linkage studies, we suggest reporting on the estimated rate of successful linkage, use of deterministic versus probabilistic linkageterm, quality and type of variables used for linkage, and results of any linkage validation. If linkage of recordsterm·mth across databases was conducted specifically for the study, methods of linkage and linkage qualityterm·var evaluation should be reported, including information on who performed the linkage. As available, details should be provided on blocking variables, completeness of linkage variables, linkage rules, thresholds, and manual review [ 44 ]. If linkage was conducted prior to the study (i.e., for previous studies or for general use) or if data linkage was undertaken by an external provider, such as a data linkage centre, then a reference is needed describing the data resource and linkage methods.

Data describing linkage methods and evaluating their success are critical to permit the reader to assess the impact of any linkage errorterm and related biasterm [ 51 ]. Specifically, the reader should know whether the type of linkage used was deterministic and/or probabilistic, in order to determine whether linkage could be affected by false matches or missed matchesterm. Deterministic linkageterm is useful when a unique identifier is available across the different data sources. When such an identifier is unavailable, a description of the recordterm·mth linkage rules applied (or statistical linkage keys) is critical. In contrast, probabilistic linkageterm uses multiple identifiers, sometimes with different weights, and matches are considered present above a specific threshold. Mixed methods also may be used. For instance, deterministic linkageterm may be used for some recordsterm·mth, and probabilistic linkageterm may be applied when unique identifiers are unavailable for other recordsterm·mth. Linkage biasterm occurs when associations are present between the probability of linkage errorterm (e.g., false and missing matches) and the variables of interest. For example, linkage ratesterm may vary by patient characteristics, e.g., age, gender, and health status. Even small errors in the linkage process can introduce biasterm and lead to results that can overestimate or underestimate the associations under study [ 52 ]. Authors should report linkage errorterm using standard approaches including comparisons with gold standards or reference datasets, sensitivity analysesterm, and comparing characteristics of linked and unlinked data [ 53 ]. Reporting linkage errorterm allows the reader to determine the quality of the linkage and the possibility of biasterm related to linkage errorterm.

Results (Participants) RECORDterm·mth ITEM 13.1: Describe in detail the selection of the persons included in the study (i.e., study populationterm selection), including filtering based on data quality, data availability and linkage. The selection of included persons can be described in the text and/or by means of the study flow diagramterm.

Example An example of good reporting is given in the following excerpt: We identified 161,401 Medicare beneficiaries given a diagnosis of one or more cases of cancer of the lung and bronchus in the SEER [Surveillance, Epidemiology, and End Results] registries between 1998 and 2007. Among these patients, we identified a total of 163,379 separate diagnoses of incident lung cancer. (Some patients had two cases of primary lung cancer separated by more than a year during the study period). Fig 1 shows the derivation of the final cohort of 46,544 patients with 46,935 cases of NSCLC [non-small cell lung cancer] [ 54 ]. (See figure 5 for the example flow diagramterm, available at http://record-statement.org/images/figure5.jpg .) Explanation The authors should provide a clear description of the derivation of the study populationterm(s) from the original database of routinely collected health dataterm, as differences between the study populationterm and the database populationterm need to be documented to enable application of the results (See also RECORDterm·mth item 6.1). Researchers using routine data sources frequently limit their study populationterm based on factors such as the quality of available data. For example, they may restrict the study period to a time when the data quality is known to be acceptable, resulting in the exclusion of potential participants. Studies may exclude medical practices with inconsistent electronic health recordterm·mth entry or wait for those practices to become consistent [ 38 , 55 ]. The study populationterm also may be restricted based on data availability. For example, in studies utilising United States Medicare data, beneficiaries currently registered in a health maintenance organisation are frequently excluded because of lack of recordsterm·mth of clinical events [ 54 , 56 ]. When using data sources in which eligibility fluctuates over time (e.g., insurance databases) researchers need to specify clearly how eligibility was defined and how changes in eligibility were managed in their study. If a study utilises linked routine data, the study populationterm is frequently reduced through restriction to individuals for whom linked data are available [ 57 ]. Highly restricted cohorts may also be used for methodological reasons to eliminate some sources of confounding.

Thus, steps taken to derive the final study populationterm(s), inclusion and exclusion criteria, and inclusion and exclusion of study participants at different stages in cohort creation and analysis should be clearly defined in the manuscript, either in the text or using a suitable flow diagramterm. Study populationsterm may be derived using different codes and/or algorithms (see RECORDterm·mth item 6.1), and different use of codes over time may impact the study populationterm [ 58 , 59 ]. Some studies may also have used several case definitions that are more or less sensitive/specific, which may have an impact on subsequent analyses. Delineation of these steps is important in assessing the external validity of study findings and, in certain circumstances, assessing possible selection biasterm. Sensitivity analysesterm may be reported to evaluate the potential impact of missingness of dataterm·var and representativenessterm·var of the study populationterm. Providing information on selection of the study populationterm(s) from the initial database also permits the study to be replicated. Subsidiary analyses may have been performed on different study populationsterm and may potentially be reported in online appendices.

Discussion (Limitations) RECORDterm·mth ITEM 19.1: Discuss the implications of using data that were not created or collected to answer the specific research question(s). Include discussion of misclassification biasterm, unmeasured confoundingterm, missing dataterm, and changing eligibilityterm over time, as they pertain to the study being reported.

Examples The following papers describe limitations associated with the use of administrative data: “Third, this study was a retrospective, claims-based analysis. Only PET [positron emission tomography] scans paid for by Medicare could be detected in the analysis. To minimize the proportion of missed claims, all analyses were limited to Medicare beneficiaries with both Medicare Part A and Part B coverage and no enrollment in managed care or Medicare Part C for the 12 months before and after diagnosis. Fourth, patients in the SEER registry are more likely to be nonwhite, to live in areas with less poverty, and to live in urban areas, which may limit the generalizability of the findings. Fifth, during the study period, disease stage was based on SEER data obtained over 4 months or until first surgery. In 2004, data collection for SEER changed to the collaborative staging system. It is unclear how our results would differ with this newer approach” [ 54 ].

“Despite several strengths of the SEER-Medicare data, including a comparatively large sample size, generalizability to the US population, and detailed information on prescriptions, our study was limited by the lack of laboratory data on cholesterol, triglyceride, and glucose levels that would have informed the extent of metabolic disturbances in the population…thus having laboratory-based data could have reduced residual confounding by severity of metabolic disease. We also lacked more granular data on cancer progression, which could have confounded the association between statin use and death, given that statin treatment may be withheld or discontinued in patients with short expected survival time” [ 60 ].

Explanation Routine health data are not typically collected with a specific a priori research question in mind, and the reasons motivating the data collection may vary. Numerous potential areas of biasterm, including all the usual sources of biasterm associated with observational research but also some more specific to observational research using routine data, endanger the conclusions of researchers. The following should be discussed by authors as potential sources of biasterm: (1) codes or algorithmsterm to identify study populationsterm, outcomes, confounders, or effect modifiers (misclassification biasterm); (2) missing variables (unmeasured confoundingterm); (3) missing dataterm; and (4) changes in eligibility over time.

The rationale underlying routine data collection may affect the quality and applicability of the data to research questions being examined. For example, registries used for retrospective analyses may implement better quality control than organizations collecting other types of routine data, although this may vary. Similarly, some administrative data are subject to careful quality control, while other data are not. Administrative data are particularly subject to errors in upcoding or opportunistic coding. For example, when hospital reimbursement is based on complexity of the case mix, hospitals may maximize reimbursement by liberally applying more complex disease codes to patient recordsterm·mth [ 61 ]. In addition, changes in coding strategies may impact validity or consistency of data. For example, the introduction of provider billing incentive codes may change the likelihood of a code being used over time [ 62 , 63 ]. Other codes may be avoided because of patient stigmatization or provider penalties [ 64 ]. In addition, changes in versions of code classification systems (e.g., from International Classification of Diseases (ICD)-9 to ICD-10) may alter the validity of ascertainment using coded data [ 65 , 66 ]. Variation in clinical practice across hospitals and populations may result in laboratory investigations being undertaken in specific locations and/or practices, which may impact a diagnostic algorithm. If any of these potential sources of misclassification biasterm are present, they should be discussed as study limitations.

Unmeasured confoundingterm is defined as confounding associated with variables not included in the data under study, leading to residual confounding biasterm [ 67 ]. While it is a potential source of biasterm in all observational research, it is particularly prominent in studies using routinely collected data. The analysis may require variables that were not considered when the databases were being planned or when the data were collected. A variety of methods have been proposed to address this potential source of biasterm [ 68 – 71 ], including propensity scores. However, propensity score analyses, like standard regression analyses and matching, can only guarantee a balance of study participants on variables that are available in the data. A particular type of unmeasured confoundingterm is confounding by indicationterm; this is often an issue when examining effectiveness and safety of (drug) treatments using routinely collected data. Hence, the prognosis of those receiving the (drug) treatment may be better or worse than those who are not, but information on prognosis and/or severity of the underlying illness may not be available in the data [ 72 ]. Such issues should be discussed by the authors, and the methods used to take this into account (when possible) should be reported.

Missing dataterm are problematic for all observational research and have been addressed in Box 6 of the STROBE explanatory article [ 10 ]. Missing dataterm are a particular issue for routinely collected data, as researchers cannot control data collection [ 73 ]. Missing dataterm can result in selection biasterm if there are missing values in variables used to define the study cohort or missing identifiers that prevent recordsterm·mth from being linked, particularly if the missing dataterm occur nonrandomly. Missing variables create similar challenges. Authors should delineate the missing variables suspected of causing unmeasured confoundingterm, the reason these variables were missing, how this may have affected study results, and the methods used to adjust for missing variables. For example, smoking status has a strong effect on Crohn’s disease severity and has been associated with outcomes of this disease. However, smoking status is rarely included in health administrative dataterm. In a study assessing the association between socioeconomic status and Crohn’s disease outcomes, smoking status was discussed as a potential unmeasured confoundingterm factor [ 74 ]. Frequently, missing dataterm/missing variables are discovered only after initiation of research using routine health data, making it necessary for investigators to deviate from their original research protocol. Details of deviation from the protocol, irrespective of the reason for deviation, should always be reported. Reasons for the deviation and the implications on the research and conclusions should be discussed.

Another important potential limitation is changes in coding practices or eligibility criteria resulting from a change in the composition of the database populationterm, study populationterm, or both over time. The definition of the database populationterm may change under a number of circumstances, e.g., if enrolling practices cease collaboration with the database, change computer software, or change criteria for enrolment in the database change, such as a registry. The study populationterm in administrative data sources (e.g., insurance databases) may change if persons’ eligibility is not constant over time, because of changes in employment, residency status, or medical care provider. A change in the way recordsterm·mth are coded (e.g., upcoding or changes in coding systems, as described above) may alter the study populationterm [ 63 , 75 , 76 ]. When discussing limitations, authors should explain how changing eligibilityterm was handled in the analysis so the reader can assess the potential for biasterm. As detailed by STROBE, the discussion should include the direction and magnitude of any potential biasterm and efforts taken to address such biasterm.

Other Information RECORDterm·mth ITEM 22.1: Authors should provide information on how to access any supplemental information such as the study protocolterm, raw dataterm, or programming codeterm.

Examples The article by Taljaard and colleagues represents the full research protocol for a study using the Canadian Community Health Survey [ 77 ].

In their article, GuttmannpersName and colleagues invite requests for the study protocolterm: “Data sharing: The technical appendix, dataset creation plan/protocol, and statistical code are available from the corresponding author at [email address]” [ 78 ].

Explanation We strongly support the dissemination of detailed information on study methods and results. When possible, we encourage the prior or simultaneous publication of the study protocolterm, raw dataterm results, and, if applicable, programming codeterm. This information is useful to peer reviewers and readers in assessing the validity of study findings. A number of opportunities are available to researchers for open publication of such data. These include online journal supplementary material, personal websites, institutional websites, science-based social media sites (e.g., ResearchGate.net and Academia.edu), data repositoriesterm (e.g., Dryad or Figshare), or government open data websites [ 79 ]. We recognize that some research organizations, corporations, institutions, or laws may prohibit or restrict the free availability of such information. While a discussion of ownership and use of this intellectual property is outside the scope of the RECORDterm·mth guidelines, posting of such data should always be performed within the legal and ethical guidelines of the researchers’ institutional environment, with the guidance of journal editors. This information would also be useful to other researchers who may wish to access these data to replicate, reproduce, or expand upon the research described in the manuscript. Whatever the format or extent of available supplemental information, we recommend that reference to the location of this information be clearly stated in the manuscript.

Discussion The RECORDterm·mth guidelines are specific to observational research conducted using routinely collected health dataterm and serve to supplement, not replace, the STROBE guidelines. RECORDterm·mth was created as a guide for authors, journal editors, peer reviewers, and other stakeholders to encourage transparency and completeness of reporting of research conducted using routinely collected health dataterm. The checklist is intended for use by any researcher using such data, and we encourage wide dissemination to all interested parties. We anticipate that endorsement and implementation of RECORDterm·mth by journals will improve the transparency of reporting of research using routinely collected health dataterm.

Limitations Both STROBE and RECORDterm·mth are intended for application only to observational research studies. However, routinely collected health dataterm are sometimes used for research conducted with other study designs, such as cluster-randomized trialsterm for health system evaluation. In addition, linkage of data from randomized trials to administrative data can be used for long-term follow-up of outcomes, and associated studies would not be considered observational. As the field evolves, we expect to expand RECORDterm·mth to other research designs using similarly rigorous methods.

While RECORDterm·mth represents our best attempt to reflect the interest and priorities of stakeholders, we recognize that the methods used to conduct research using routinely collected health dataterm are changing rapidly, and the availability of types of data for such research is expanding. For example, mobile health applicationsterm (mHealth apps) are becoming widely available for smartphones and wearable technologies. While limited research is presently conducted using these data sources, we anticipate rapid growth in the use of these data in the near future, and new methodologies will be created to manage this resource. In addition, the working committee decided to focus on health data, and not on all data sources used to conduct health-related research (e.g., environmental data, financial data, etc.). Therefore, the RECORD checklistterm may not reflect themes that will become important in the future, and revision may be necessary at some point.

Extensive efforts were made to include a broad representation of stakeholders in the creation of these guidelines. We recruited stakeholders through open calls and targeted invitations using a variety of channels [ 16 ]. However, stakeholder representation was predominantly from regions conducting research using routinely collected health dataterm, with only a few representatives from developing nations and non-English speaking countries. Nevertheless, we believe that the stakeholder group was representative of the current community of researchers and users of the generated knowledge. While a great deal of input was obtained through surveys and feedback from the stakeholder group, feasibility dictated that the statements were crafted by a smaller working committee consisting of 19 members who met in person, as previously suggested in the literature [ 17 ]. In the future, technology and social media may allow for more active participation by larger groups in working committee meetings.

Future Directions and Community Engagement As the availability of routinely collected health dataterm expands, we expect more involvement of researchers from regions in which such data are not currently accessible. Through the record-statement.org website and message board, we expect ongoing commentary and discussion on the RECORDterm·mth document from interested parties, which may result in official revisions in the future. Through this online community, RECORDterm·mth will become a living documentterm that can adapt to changes in the field.

Publication of a reporting guideline and endorsement by journals are not sufficient to improve research reporting [ 80 ]. The manner in which the guidelines are implemented by researchers, journals, and peer reviewers are of key importance to RECORDterm·mth having a measurable impact [ 81 ]. Therefore, the online message board will include a discussion forum on implementation. We also encourage assessment of the impact of RECORDterm·mth on reporting in the field to ensure that the guidelines provide measurable benefit.

Conclusions The RECORDterm·mth statement expands the STROBE criteria to observational studies conducted using routinely collected health dataterm. With the input of the research and publishing community, we have created reporting guidelines in the form of a checklist and this accompanying explanatory document. Reporting guidelines have been demonstrated to improve reporting of research, thereby allowing for consumers of the research to be aware of the strengths, limitations, and accuracy of conclusions [ 12 , 82 – 84 ]. While we anticipate that RECORDterm·mth will change with the evolution of research methods in the field, these guidelines will help facilitate adequate reporting of research over the coming years. With implementation by authors, journal editors, and peer reviewers, we anticipate that RECORDterm·mth will result in transparency, reproducibility, and completeness of reporting of research conducted using routinely collected health dataterm.

Box 1. Definitions of Population Terms (Source, Database, and Study Populationsterm).

There are three levels of population hierarchyterm that are relevant in studies using routinely collected data and will be referred to throughout the statement. These populations include the source populationterm, which represents that from which the database populationterm is derived and hence about which the researchers want to make inferences; the database populationterm, which is derived from the source populationterm and comprises people with data included in the data source; and the study populationterm, identified from within the database populationterm by the researchers using codes and algorithms ( Fig 1 ) [ 7 ]. For example, in the case of the Clinical Practice Research Datalink (CPRD), the source populationterm comprises all people attending general practitioners in the United Kingdom. The database populationterm comprises those individuals included in CPRD, while the study populationterm comprises those selected from within CPRD using codes and algorithms to be described in the specific study.

Supporting Information (XLSX) Click here for additional data file.