standOff. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote shown on the same row, so the borrowing can be checked against the thing borrowed from without leaving this page. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.| relation to other work | claim_relations | how this paper stands to another: agrees, contradicts, extends, qualifies |
| claim type | claim_type | the shape of the claim: existence, mechanism, method, critique, prediction, null |
| concept edge | concept_edges | an asserted relation between two concepts, with the sentence asserting it |
| concept | concepts | an idea the paper works with, quoted where the paper uses it |
| contribution | contribution | what the paper claims to add |
| limitation | limitations | what the paper says its evidence cannot support |
| mechanism | mechanisms | how the reported effect is said to come about |
| method | method | how the paper produced its evidence |
| object of study | object_of_study | what the paper is about — the thing it investigates |
| phenomenon | phenomenon | the effect or regularity under study |
| research question | research_question | the question the paper sets itself |
| sense | senses | a distinct use of a term, quoted where the paper fixes it |
| unrecordable relation | unhoused_relations | a relation the paper asserts that this vocabulary cannot express, with the reason |
The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.
RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review
Abstract
object_of_studycontributionconcepts↗The RE-AIM planning and evaluation framework was conceptualized two decades ago. As one of the most frequently applied implementation frameworks, RE-AIM has now been cited in over 2,800 publications. This paper describes the application and evolution of RE-AIM as well as lessons learned from its use. RE-AIM has been applied most often in public health and health behavior change research, but increasingly in more diverse content areas and within clinical, community, and corporate settings. We discuss challenges of using RE-AIM while encouraging a more pragmatic use of key dimensions rather than comprehensive applications of all elements. Current foci of RE-AIM include increasing the emphasis on cost and adaptations to programs and expanding the use of qualitative methods to understand “how” and “why” results came about. The framework will continue to evolve to focus on contextual and explanatory factors related to RE-AIM outcomes, package RE-AIM for use by non-researchers, and integrate RE-AIM with other pragmatic and reporting frameworks.
Introduction
phenomenonconcepts↗concepts↗concepts↗concepts↗concepts↗concepts↗The RE-AIM framework ( 1 ) was developed to address the issue that the translation of scientific advances into practice, and especially into public health impact and policy, have been slow and inequitable ( 2 – 6 ). RE-AIM and other models ( 7 ) have helped balance the traditional focus on internal over external validity. Unique features of RE-AIM include an explicit focus on issues, dimensions, and steps in the design, dissemination, and implementation process that can either facilitate or impede success in achieving broad and equitable population-based impact.
research_questionmethod↗The seminal RE-AIM paper ( 1 ) has been cited over 2,800 times, and the RE-AIM framework has been applied to study planning or evaluation in over 450 publications ( 7 ). RE-AIM is one of the most frequently used frameworks for planning and evaluation of grant applications at most of the leading U.S. health and medical research agencies ( 8 ) and has been used widely (nationally and internationally) ( 9 , 10 ) and across populations, settings, and health conditions ( 11 – 24 ). Generally, RE-AIM does seem to translate and be useful in the different countries and cultures in which use has been reported. Some international applications include low- and middle-income countries ( 25 ) including Australia ( 26 – 30 ), the Netherlands ( 31 – 34 ), and Brazil ( 35 , 36 ). One interesting application was the use of RE-AIM to help plan and evaluate interventions to reduce the use of coal-fired indoor cook stoves in Africa ( 37 ). In this article, we summarize the history of the RE-AIM framework, discusses current applications of RE-AIM for research and practice, and outline opportunities for future application.
Historical Perspectives
The dimensions of the RE-AIM framework were originally introduced to encourage scientists to be more transparent and consider internal and external validity across pilot, efficacy, effectiveness, demonstration, and translational research ( 23 , 38 ). Most peer reviewed publications previously emphasized efficacy, leaving researchers, and practitioners with little information about the generalizability of the intervention context, implementation personnel and conditions, and findings. The main goal from its conception was to improve assessment and reporting along the five RE-AIM dimensions, not necessarily intervening to improve all dimensions (see Table 1 ).
The RE-AIM dimensions: definitions, evolution, and examples from the literature.
concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationsconcept_edgesconcept_edgesconcept_edgesconcept_edgesconcept_edgesconcept_edgesconcept_edgesThe RE-AIM dimensions include reach (R), effectiveness (E), and maintenance (M)–which operate at the individual-level (i.e., those who are intended to benefit), and adoption (A), implementation (I), and maintenance (M), which focus on the staff and setting levels. Setting-level RE-AIM factors are often multi-level and address context and external validity issues important to population impact. For example, settings may include clinics, schools, or worksites nested within communities or larger systems, and within these settings are nested clinicians, teachers, or human resources staff responsible for implementation.
concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationsunhoused_relationssensessensesconcept_edgesconcept_edgesAll RE-AIM dimensions are complex, but implementation currently has the most indices. It focuses on fidelity to an intervention: the extent to which the program is implemented consistently across different settings, staff, and patients. It also includes adaptations made ( 58 ) and costs from multiple stakeholder perspectives ( 59 ). Maintenance has indices at the individual- (long-term effectiveness) and setting-level (sustainability after original research funded is completed).
mechanisms↗concepts↗concepts↗sensesThe framework's operational components have been increasingly applied over the years. For example, in the past, studies reported participant characteristics that differed between study conditions or between those retained and those lost to follow-up. However, studies using RE-AIM compared the representativeness of individuals who enrolled in a study to the characteristics of the intended population. These comparisons used in RE-AIM studies increased understanding about access, awareness, appropriateness, and likely generalizability of recruitment strategies and intervention approaches.
In the past, clinical effectiveness research focused relatively narrowly on physiologic outcomes. RE-AIM expanded this focus to multiple factors that impact public health . This approach to assessing broader impacts aided in understanding comprehensive effects of a program on quality of life, including unintended consequences (e.g., increasing health inequity or the social stigma of labeling someone with a chronic condition).
limitationsThere have been several literature reviews on use of RE-AIM ( 11 , 42 , 52 , 60 – 62 ). The most comprehensive reviews have spanned the literature from 2000 to 2012 or 2015 ( 10 , 42 , 52 ). Notably, these reviews of different content areas reached similar conclusions: that adoption and maintenance, as well as representativeness across individual- and organizational-levels, were reported far less frequently. They identified frequent issues with confusing different dimensions, in particular reach (at the individual-level) and adoption (at the setting-level). These observations are not limited to the United States alone.
To enhance development and application of the framework, several scientists contributed to a RE-AIM research consortium funded by the Robert Wood Johnson Foundation ( 63 – 66 ). This work led to the development of a website, www.re-aim.org , in 2004 ( 64 ). The website serves as a repository of various resources and tools including self-quizzes, checklists, figures, tables, measures, tips for using RE-AIM, and increasingly, other social media tools. These are available to facilitate the operationalization and application of RE-AIM across diverse interventions, settings, and populations. To enhance a dialogue within the broader research community, monthly webinars are held about RE-AIM related issues; archived recordings are available on the website ( www.re-aim.org ).
From Past to Present
Below, we summarize five general areas currently being examined using RE-AIM ( Table 1 ). concepts↗claim_relationsThe first is to understand and maximize the potential of RE-AIM to assess adaptations prior to, during, and after program implementation ( 54 , 55 , 67 ). Adaptations naturally occur during the implementation of programs ( 68 ). Mittman et al. ( 69 ) suggest that instead of ignoring or suppressing this phenomenon, we should find ways to document and assess these changes. Recent Patient-Centered Outcomes Research Institute (PCORI) Methodology Standards ( 70 ) suggest that adaptations should be systematically documented. RE-AIM has great potential to provide guidance about documenting adaptations. It can also provide guidance about how to evaluate the impact of these adaptations, as well as their purpose ( 58 ). RE-AIM has been used to expand the widely known Stirman framework ( 71 ) on adaptations with additional components to address “who, what, why, where, and when” questions ( 67 ). RE-AIM considers adaptations in a longitudinal, multi-method, and multi-level manner and includes data collection at multiple time points and from multiple stakeholders, using multiple data collection approaches ( 54 , 55 , 67 ).
mechanisms↗concepts↗concepts↗concept_edgesSecond, there has been a recent focus on more qualitative RE-AIM assessments. Most evaluations and uses of the framework have emphasized descriptive or quantitative data, often focusing on key aspects such as the percentage of potentially eligible persons or settings that participate. A qualitative focus as presented by Holtrop et al. ( 72 ) can enhance understanding about what happened as well as the “how” and the “why.”
limitationsconcepts↗concepts↗unhoused_relationssensesThird, we recognized the need for more pragmatic uses of the framework rather than trying to comprehensively assess all RE-AIM dimensions in all applications, especially when not having many evaluation resources ( 67 , 73 ). All studies or evaluations, and particularly those without large evaluation budgets, do not need to assess all components of RE-AIM. Rather, they should address those components most valued and appropriate for their particular question, setting, stakeholders, and stage of research. An a priori decision should be made, however, to select the dimensions on which to focus for evaluation and on which to use for planning and improvement (i.e., beyond the evaluation scope). In some cases, the decision to capture all five dimensions is made a priori to understand individual impacts, contextual implications, and feasibility of ongoing data collection. This is demonstrated in two recent applications of RE-AIM before, during, and after program implementation—and on limited funds ( 74 , 75 ). Both applications highlighted the need for stakeholder buy-in ( 54 ) and operationalization ( 67 ) of each dimension that holds value for these stakeholders.
concepts↗claim_relationsFourth, assessment of costs, from the perspective of multiple stakeholders and across the various RE-AIM dimensions, is another area of emphasis ( 5 ). Building upon earlier work by Ritzwoller et al. ( 76 ), recent RE-AIM cost assessments have focused on the multilevel nature of implementation, different stakeholder perspectives, and cost estimates for replicating a program or policy in different settings. Rhodes et al. ( 59 ) have provided templates to assess costs at the patient-, staff-, clinic-, and organizational-levels. Costs to deliver programs are associated with activities to address and enhance each RE-AIM dimension ( Figure 1 ). In the future, we anticipate more consistent reporting of costs and burden and more frequent comparative effectiveness research about cost-effective methods to enhance value and various RE-AIM dimensions.
Revised, enhanced RE-AIM/PRISM 2019 model.
concepts↗claim_relationsFinally, Glasgow et al. ( 77 ) have recently advocated for an extension of RE-AIM concepts and dimensions, termed an Expanded CONSORT Figure to enhance transparent reporting, and potentially, replication. The goal is to expand the CONSORT reporting criteria required for randomized studies ( 78 ) to (a) include factors related to setting and staff level participation and representativeness, which begin before individual participants are recruited, and (b) extend the temporal focus beyond the end of a study. The expanded CONSORT figure and a related downloadable template summarize issues of exclusion and inclusion criteria for settings (e.g., communities or healthcare networks) and delivery staff, [e.g., evaluating the percent and characteristics of settings and staff that participate or do not (adoption)], reasons for participation or non-participation, and intervention sustainability after project support ends ( 79 ).
concepts↗concepts↗concepts↗unhoused_relationsconcept_edgesconcept_edgesBased on these observations, we have developed a new RE-AIM figure to highlight the various changes to the model, as well as new emphases, including explicit inclusion of costs and adaptations, as shown in Figure 1 . The figure also emphasizes key multi-level contextual factors (both the internal and external context) that influence RE-AIM outcomes as discussed below. Two crosscutting issues are: (a) that it is critical that there is alignment across setting and context, the intervention and implementation strategies; and (b) it is important to include qualitative assessments to determine how and why various RE-AIM outcomes are produced.
What Will the Future Bring?
With the historical context and current application of RE-AIM in mind, we outline five future directions for researchers and practitioners interested in using RE-AIM.
concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationssensesconcept_edgesconcept_edgesconcept_edgesFirst, there have been recent calls to more explicitly describe strategies and context ( 80 , 81 ) as well as test mediating relationships between implementation strategies and implementation outcomes ( 82 ). We also see this as an emerging area for RE-AIM. The most well-articulated attempt to do this so far is the Practical, Robust, Implementation, and Sustainability Model (PRISM) ( 83 , 84 ) that focuses on specific contextual factors from external macro-level factors such as policies, guidelines, and incentives, to more local organizational-level factors. It focuses on the fit between the characteristics of an intervention (i.e., Rogers' constructs of relative advantage, complexity, compatibility, observability, trialability, and cost) ( 85 ) and the particular intervention and implementation system. A somewhat unique factor of PRISM is its focus on enhancing setting-level maintenance characteristics by addressing the “implementation and sustainability infrastructure”—including job requirements, ongoing audit and feedback, and institutionalization of intervention activities ( 84 , 86 ).
mechanisms↗limitationsconcepts↗concepts↗unhoused_relationsSecond, mixed-methods should be used across framework components to identify explanatory processes across RE-AIM dimensions. To date, quantitative measures alone have been insufficient to strongly predict dissemination (reach and adoption), implementation, and maintenance outcomes. Using mixed-methods approaches can help identify factors that are causally related to different RE-AIM outcomes in different situations ( 72 ). Qualitative information integrated with newer predictive modeling approaches should provide more detailed guidance about actions that can be taken to enhance outcomes by addressing empirically-derived causal relationships.
mechanisms↗concepts↗concept_edgesThird, we encourage more iterative applications of RE-AIM and use of the framework during the implementation period, not just for initial planning and summative evaluation. Rapid, iterative use and analysis of brief practical measures of RE-AIM factors can inform adaptations ( 55 , 66 , 71 ). In brief, RE-AIM can be used as part of a participatory approach (Estabrooks et al., under review), to determine which dimensions should be assessed, described, or targeted for intervention. For example, a recruitment strategy may need to be adapted over the life-course of an intervention (to improve reach) or a new training strategy may be employed (to improve adoption and implementation). While often examined and interpreted independently, these adaptations can work together to be empirically robust and practically meaningful.
limitationsconcepts↗concepts↗unhoused_relationsAnother issue to be addressed is use of RE-AIM by non-researchers and groups such as state health programs or program evaluators without substantial funds (grants/contracts). Using RE-AIM in low-resource and real-world settings can be challenging but successful ( 2 ). Preliminary findings assessing such use are that RE-AIM is used widely, and seems to be relatively intuitive, but there are challenges implementing it at a detailed level and assessing all components. The development of user-friendly tools and aids using human centered design, as well as more examples of the application of RE-AIM for such users is an important future direction.
concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationsclaim_relationsclaim_relationsclaim_relationsFinally, we think there is great opportunity for RE-AIM to be used in combination with other approaches such as the Pragmatic Explanatory Continuum Indicator Summary (PRECIS) model ( 87 , 88 ), where RE-AIM factors can be combined with the PRECIS-2 dimensions to determine how pragmatic a study is and how generalizable it is likely to be. Such use is illustrated in a recent systematic review by Luoma et al. ( 89 ), who demonstrated how reviews can simultaneously summarize effectiveness (using Cochrane-type criteria) and pragmatism (using a combination of PRECIS-2 and RE-AIM factors). RE-AIM and its Expanded CONSORT extension could also be integrated with the Standards for Reporting Implementation Studies (StaRI) ( 90 ) or other dissemination and implementation (D&I) research reporting criteria.
Conclusion
claim_typeRE-AIM has been applied in research and practice for 20 years. Although its original components have remained, much has been modified and evolved to address emerging issues such as adaptation and dissemination costs. We expect that RE-AIM will continue to evolve to better address and enhance its original purpose—to increase the prevalence of relevant research that can be applied broadly across a wide variety of populations and settings to achieve a large, equitable, and replicable public health impact. We invite researchers and practitioners to contribute to the expanded use of RE-AIM before, during, and after intervention delivery.
Author Contributions
All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; 16 of 16 relations here carry that mark. The rest are the tagger’s reading of the sentence quoted beside them.
This record holds no controls. A layer is empty when the tagger recorded nothing in it — which is not the same as the paper having nothing to record.
| facet | tag | field | conf. | whose words |
|---|---|---|---|---|
| object_of_study | the 20-year evolution, application, and prospective use of the RE-AIM planning and evaluation framework | object_of_study | the paper's words | |
| research_question | How has RE-AIM evolved from its original dimensions, how is it now applied, and what directions should guide its future use? | research_question | the paper's words | |
| contribution | a 20-year synthesis that clarifies RE-AIM's levels and dimensions, documents shifts toward adaptation, cost, qualitative and pragmatic use, and proposes contextual, iterative, and integrated future directions | contribution | the paper's words | |
| claim_type | existence | claim_type | the paper's words | |
| phenomenon | slow and inequitable translation of scientific advances into practice, public-health impact, and policy | phenomenon | the paper's words | |
| method | review also in: review | method | the paper's words | |
| mechanisms | mechanisms | mechanisms[0] | 0.99 | the tagger's words |
| mechanisms | mechanisms | mechanisms[1] | 0.99 | the tagger's words |
| mechanisms | mechanisms | mechanisms[2] | 0.99 | the tagger's words |
| mechanisms | mechanisms | mechanisms[3] | 0.99 | the tagger's words |
| limitations | comprehensive assessment of every RE-AIM component is unnecessary and may be infeasible when evaluation resources are limited | limitations[0] | 0.99 | the tagger's words |
| limitations | quantitative measures alone have been insufficient to strongly predict reach, adoption, implementation, and maintenance outcomes | limitations[1] | 0.99 | the tagger's words |
| limitations | detailed use and assessment of every component remains challenging for non-researchers in low-resource real-world settings | limitations[2] | 0.99 | the tagger's words |
| limitations | the reviewed literature reported adoption, maintenance, and representativeness much less often than other dimensions | limitations[3] | 0.99 | the tagger's words |
| concepts | RE-AIM framework also in: RE-AIM framework | concepts[0] | 0.99 | the paper's words |
| concepts | implementation frameworks also in: implementation frameworks | concepts[1] | 0.99 | the paper's words |
| concepts | translation of scientific advances into practice | concepts[2] | 0.99 | the paper's words |
| concepts | public health impact also in: public health impact | concepts[3] | 0.99 | the paper's words |
| concepts | traditional focus on internal over external validity | concepts[4] | 0.99 | the paper's words |
| concepts | external validity also in: external validity | concepts[5] | 0.99 | the paper's words |
| concepts | broad and equitable population-based impact | concepts[6] | 0.99 | the paper's words |
| concepts | RE-AIM dimensions also in: RE-AIM dimensions | concepts[7] | 0.99 | the paper's words |
| concepts | reach also in: reach | concepts[8] | 0.99 | the paper's words |
| concepts | effectiveness also in: effectiveness | concepts[9] | 0.99 | the paper's words |
| concepts | maintenance also in: maintenance | concepts[10] | 0.99 | the paper's words |
| concepts | adoption also in: adoption | concepts[11] | 0.99 | the paper's words |
| concepts | implementation also in: implementation | concepts[12] | 0.99 | the paper's words |
| concepts | individual-level also in: individual-level | concepts[13] | 0.99 | the paper's words |
| concepts | staff and setting levels also in: staff and setting levels | concepts[14] | 0.99 | the paper's words |
| concepts | population impact also in: population impact | concepts[15] | 0.99 | the paper's words |
| concepts | clinics, schools, or worksites also in: clinics, schools, or worksites | concepts[16] | 0.99 | the paper's words |
| concepts | communities or larger systems also in: communities or larger systems | concepts[17] | 0.99 | the paper's words |
| concepts | clinicians, teachers, or human resources staff | concepts[18] | 0.99 | the paper's words |
| concepts | fidelity to an intervention also in: fidelity to an intervention | concepts[19] | 0.99 | the paper's words |
| concepts | extent to which the program is implemented consistently | concepts[20] | 0.99 | the paper's words |
| concepts | adaptations also in: adaptations | concepts[21] | 0.99 | the paper's words |
| concepts | costs also in: costs | concepts[22] | 0.99 | the paper's words |
| concepts | long-term effectiveness also in: long-term effectiveness | concepts[23] | 0.99 | the paper's words |
| concepts | sustainability after original research funded is completed | concepts[24] | 0.99 | the paper's words |
| concepts | representativeness also in: representativeness | concepts[25] | 0.99 | the paper's words |
| concepts | recruitment strategies also in: recruitment strategies | concepts[26] | 0.99 | the paper's words |
| concepts | qualitative RE-AIM assessments also in: qualitative RE-AIM assessments | concepts[27] | 0.99 | the paper's words |
| concepts | understanding about what happened also in: understanding about what happened | concepts[28] | 0.99 | the paper's words |
| concepts | pragmatic uses of the framework also in: pragmatic uses of the framework | concepts[29] | 0.99 | the paper's words |
| concepts | stakeholder buy-in also in: stakeholder buy-in | concepts[30] | 0.99 | the paper's words |
| concepts | Expanded CONSORT Figure also in: Expanded CONSORT Figure | concepts[31] | 0.99 | the paper's words |
| concepts | new RE-AIM figure also in: new RE-AIM figure | concepts[32] | 0.99 | the paper's words |
| concepts | multi-level contextual factors also in: multi-level contextual factors | concepts[33] | 0.99 | the paper's words |
| concepts | alignment also in: alignment | concepts[34] | 0.99 | the paper's words |
| concepts | mixed-methods also in: mixed-methods | concepts[35] | 0.99 | the paper's words |
| concepts | explanatory processes also in: explanatory processes | concepts[36] | 0.99 | the paper's words |
| concepts | Rapid, iterative use and analysis also in: Rapid, iterative use and analysis | concepts[37] | 0.99 | the paper's words |
| concepts | implementation and sustainability infrastructure | concepts[38] | 0.99 | the paper's words |
| concepts | job requirements also in: job requirements | concepts[39] | 0.99 | the paper's words |
| concepts | ongoing audit and feedback also in: ongoing audit and feedback | concepts[40] | 0.99 | the paper's words |
| concepts | institutionalization of intervention activities | concepts[41] | 0.99 | the paper's words |
| concepts | Practical, Robust, Implementation, and Sustainability Model (PRISM) | concepts[42] | 0.99 | the paper's words |
| concepts | PRISM also in: PRISM | concepts[43] | 0.99 | the paper's words |
| concepts | human centered design also in: human centered design | concepts[44] | 0.99 | the paper's words |
| concepts | user-friendly tools and aids also in: user-friendly tools and aids | concepts[45] | 0.99 | the paper's words |
| concepts | PRECIS-2 dimensions also in: PRECIS-2 dimensions | concepts[46] | 0.99 | the paper's words |
| concepts | Standards for Reporting Implementation Studies (StaRI) | concepts[47] | 0.99 | the paper's words |
| concepts | StaRI also in: StaRI | concepts[48] | 0.99 | the paper's words |
| unhoused_relations | reach → individual-level Reach operates at the individual level. | unhoused_relations[0] | 0.99 | the tagger's words |
| unhoused_relations | fidelity to an intervention → extent to which the program is implemented consistently Fidelity means the extent to which the program is implemented consistently across settings, staff, and patients. | unhoused_relations[1] | 0.99 | the tagger's words |
| unhoused_relations | maintenance → long-term effectiveness At the individual level, maintenance has long-term effectiveness as an index. | unhoused_relations[2] | 0.99 | the tagger's words |
| unhoused_relations | maintenance → sustainability after original research funded is completed At the setting level, maintenance has post-funding sustainability as an index. | unhoused_relations[3] | 0.99 | the tagger's words |
| unhoused_relations | adoption → staff and setting levels Adoption focuses on staff and setting levels. | unhoused_relations[4] | 0.99 | the tagger's words |
| unhoused_relations | pragmatic uses of the framework → stakeholder buy-in Pragmatic RE-AIM applications depend on stakeholder-valued dimension selection and stakeholder buy-in. | unhoused_relations[5] | 0.99 | the tagger's words |
| unhoused_relations | alignment → multi-level contextual factors Alignment among context, intervention, and implementation strategies is critical to the revised model. | unhoused_relations[6] | 0.99 | the tagger's words |
| unhoused_relations | mixed-methods → explanatory processes Mixed methods should be used to identify explanatory processes across RE-AIM dimensions. | unhoused_relations[7] | 0.99 | the tagger's words |
| unhoused_relations | human centered design → user-friendly tools and aids Human-centered design is proposed for developing user-friendly tools and aids for non-researchers. | unhoused_relations[8] | 0.99 | the tagger's words |
| unhoused_relations | Practical, Robust, Implementation, and Sustainability Model (PRISM) → PRISM PRISM is the parenthetical abbreviation of the Practical, Robust, Implementation, and Sustainability Model. | unhoused_relations[9] | 0.99 | the tagger's words |
| unhoused_relations | Standards for Reporting Implementation Studies (StaRI) → StaRI StaRI is the parenthetical abbreviation of Standards for Reporting Implementation Studies. | unhoused_relations[10] | 0.99 | the tagger's words |
| unhoused_relations | RE-AIM framework → PRECIS-2 dimensions RE-AIM factors can be combined with PRECIS-2 dimensions to assess pragmatism and likely generalizability. | unhoused_relations[11] | 0.99 | the tagger's words |
| senses | `maintenance` has TWO LEVEL-SPECIFIC SENSES in this review: long-term effectiveness for individuals and sustainability after research funding ends for settings | senses[0] | 0.99 | the tagger's words |
| senses | `implementation` is broader than fidelity alone: its current indices also include adaptations and costs from multiple stakeholder perspectives | senses[1] | 0.99 | the tagger's words |
| senses | `pragmatic` means choosing in advance the dimensions appropriate to the question, setting, stakeholders, and research stage, not automatically measuring all five | senses[2] | 0.99 | the tagger's words |
| senses | `context` is explicitly MULTI-LEVEL and both internal and external; it includes policies and incentives at the macro level and local organizational factors | senses[3] | 0.99 | the tagger's words |
| senses | `representativeness` is assessed against the intended population, which turns recruitment reporting into evidence about access, appropriateness, and likely generalizability | senses[4] | 0.99 | the tagger's words |
| claim_relations | extends → Stirman framework on adaptations note: The paper explicitly says RE-AIM has been used to expand the Stirman framework. | claim_relations[0] | 0.99 | the tagger's words |
| claim_relations | builds-on → earlier RE-AIM cost work by Ritzwoller et al. note: The paper explicitly says the cost assessment builds upon this earlier work. | claim_relations[1] | 0.99 | the tagger's words |
| claim_relations | extends → CONSORT reporting criteria note: The stated goal is to expand CONSORT across settings, staff, representativeness, and time. | claim_relations[2] | 0.99 | the tagger's words |
| claim_relations | complements → Pragmatic Explanatory Continuum Indicator Summary (PRECIS) model note: The paper says RE-AIM can be used in combination with PRECIS; 'can' makes this prospective. | claim_relations[3] | 0.99 | the tagger's words |
| claim_relations | supports → systematic review by Luoma et al. note: The cited review is presented as a demonstration of combined effectiveness and pragmatism assessment. | claim_relations[4] | 0.99 | the tagger's words |
| claim_relations | complements → Standards for Reporting Implementation Studies (StaRI) note: The paper says RE-AIM could be integrated with StaRI; this is explicitly prospective. | claim_relations[5] | 0.99 | the tagger's words |
| concept_edges | reach —part-of→ RE-AIM dimensions the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical membership stated by 'include'. · the relation stated: Reach is included among the RE-AIM dimensions. also in: reach · RE-AIM dimensions | concept_edges[0] | 0.99 | the tagger's words |
| concept_edges | effectiveness —part-of→ RE-AIM dimensions the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical membership stated by 'include'. · the relation stated: Effectiveness is included among the RE-AIM dimensions. also in: effectiveness · RE-AIM dimensions | concept_edges[1] | 0.99 | the tagger's words |
| concept_edges | maintenance —part-of→ RE-AIM dimensions the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical membership stated by 'include'. · the relation stated: Maintenance is included among the RE-AIM dimensions. also in: maintenance · RE-AIM dimensions | concept_edges[2] | 0.99 | the tagger's words |
| concept_edges | adoption —part-of→ RE-AIM dimensions the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical membership stated by 'include'. · the relation stated: Adoption is included among the RE-AIM dimensions. also in: adoption · RE-AIM dimensions | concept_edges[3] | 0.99 | the tagger's words |
| concept_edges | implementation —part-of→ RE-AIM dimensions the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical membership stated by 'include'. · the relation stated: Implementation is included among the RE-AIM dimensions. also in: implementation · RE-AIM dimensions | concept_edges[4] | 0.99 | the tagger's words |
| concept_edges | clinics, schools, or worksites —part-of→ communities or larger systems the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical nesting is explicit. · the relation stated: Clinics, schools, or worksites are nested within communities or larger systems. | concept_edges[5] | 0.99 | the tagger's words |
| concept_edges | clinicians, teachers, or human resources staff —part-of→ clinics, schools, or worksites the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical nesting is explicit across adjacent sentences. · the relation stated: Implementation personnel are nested within the named settings. | concept_edges[6] | 0.99 | the tagger's words |
| concept_edges | adaptations —part-of→ implementation the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical inclusion is explicit. · the relation stated: Implementation includes adaptations made to the program. also in: adaptations · implementation | concept_edges[7] | 0.99 | the tagger's words |
| concept_edges | costs —part-of→ implementation the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical inclusion is explicit. · the relation stated: Implementation includes costs from multiple stakeholder perspectives. also in: costs · implementation | concept_edges[8] | 0.99 | the tagger's words |
| concept_edges | qualitative RE-AIM assessments —causes→ understanding about what happened direction on the measure: increases · the relation itself is separately evidenced · strength of the relation: 0.65 — the record marks this weak · note: The paper uses the modal 'can enhance', so this is a hedged positive causal claim. · the relation stated: A qualitative focus can enhance understanding of what happened and how and why. | concept_edges[9] | 0.99 | the tagger's words |
| concept_edges | costs —part-of→ new RE-AIM figure the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical inclusion is explicit. · the relation stated: Costs are explicitly included in the new RE-AIM figure. also in: costs · new RE-AIM figure | concept_edges[10] | 0.99 | the tagger's words |
| concept_edges | adaptations —part-of→ new RE-AIM figure the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical inclusion is explicit. · the relation stated: Adaptations are explicitly included in the new RE-AIM figure. also in: adaptations · new RE-AIM figure | concept_edges[11] | 0.99 | the tagger's words |
| concept_edges | job requirements —part-of→ implementation and sustainability infrastructure the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical inclusion follows 'including'. · the relation stated: Job requirements are included in implementation and sustainability infrastructure. | concept_edges[12] | 0.99 | the tagger's words |
| concept_edges | ongoing audit and feedback —part-of→ implementation and sustainability infrastructure the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical inclusion follows 'including'. · the relation stated: Ongoing audit and feedback are included in implementation and sustainability infrastructure. | concept_edges[13] | 0.99 | the tagger's words |
| concept_edges | institutionalization of intervention activities —part-of→ implementation and sustainability infrastructure the relation itself is separately evidenced · strength of the relation: 1 · note: Categorical inclusion follows 'including'. · the relation stated: Institutionalization of intervention activities is included in implementation and sustainability infrastructure. | concept_edges[14] | 0.99 | the tagger's words |
| concept_edges | Rapid, iterative use and analysis —causes→ adaptations the relation itself is separately evidenced · strength of the relation: 0.65 — the record marks this weak · note: The paper's 'can inform' wording is a hedged positive influence claim. · the relation stated: Rapid, iterative use and analysis of RE-AIM measures can inform adaptations. also in: Rapid, iterative use and analysis · adaptations | concept_edges[15] | 0.99 | the tagger's words |
Tags that could not be placed in the text:
What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.