standOff. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote shown on the same row, so the borrowing can be checked against the thing borrowed from without leaving this page. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.| relation to other work | claim_relations | how this paper stands to another: agrees, contradicts, extends, qualifies |
| claim type | claim_type | the shape of the claim: existence, mechanism, method, critique, prediction, null |
| concept edge | concept_edges | an asserted relation between two concepts, with the sentence asserting it |
| concept | concepts | an idea the paper works with, quoted where the paper uses it |
| contribution | contribution | what the paper claims to add |
| limitation | limitations | what the paper says its evidence cannot support |
| mechanism | mechanisms | how the reported effect is said to come about |
| method | method | how the paper produced its evidence |
| object of study | object_of_study | what the paper is about — the thing it investigates |
| phenomenon | phenomenon | the effect or regularity under study |
| research question | research_question | the question the paper sets itself |
| sense | senses | a distinct use of a term, quoted where the paper fixes it |
| unrecordable relation | unhoused_relations | a relation the paper asserts that this vocabulary cannot express, with the reason |
From Classification to Causality: Advancing Understanding of Mechanisms of Change in Implementation Science
Abstract
contributionclaim_typelimitationsconcepts↗concepts↗concepts↗concepts↗concepts↗concepts↗claim_relationsThe science of implementation has offered little toward understanding how different implementation strategies work. To improve outcomes of implementation efforts, the field needs precise, testable theories that describe the causal pathways through which implementation strategies function. In this perspective piece, we describe a four-step approach to developing causal pathway models for implementation strategies.
object_of_studylimitationsconcepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗First, it is important to ensure that implementation strategies are appropriately specified. Some strategies in published compilations are well defined but may not be specified in terms of its core component that can have a reliable and measureable impact. Second, linkages between strategies and mechanisms need to be generated. Existing compilations do not offer mechanisms by which strategies act, or the processes or events through which an implementation strategy operates to affect desired implementation outcomes. Third, it is critical to identify proximal and distal outcomes the strategy is theorized to impact, with the former being direct, measurable products of the strategy and the latter being one of eight implementation outcomes ( 1 ). Finally, articulating effect modifiers, like preconditions and moderators, allow for an understanding of where, when, and why strategies have an effect on outcomes of interest.
concepts↗concepts↗concepts↗concepts↗concepts↗We argue for greater precision in use of terms for factors implicated in implementation processes; development of guidelines for selecting research design and study plans that account for practical constructs and allow for the study of mechanisms; psychometrically strong and pragmatic measures of mechanisms; and more robust curation of evidence for knowledge transfer and use.
Keywords: implementation, mechanism, mediator, moderator, theory, causal pathway, strategy
Background: Why Build Causal Pathway Models?
phenomenonmechanisms↗limitationslimitationslimitationsconcepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsconcept_edgesIn recent years, there has been growing recognition of the importance of implementing evidence-based practices as a way to improve the quality of health care and public health. However, the results of implementation efforts have been mixed. About two-thirds of efforts fail to achieve the intended change ( 2 ), and nearly half have no effect on outcomes of interest ( 3 ). Implementation strategies are often mismatched to barriers [e.g., training, a strategy that could affect implementation outcomes through changes in an individual’s knowledge (intrapersonal-level), is used inappropriately to address an organizational-level barrier like poor culture] ( 4 ), and implementation efforts are increasingly complex and costly without enhanced impact ( 5 ). These suboptimal outcomes are due, in large part, to the dearth of tested theory in the field of implementation science ( 6 ). In particular, the field has a limited understanding of how different implementation strategies work—the specific causal mechanisms through which implementation strategies influence care delivery [ 7 ; Lewis et al. (under review) 1 ]. As a consequence, implementation science has been limited in its ability to effectively inform implementation practice by providing guidance about when and in what contexts specific implementation strategies should be used and, just as importantly, when they should not.
concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗sensesclaim_relationsThe National Academy of Science defines “science” as “the use of evidence to construct testable explanations and predictions of natural phenomena, as well as the knowledge generated through this process.” ( 8 ) The field of implementation has spent the past two decades building and organizing knowledge, but we are far from having testable explanations that afford us the ability to generate predictions. To improve outcomes of implementation efforts, the field needs testable theories that describe the causal pathways through which implementation strategies function ( 6 , 9 ). Unlike frameworks, which offer a basic conceptual structure underlying a system or concept ( 10 ), theories provide a testable way of explaining phenomena by specifying relations among variables, thus enabling prediction of outcomes ( 10 , 11 ).
concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationsunhoused_relationsunhoused_relationssensesclaim_relationsCausal pathway models represent interrelations among variables and outcomes of interest in a given context (i.e., the building blocks of implementation theory). Specifying the structure of causal relations enables scientists to empirically test whether the implementation strategies are operating via theorized mechanisms, how contextual factors moderate the causal processes through which implementation strategies operate, and how much variance in outcomes is accounted for by those mechanisms. Findings from studies based on causal models can, over time, both help the field develop more robust theories about implementation processes and advance the practice of implementation by addressing key issues. For instance, causal models can do the following: (1) inform the development of improved implementation strategies, (2) identify mutable targets for new strategies, (3) increase the impact of existing strategies, and (4) prioritize which strategies to use in which contexts.
research_questionconcepts↗concepts↗concepts↗concepts↗concepts↗In this perspective piece, we propose an approach to theory development by specifying, in the form of causal pathway models, hypotheses about the causal operation of different implementation strategies in various settings, so that these hypotheses can be tested and refined. Specifically, we offer a four-step process to developing causal pathway models for implementation strategies. Toward this end, we argue the field must move beyond having lists of variables that can rightly be considered determinants [i.e., factors that obstruct or enable change in provider behavior or health-care delivery processes ( 12 )], and toward precise articulation of mediators, moderators, preconditions, and (proximal versus distal) outcomes (see Table 1 for definitions).
Terms and definitions.
Building Causal Pathway Models
method↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗claim_relationsOur perspective draws upon Agile Science ( 13 , 14 )—a new method for developing and studying behavioral interventions that focuses on intervention modularity, causal modeling, and efficient evaluations to generate empirical evidence with clear boundary conditions (in terms of population, context, behavior, etc.) to maximize knowledge accumulation and repurposing. Agile Science has been used to investigate goal-setting interventions for physical activity, engagement strategies for mobile health applications, depression interventions for primary care, and automated dietary cues to promote weight loss ( 13 , 15 ). Applied to implementation strategies, Agile Science-informed causal pathway diagram modeling consists of at least four steps: (1) specifying implementation strategies; (2) generating strategy-mechanism linkages; (3) identifying proximal and distal outcomes; and (4) articulating moderators and preconditions. To demonstrate this approach, we offer examples of causal pathway models for a set of three diverse implementation strategies (see Figure 1 ). The strategies are drawn from the following example. A community mental health center is planning to implement measurement-based care in which providers solicit patient-reported outcome data [e.g., Patient Health Questionnaire 9-item depression symptom severity measure ( 16 )] prior to clinical encounters to inform treatment ( 17 ). The community mental health center plans to use training, financial penalty (disincentives), and audit and feedback as they are common strategies used to support measurement-based care implementation ( 18 ).
Causal model diagrams.
Step 1: Specifying Implementation Strategies
mechanisms↗limitationsconcepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationssensesThe Expert Recommendations for Implementing Change study yielded a compilation of 73 implementation strategies ( 19 ) developed by a multidisciplinary team through a structured literature review ( 20 ), Delphi process, and concept mapping exercise ( 19 , 21 , 22 ). Thus, there exists a solid foundation of strategies that are conceptually clear and well defined. However, the compilation was never explicitly linked to mechanisms. Following Kazdin ( 7 ), we define “mechanisms” as the processes or events through which an implementation strategy operates to effect desired implementation outcomes. Upon careful examination, it seems many strategies are not well enough specified to be linked to mechanisms in a coherent manner, a key step in causal model building. For instance, the compilation of 73 strategies lists “learning collaboratives,” a general approach for which the discrete strategies or core components are underspecified. This makes it difficult to identify their precise mechanisms of action ( 23 ). Underspecified strategies also leave the field vulnerable to inappropriately synthesizing data across studies ( 24 , 25 ).
concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗sensesIn our case example, training is a strategy that is underspecified. We adapted procedures from Michie et al. ( 26 ) to guide strategy specification recommending that each strategy be assessed for whether it: (1) aims to promote the adoption, implementation, sustainment, or scale-up of an evidence-based practice; (2) is a proposed “active ingredient” of adoption, implementation, sustainment, or scale-up; (3) represents the smallest component while retaining the proposed active ingredient; (4) can be used alone or in combination with other discrete strategies; (5) is observable and replicable; and (6) can have a measureable impact on specified mechanisms of implementation (and, if so, whether putative mechanisms can be listed). If strategies do not meet these criteria, they require revision and further specification. This could involve suggesting alternative definitions, eliminating an implementation strategy altogether, or articulating a new, narrower strategy that is a component or a type of the original strategy. Training would meet all but the third and sixth criteria (listed previously), because training can be comprised of several active ingredients (e.g., didactics, modeling, role play/rehearsal, feedback, shadowing) each of which may operate on an unique mechanism. In this case, training ought to be more narrowly defined to make clear its core components.
Step 2: Generating Strategy-Mechanism Linkages
mechanisms↗mechanisms↗mechanisms↗mechanisms↗mechanisms↗mechanisms↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationssensesOnce specified, an implementation strategy needs to be linked to the mechanisms hypothesized to underlie its functioning. Mechanisms explain how an implementation strategy has an effect by describing the actions that lead from the administration of the strategy to the implementation outcomes (see Table 1 for definitions). Statistically speaking, mechanisms are always mediators, but mediators may not be mechanisms. Similarly, moderators can point toward mechanisms but are not themselves reliably mechanisms. Determinants may explain why an implementation strategy did or did not have an effect, but mechanisms explain how a strategy had an effect, by, for example, altering the status of a determinant. Determinants are naturally occurring, and often but not always, malleable factors that could prevent or enable the strategy to affect the desired outcomes. Mechanisms are intentionally activated by the application of an implementation strategy and can operate at different levels of analysis, such as at the levels of intrapersonal (e.g., learning), interpersonal (e.g., sharing), organizational (e.g., leading), community (e.g., restructuring), and macro policy (e.g., guiding) ( 27 ). For an implementation effort to be successful, chosen strategies should be compatible with and able to act on the local determinants [e.g., provider habit (determinant) is addressed with clinical decision support (strategy) via self-reflection/reflecting (mechanism)]. Although commonly used in implementation science, we propose that the notion of a determinant is insufficiently specific as researchers have used it to refer to at least two types of variables in a causal process: proximal outcomes and effect modifiers (see text footnote 1). Our discussion below uses these more precise terms instead.
mechanisms↗limitationsconcepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationsclaim_relationsMost implementation strategies likely act via multiple mechanisms, although it remains an empirical question whether one mechanism is primary and others are ancillary. It is also likely that the same mechanism might be involved in the operation of multiple implementation strategies. Initial assessment of strategy-mechanism linkages is made in the context of the broader scientific knowledge base about how a strategy produces an outcome ( 7 ). For instance, many strategies have their own literature base (e.g., audit and feedback) ( 28 ) that offer theoretical and empirical insights about which mechanisms might be underlying the functioning of those strategies [e.g., reflecting, learning, and engaging ( 28 )]. Effort should always be made to draw upon and test existing theories, but if none offer sufficient guidance, hypothesizing variables that may have causal influence remains critical. In this way, over time, the initially formulated strategy-mechanism linkages can be reassessed and refined as studies begin to test them empirically. While such empirical evaluations are currently rare—across two systematic reviews of implementation mechanisms, only 31 studies were identified and no mechanisms were empirically established (see text footnote 1; 29 )—the causal pathway models we propose here are explicitly intended to facilitate evaluations of the mechanistic processes through which implementation strategies operate.
Step 3: Identifying Proximal and Distal Outcomes
concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationssensesImplementation scientists have isolated eight outcomes as the desired endpoints of implementation efforts: acceptability, feasibility, appropriateness, adoption, penetration, fidelity, cost, and sustainability ( 1 ). Many of these outcomes are appropriately construed as latent variables, but others are manifest/observable in nature ( 30 ); a recent systematic review offers measures of these outcomes and measure meta-data ( 31 ). In terms of the causal processes through which implementation strategies operate, these outcomes are often best conceptualized as distal outcomes that the implementation process is intended to achieve, and each of them may be more salient at one phase of implementation than another. For instance, with the Exploration, Preparation, Implementation, Sustainment Framework ( 32 ), acceptability of an evidence-based practice may be most salient in the exploration phase, whereas fidelity may be the goal of an implementation phase. Despite the plausible temporal interrelations among the outcomes, mounting evidence indicates that not all implementation strategies influence each of the aforementioned outcomes (e.g., workshop training can influence adoption but not fidelity) ( 33 ). To fully establish the plausibility of an implementation mechanism and a testable causal pathway, proximal outcomes must be expounded.
mechanisms↗limitationslimitationsconcepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationssensesconcept_edgesProximal outcomes are direct, measurable, and typically observable, products of the implementation strategy that occur because of its specific mechanism of action. That is, affecting a proximal outcome in the intended direction can confirm/disconfirm activation of the putative mechanism, offering a low-inference way to establish evidence for a theorized mechanism. Most often, mechanisms themselves cannot be directly measured, forcing (either high-inference assessment or) reliance on the observation of change in a proximal outcome of interest. For instance, didactic education, as an active ingredient of training, acts primarily through the mechanism of learning on the proximal outcome of knowledge to influence the distal implementation outcome of perceived acceptability or even adoption. Practice with feedback acts through the mechanism of reflecting on proximal outcomes of skills and confidence to influence the distal implementation outcome of adoption or even fidelity. To identify proximal outcomes, one must answer the question, “How will I know if this implementation strategy had an effect via the mechanism that I think it is activating?” or “What will be different if the hypothesized mechanisms for this strategy is at play?” It is very common for mechanisms and proximal outcomes to be conflated in the literature given that researchers often test mediation models examining the impact of a strategy on a distal implementation outcome via a more proximal outcome. The way we are using the terms, a mechanism is a process through which an implementation strategy operates, and a proximal outcome is a measurable effect of that process that is in the causal pathway toward the distal implementation outcomes.
Step 4: Articulating Effect Modifiers
concepts↗concepts↗concepts↗unhoused_relationsunhoused_relationssensessensesFinally, there are two types of effect modifiers that are important to articulate, both of which can occur across multiple levels of analysis: moderators and preconditions. Moderators are factors that increase or decrease the level of influence of an implementation strategy on an outcome. See Figure 1 in which an example for intra-individual and organizational-level moderators for audit and feedback are articulated. Theoretically, moderators are factors that interact with a strategy’s mechanism of action, even if exactly how they interact mechanistically are not understood. Preconditions are factors that are necessary for an implementation mechanism to be activated at all (see Figure 1 ). They are necessary conditions that need to be in place for the causal process that leads from an implementation strategy to its proximal and distal outcomes to take place. Both moderators and preconditions are most often mischaracterized as “determinants” in the implementation science literature base, which may limit our ability to understand the nature of the relations between a strategy and the individual and contextual factors that modify its effects, and, in turn, where, when, and why strategies have an effect on outcomes of interest.
Future Directions: What the Field of Implementation Needs to Fully Establish Itself as a Science
limitationsunhoused_relationsIn order to fully establish itself as a science by offering testable explanations and enabling the generation of predictions, we offer four critical steps for the field of implementation: (1) specify implementation strategies; (2) generate implementation strategy-mechanism linkages; (3) identify proximal and distal outcomes; and (4) articulate effect modifiers. In addition to these steps, we suggest that future research should strive for the generation of precise terms for factors implicated in implementation processes and use them consistently across studies. In a systematic review of implementation mechanisms, researchers conflated preconditions, predictors, moderators, mediators, and proximal outcomes (see text footnote 1). In addition, there is room for the field to develop guidelines for selecting research designs and study plans that account for practical constraints of the contexts in which implementation is studied and allow for mechanism evaluation. The types of causal pathway models that we advocated for here, paired with an understanding of the constraints of a particular study site, would enable researchers to select appropriate methods and designs to evaluate hypothesized relations by carefully considering the temporal dynamics such as how often a mechanism should be measured and how much the outcome is expected to change and when.
limitationsconcepts↗concepts↗concepts↗concepts↗concepts↗unhoused_relationsclaim_relationsIn order to truly advance the field, much work needs to be done to identify or develop psychometrically strong and pragmatic measures of implementation mechanisms. Empirically evaluating causal pathway models requires psychometrically strong measures of mechanisms that are also pragmatic, yet none of the seven published reviews of implementation-relevant measures focus on mechanisms. It is likely that measure development will be necessary to advance the field. Finally, implementation science could benefit from the building of more robust curation of evidence for knowledge transfer and use. Other fields house web-based databases for collecting, organizing, and synthesizing empirical findings [e.g., Science of Behavior Change ( 34 )]. In doing so, fields can accumulate knowledge more rapidly and users of knowledge can determine what is working, when, and why, as well as what generalizes and what does not. Such curation of evidence can more efficiently lead to the development of improved implementation strategies (e.g., through strategy specification), identification of mutable targets for new strategies (e.g., mechanisms revealed for existing strategies that may not be pragmatic), and prioritization of strategy use for a given context (e.g., given knowledge of preconditions and moderators).
Author Contributions
CL and PK are co-first authors, who co-led manuscript development. CL and BW are co-PIs on an R01 proposal that led to the inception of this manuscript. All authors (CL, PK, BP, AL, LT, SJ, CW-B, and BW) contributed to idea development, writing, and editing of this manuscript and agreed with its content.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer TW declared a past co-authorship with one of the author BP to the handling Editor.
Acknowledgments
BP would like to acknowledge funding from the National Institute of Mental Health (K01MH113806).
Footnotes
References
Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; 2 of 2 relations here carry that mark. The rest are the tagger’s reading of the sentence quoted beside them.
This record holds no controls. A layer is empty when the tagger recorded nothing in it — which is not the same as the paper having nothing to record.
| facet | tag | field | conf. | whose words |
|---|---|---|---|---|
| object_of_study | how implementation strategies operate through mechanisms, modifiers, and proximal and distal outcomes | object_of_study | the paper's words | |
| research_question | How can implementation science specify testable causal pathways that explain where, when, why, and through which mechanisms strategies affect outcomes? | research_question | the paper's words | |
| contribution | a four-step approach for specifying strategies, linking them to mechanisms, identifying proximal and distal outcomes, and articulating moderators and preconditions | contribution | the paper's words | |
| claim_type | method | claim_type | the paper's words | |
| phenomenon | implementation strategy effects depend on causal mechanisms, contextual modifiers, and measurable outcome pathways that the field has rarely tested | phenomenon | the paper's words | |
| method | conceptual-analysis also in: conceptual-analysis | method | the paper's words | |
| mechanisms | mechanisms also in: causal mechanisms | mechanisms[0] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: mechanisms of action | mechanisms[1] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: learning | mechanisms[2] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: sharing | mechanisms[3] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: leading | mechanisms[4] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: restructuring | mechanisms[5] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: guiding | mechanisms[6] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: self-reflection/reflecting | mechanisms[7] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: reflecting | mechanisms[8] | 0.92 | the tagger's words |
| mechanisms | mechanisms also in: engaging | mechanisms[9] | 0.92 | the tagger's words |
| limitations | Implementation science has offered little explanation of how different strategies work. | limitations[0] | 0.94 | the tagger's words |
| limitations | Published strategies may omit a sufficiently specified core component. | limitations[1] | 0.94 | the tagger's words |
| limitations | About two-thirds of cited implementation efforts fail to achieve intended change, and nearly half show no effect on outcomes of interest. | limitations[2] | 0.94 | the tagger's words |
| limitations | Implementation strategies are often mismatched to the level of the barrier they target. | limitations[3] | 0.94 | the tagger's words |
| limitations | The field has a dearth of tested theory and limited knowledge of specific causal mechanisms. | limitations[4] | 0.94 | the tagger's words |
| limitations | Broad approaches such as learning collaboratives may be too underspecified to link coherently to mechanisms. | limitations[5] | 0.94 | the tagger's words |
| limitations | Empirical evaluations of implementation mechanisms are rare, and the cited reviews established none empirically. | limitations[6] | 0.94 | the tagger's words |
| limitations | Mechanisms usually cannot be measured directly, requiring inference or reliance on proximal outcomes. | limitations[7] | 0.94 | the tagger's words |
| limitations | The literature often conflates mechanisms and proximal outcomes. | limitations[8] | 0.94 | the tagger's words |
| limitations | Implementation research has conflated preconditions, predictors, moderators, mediators, and proximal outcomes. | limitations[9] | 0.94 | the tagger's words |
| limitations | None of seven cited reviews of implementation-relevant measures focuses on mechanisms. | limitations[10] | 0.94 | the tagger's words |
| concepts | science of implementation also in: science of implementation | concepts[0] | 0.93 | the paper's words |
| concepts | implementation strategies also in: implementation strategies | concepts[1] | 0.93 | the paper's words |
| concepts | outcomes of implementation efforts also in: outcomes of implementation efforts | concepts[2] | 0.93 | the paper's words |
| concepts | testable theories also in: testable theories | concepts[3] | 0.93 | the paper's words |
| concepts | causal pathways also in: causal pathways | concepts[4] | 0.93 | the paper's words |
| concepts | causal pathway models also in: causal pathway models | concepts[5] | 0.93 | the paper's words |
| concepts | core component also in: core component | concepts[6] | 0.93 | the paper's words |
| concepts | mechanisms also in: mechanisms | concepts[7] | 0.93 | the paper's words |
| concepts | processes or events also in: processes or events | concepts[8] | 0.93 | the paper's words |
| concepts | implementation outcomes also in: implementation outcomes | concepts[9] | 0.93 | the paper's words |
| concepts | proximal outcomes also in: proximal outcomes | concepts[10] | 0.93 | the paper's words |
| concepts | distal outcomes also in: distal outcomes | concepts[11] | 0.93 | the paper's words |
| concepts | effect modifiers also in: effect modifiers | concepts[12] | 0.93 | the paper's words |
| concepts | preconditions also in: preconditions | concepts[13] | 0.93 | the paper's words |
| concepts | moderators also in: moderators | concepts[14] | 0.93 | the paper's words |
| concepts | research design also in: research design | concepts[15] | 0.93 | the paper's words |
| concepts | study plans also in: study plans | concepts[16] | 0.93 | the paper's words |
| concepts | practical constructs also in: practical constructs | concepts[17] | 0.93 | the paper's words |
| concepts | psychometrically strong and pragmatic measures | concepts[18] | 0.93 | the paper's words |
| concepts | knowledge transfer and use also in: knowledge transfer and use | concepts[19] | 0.93 | the paper's words |
| concepts | evidence-based practices also in: evidence-based practices | concepts[20] | 0.93 | the paper's words |
| concepts | health care also in: health care | concepts[21] | 0.93 | the paper's words |
| concepts | public health also in: public health | concepts[22] | 0.93 | the paper's words |
| concepts | implementation efforts also in: implementation efforts | concepts[23] | 0.93 | the paper's words |
| concepts | barriers also in: barriers | concepts[24] | 0.93 | the paper's words |
| concepts | training also in: training | concepts[25] | 0.93 | the paper's words |
| concepts | knowledge also in: knowledge | concepts[26] | 0.93 | the paper's words |
| concepts | organizational-level barrier also in: organizational-level barrier | concepts[27] | 0.93 | the paper's words |
| concepts | poor culture also in: poor culture | concepts[28] | 0.93 | the paper's words |
| concepts | suboptimal outcomes also in: suboptimal outcomes | concepts[29] | 0.93 | the paper's words |
| concepts | dearth of tested theory also in: dearth of tested theory | concepts[30] | 0.93 | the paper's words |
| concepts | causal mechanisms also in: causal mechanisms | concepts[31] | 0.93 | the paper's words |
| concepts | care delivery also in: care delivery | concepts[32] | 0.93 | the paper's words |
| concepts | implementation practice also in: implementation practice | concepts[33] | 0.93 | the paper's words |
| concepts | science also in: science | concepts[34] | 0.93 | the paper's words |
| concepts | evidence also in: evidence | concepts[35] | 0.93 | the paper's words |
| concepts | testable explanations also in: testable explanations | concepts[36] | 0.93 | the paper's words |
| concepts | predictions also in: predictions | concepts[37] | 0.93 | the paper's words |
| concepts | frameworks also in: frameworks | concepts[38] | 0.93 | the paper's words |
| concepts | conceptual structure also in: conceptual structure | concepts[39] | 0.93 | the paper's words |
| concepts | theories also in: theories | concepts[40] | 0.93 | the paper's words |
| concepts | variables also in: variables | concepts[41] | 0.93 | the paper's words |
| concepts | prediction of outcomes also in: prediction of outcomes | concepts[42] | 0.93 | the paper's words |
| concepts | causal relations also in: causal relations | concepts[43] | 0.93 | the paper's words |
| concepts | implementation theory also in: implementation theory | concepts[44] | 0.93 | the paper's words |
| concepts | contextual factors also in: contextual factors | concepts[45] | 0.93 | the paper's words |
| concepts | causal processes also in: causal processes | concepts[46] | 0.93 | the paper's words |
| concepts | variance in outcomes also in: variance in outcomes | concepts[47] | 0.93 | the paper's words |
| concepts | improved implementation strategies also in: improved implementation strategies | concepts[48] | 0.93 | the paper's words |
| concepts | mutable targets also in: mutable targets | concepts[49] | 0.93 | the paper's words |
| concepts | existing strategies also in: existing strategies | concepts[50] | 0.93 | the paper's words |
| concepts | contexts also in: contexts | concepts[51] | 0.93 | the paper's words |
| concepts | determinants also in: determinants | concepts[52] | 0.93 | the paper's words |
| concepts | provider behavior also in: provider behavior | concepts[53] | 0.93 | the paper's words |
| concepts | health-care delivery processes also in: health-care delivery processes | concepts[54] | 0.93 | the paper's words |
| concepts | mediators also in: mediators | concepts[55] | 0.93 | the paper's words |
| concepts | proximal versus distal also in: proximal versus distal | concepts[56] | 0.93 | the paper's words |
| concepts | Agile Science also in: Agile Science | concepts[57] | 0.93 | the paper's words |
| concepts | behavioral interventions also in: behavioral interventions | concepts[58] | 0.93 | the paper's words |
| concepts | intervention modularity also in: intervention modularity | concepts[59] | 0.93 | the paper's words |
| concepts | causal modeling also in: causal modeling | concepts[60] | 0.93 | the paper's words |
| concepts | efficient evaluations also in: efficient evaluations | concepts[61] | 0.93 | the paper's words |
| concepts | empirical evidence also in: empirical evidence | concepts[62] | 0.93 | the paper's words |
| concepts | boundary conditions also in: boundary conditions | concepts[63] | 0.93 | the paper's words |
| concepts | measurement-based care also in: measurement-based care | concepts[64] | 0.93 | the paper's words |
| concepts | patient-reported outcome data also in: patient-reported outcome data | concepts[65] | 0.93 | the paper's words |
| concepts | Patient Health Questionnaire 9-item depression symptom severity measure | concepts[66] | 0.93 | the paper's words |
| concepts | financial penalty also in: financial penalty | concepts[67] | 0.93 | the paper's words |
| concepts | disincentives also in: disincentives | concepts[68] | 0.93 | the paper's words |
| concepts | audit and feedback also in: audit and feedback | concepts[69] | 0.93 | the paper's words |
| concepts | Expert Recommendations for Implementing Change study | concepts[70] | 0.93 | the paper's words |
| concepts | Delphi process also in: Delphi process | concepts[71] | 0.93 | the paper's words |
| concepts | concept mapping exercise also in: concept mapping exercise | concepts[72] | 0.93 | the paper's words |
| concepts | learning collaboratives also in: learning collaboratives | concepts[73] | 0.93 | the paper's words |
| concepts | general approach also in: general approach | concepts[74] | 0.93 | the paper's words |
| concepts | core components also in: core components | concepts[75] | 0.93 | the paper's words |
| concepts | mechanisms of action also in: mechanisms of action | concepts[76] | 0.93 | the paper's words |
| concepts | active ingredient also in: active ingredient | concepts[77] | 0.93 | the paper's words |
| concepts | adoption also in: adoption | concepts[78] | 0.93 | the paper's words |
| concepts | sustainment also in: sustainment | concepts[79] | 0.93 | the paper's words |
| concepts | scale-up also in: scale-up | concepts[80] | 0.93 | the paper's words |
| concepts | discrete strategies also in: discrete strategies | concepts[81] | 0.93 | the paper's words |
| concepts | didactics also in: didactics | concepts[82] | 0.93 | the paper's words |
| concepts | modeling also in: modeling | concepts[83] | 0.93 | the paper's words |
| concepts | role play/rehearsal also in: role play/rehearsal | concepts[84] | 0.93 | the paper's words |
| concepts | feedback also in: feedback | concepts[85] | 0.93 | the paper's words |
| concepts | shadowing also in: shadowing | concepts[86] | 0.93 | the paper's words |
| concepts | strategy-mechanism linkages also in: strategy-mechanism linkages | concepts[87] | 0.93 | the paper's words |
| concepts | intrapersonal also in: intrapersonal | concepts[88] | 0.93 | the paper's words |
| concepts | interpersonal also in: interpersonal | concepts[89] | 0.93 | the paper's words |
| concepts | organizational also in: organizational | concepts[90] | 0.93 | the paper's words |
| concepts | community also in: community | concepts[91] | 0.93 | the paper's words |
| concepts | macro policy also in: macro policy | concepts[92] | 0.93 | the paper's words |
| concepts | provider habit also in: provider habit | concepts[93] | 0.93 | the paper's words |
| concepts | clinical decision support also in: clinical decision support | concepts[94] | 0.93 | the paper's words |
| concepts | self-reflection/reflecting also in: self-reflection/reflecting | concepts[95] | 0.93 | the paper's words |
| concepts | primary also in: primary | concepts[96] | 0.93 | the paper's words |
| concepts | ancillary also in: ancillary | concepts[97] | 0.93 | the paper's words |
| concepts | empirical evaluations also in: empirical evaluations | concepts[98] | 0.93 | the paper's words |
| concepts | systematic reviews also in: systematic reviews | concepts[99] | 0.93 | the paper's words |
| concepts | acceptability also in: acceptability | concepts[100] | 0.93 | the paper's words |
| concepts | feasibility also in: feasibility | concepts[101] | 0.93 | the paper's words |
| concepts | appropriateness also in: appropriateness | concepts[102] | 0.93 | the paper's words |
| concepts | penetration also in: penetration | concepts[103] | 0.93 | the paper's words |
| concepts | fidelity also in: fidelity | concepts[104] | 0.93 | the paper's words |
| concepts | cost also in: cost | concepts[105] | 0.93 | the paper's words |
| concepts | sustainability also in: sustainability | concepts[106] | 0.93 | the paper's words |
| concepts | latent variables also in: latent variables | concepts[107] | 0.93 | the paper's words |
| concepts | manifest/observable also in: manifest/observable | concepts[108] | 0.93 | the paper's words |
| concepts | implementation process also in: implementation process | concepts[109] | 0.93 | the paper's words |
| concepts | workshop training also in: workshop training | concepts[110] | 0.93 | the paper's words |
| concepts | products of the implementation strategy | concepts[111] | 0.93 | the paper's words |
| concepts | didactic education also in: didactic education | concepts[112] | 0.93 | the paper's words |
| concepts | learning also in: learning | concepts[113] | 0.93 | the paper's words |
| concepts | Practice with feedback also in: Practice with feedback | concepts[114] | 0.93 | the paper's words |
| concepts | reflecting also in: reflecting | concepts[115] | 0.93 | the paper's words |
| concepts | skills also in: skills | concepts[116] | 0.93 | the paper's words |
| concepts | confidence also in: confidence | concepts[117] | 0.93 | the paper's words |
| concepts | measurable effect also in: measurable effect | concepts[118] | 0.93 | the paper's words |
| concepts | intra-individual also in: intra-individual | concepts[119] | 0.93 | the paper's words |
| concepts | organizational-level moderators also in: organizational-level moderators | concepts[120] | 0.93 | the paper's words |
| concepts | necessary conditions also in: necessary conditions | concepts[121] | 0.93 | the paper's words |
| concepts | measure development also in: measure development | concepts[122] | 0.93 | the paper's words |
| concepts | implementation-relevant measures also in: implementation-relevant measures | concepts[123] | 0.93 | the paper's words |
| concepts | curation of evidence also in: curation of evidence | concepts[124] | 0.93 | the paper's words |
| concepts | web-based databases also in: web-based databases | concepts[125] | 0.93 | the paper's words |
| concepts | Science of Behavior Change also in: Science of Behavior Change | concepts[126] | 0.93 | the paper's words |
| unhoused_relations | causal pathway models → implementation theory Causal pathway models provide building blocks for implementation theory. — the relation stated: building-block relation | unhoused_relations[0] | 0.93 | the tagger's words |
| unhoused_relations | implementation strategies → care delivery Strategies are theorized to influence care delivery through mechanisms. — the relation stated: hypothesized causal influence | unhoused_relations[1] | 0.92 | the tagger's words |
| unhoused_relations | contextual factors → causal processes Contextual factors moderate causal processes. — the relation stated: moderates | unhoused_relations[2] | 0.94 | the tagger's words |
| unhoused_relations | causal pathway models → improved implementation strategies Causal-model findings can inform improved strategy development. — the relation stated: informs development | unhoused_relations[3] | 0.92 | the tagger's words |
| unhoused_relations | causal pathway models → mutable targets Causal-model findings can identify mutable targets for new strategies. — the relation stated: identifies | unhoused_relations[4] | 0.92 | the tagger's words |
| unhoused_relations | implementation strategies → determinants Chosen strategies should be compatible with local determinants. — the relation stated: compatibility requirement | unhoused_relations[5] | 0.93 | the tagger's words |
| unhoused_relations | determinants → implementation strategies Determinants may prevent or enable a strategy's effect. — the relation stated: prevents or enables effects | unhoused_relations[6] | 0.93 | the tagger's words |
| unhoused_relations | mechanisms → implementation strategies The same mechanism may participate in multiple strategies. — the relation stated: many-to-many involvement | unhoused_relations[7] | 0.92 | the tagger's words |
| unhoused_relations | primary → ancillary A strategy may have a primary mechanism and ancillary mechanisms. — the relation stated: mechanism ranking | unhoused_relations[8] | 0.91 | the tagger's words |
| unhoused_relations | workshop training → adoption Workshop training can influence adoption. — the relation stated: influences | unhoused_relations[9] | 0.92 | the tagger's words |
| unhoused_relations | workshop training → fidelity Workshop training does not influence fidelity. — the relation stated: does not influence | unhoused_relations[10] | 0.94 | the tagger's words |
| unhoused_relations | proximal outcomes → mechanisms Change in a proximal outcome can confirm or disconfirm activation of a putative mechanism. — the relation stated: diagnostic indicator | unhoused_relations[11] | 0.93 | the tagger's words |
| unhoused_relations | proximal outcomes → mechanisms Observable proximal outcomes are used when mechanisms cannot be measured directly. — the relation stated: measurement proxy | unhoused_relations[12] | 0.94 | the tagger's words |
| unhoused_relations | moderators → implementation strategies Moderators alter the level of a strategy's influence on an outcome. — the relation stated: increases or decreases influence | unhoused_relations[13] | 0.94 | the tagger's words |
| unhoused_relations | preconditions → mechanisms Preconditions must be present for mechanism activation. — the relation stated: necessary prerequisite | unhoused_relations[14] | 0.95 | the tagger's words |
| unhoused_relations | causal pathway models → research design Causal models paired with site constraints can enable appropriate method and design selection. — the relation stated: enables selection | unhoused_relations[15] | 0.92 | the tagger's words |
| unhoused_relations | curation of evidence → knowledge transfer and use Robust evidence curation supports knowledge transfer and use. — the relation stated: supports | unhoused_relations[16] | 0.91 | the tagger's words |
| unhoused_relations | learning collaboratives → general approach Learning collaboratives are presented as a general approach. — the relation stated: appositional classification | unhoused_relations[17] | 0.94 | the tagger's words |
| unhoused_relations | learning collaboratives → mechanisms of action Underspecified learning collaboratives make precise mechanisms difficult to identify. — the relation stated: underspecification obscures | unhoused_relations[18] | 0.93 | the tagger's words |
| senses | implementation strategies - methods used to promote adoption, implementation, sustainment, or scale-up of evidence-based practice | senses[0] | 0.9 | the tagger's words |
| senses | mechanisms - processes or events through which an implementation strategy operates to effect desired implementation outcomes | senses[1] | 0.9 | the tagger's words |
| senses | causal pathway models - contextualized interrelations among variables and outcomes that form building blocks of implementation theory | senses[2] | 0.9 | the tagger's words |
| senses | proximal outcomes - direct measurable products indicating whether a strategy activated its putative mechanism | senses[3] | 0.9 | the tagger's words |
| senses | distal outcomes - desired implementation endpoints reached later in the causal process | senses[4] | 0.9 | the tagger's words |
| senses | moderators - factors that increase or decrease a strategy's influence on an outcome | senses[5] | 0.9 | the tagger's words |
| senses | preconditions - necessary factors that must be present before an implementation mechanism can activate | senses[6] | 0.9 | the tagger's words |
| senses | determinants - naturally occurring and often malleable factors that prevent or enable strategy effects | senses[7] | 0.9 | the tagger's words |
| senses | theories - testable explanations that specify relations among variables and enable outcome prediction | senses[8] | 0.9 | the tagger's words |
| claim_relations | supports → a four-step causal-pathway modeling approach for implementation strategies note: The contribution is explicitly described as an approach rather than an empirical test. | claim_relations[0] | 0.93 | the tagger's words |
| claim_relations | builds-on → Agile Science principles of modularity, causal modeling, efficient evaluation, and boundary conditions note: The paper states that its perspective draws on Agile Science. | claim_relations[1] | 0.93 | the tagger's words |
| claim_relations | contrasts-with → framework-only organization of implementation knowledge note: The paper distinguishes predictive theory from basic conceptual frameworks. | claim_relations[2] | 0.92 | the tagger's words |
| claim_relations | qualifies → the empirical status of implementation mechanisms note: The paper reports rare evaluation and no empirically established mechanisms in the cited reviews. | claim_relations[3] | 0.94 | the tagger's words |
| claim_relations | predicts → better strategy development, mutable-target identification, impact, and contextual prioritization note: These are stated prospective benefits of findings from causal-model studies, not results demonstrated here. | claim_relations[4] | 0.9 | the tagger's words |
| claim_relations | supports → a field agenda for precise terms, mechanism-sensitive study design, pragmatic measures, and evidence curation note: The agenda is explicitly recommended, with some benefits expressed modally. | claim_relations[5] | 0.92 | the tagger's words |
| concept_edges | proximal outcomes —is-a→ products of the implementation strategy the relation itself is separately evidenced · strength of the relation: 1 · note: Explicit categorical definition. · the relation stated: Proximal outcomes are defined as direct measurable products of the implementation strategy. | concept_edges[3] | 0.95 | the tagger's words |
| concept_edges | dearth of tested theory —causes→ suboptimal outcomes the relation itself is separately evidenced · strength of the relation: 0.9 · note: Positive sign toward the adverse target; the paper says due in large part, so weight is below certainty. · the relation stated: The lack of tested theory is identified as a major source of suboptimal implementation outcomes. also in: dearth of tested theory · suboptimal outcomes | concept_edges[9] | 0.93 | the tagger's words |
Tags that could not be placed in the text:
What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.