Implementability of healthcare interventions: an overview of reviews and development of a conceptual framework. — TEI-tagged view

Marlena KlaicpersName, Suzanne KapppersName, Peter HudsonpersName, Wendy ChapmanpersName, Linda DenehypersName, David StorypersName, Jill J. FrancispersName (2022date). “Implementability of healthcare interventions: an overview of reviews and development of a conceptual framework..” Implementation Sciencetitle·j.

Object of study — the implementabilityterm·dom of healthcare interventionsterm·dom through acceptabilityterm·dom, fidelityterm·dom, feasibilityterm·key, scalabilityterm·dom and sustainabilityterm·key

Research question — whether the five concepts have been defined, operationalised or theorised in reviewsterm·mth of healthcare interventionsterm·dom

Contribution — a testable conceptual framework in which scalabilityterm·dom and sustainabilityterm·key depend on acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key

Full text — tagged

Implementabilityterm·dom of healthcare interventionsterm·dom: an overview of reviewsterm·mth and development of a conceptual framework.

Abstract Implementationterm·dom research may play an important role in reducing research wasteterm·key by identifying strategies that support translation of evidence into practice. Implementationterm·dom of healthcare interventionsterm·dom is influenced by multiple factors including the organisational context, implementationterm·dom strategies and features of the intervention as perceived by people delivering and receiving the intervention. Recently, concepts relating to perceived features of interventions have been gaining traction in published literature, namely, acceptabilityterm·dom, fidelityterm·dom, feasibilityterm·key, scalabilityterm·dom and sustainabilityterm·key. These concepts may influence uptake of healthcare interventionsterm·dom, yet there seems to be little consensus about their nature and impact. The aim of this paper is to develop a testable conceptual framework of implementabilityterm·dom of healthcare interventionsterm·dom that includes these five concepts.

A multifaceted approach was used to develop and refine a conceptual framework of implementabilityterm·dom of healthcare interventionsterm·dom. An overview of reviewsterm·mth identified reviewsterm·mth published between January 2000 and March 2021 that focused on at least one of the five concepts in relation to a healthcare interventionterm·dom. These findings informed the development of a preliminary framework of implementabilityterm·dom of healthcare interventionsterm·dom which was presented to a panel of experts. A nominal group process was used to critique, refine and agree on a final framework.

A total of 252 publications were included in the overview of reviewsterm·mth. Of these, 32% were found to be feasible, 4% reported sustainable changes in practice and 9% were scaled up to other populations and/or settings. The expert panel proposed that scalabilityterm·dom and sustainabilityterm·key of a healthcare interventionterm·dom are dependent on its acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key. Furthermore, acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key require re-evaluation over time and as the intervention is developed and then implemented in different settings or with different populations. The final agreed framework of implementabilityterm·dom provides the basis for a chronological, iterative approach to planning for wide-scale, long-term implementationterm·dom of healthcare interventionsterm·dom.

We recommend that researchers consider the factors acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key (proposed to influence sustainabilityterm·key and scalabilityterm·dom) during the preliminary phases of intervention development, evaluation and implementationterm·dom, and iteratively check these factors in different settings and over time.

The online version contains supplementary material available at 10.1186/s13012-021-01171-7.

Keywords: Implementationterm·dom strategies, Framework, Scalabilityterm·dom, Sustainabilityterm·key, Implementabilityterm·dom, Implementation scienceterm·key, Implementationterm·dom research, Healthcare interventionsterm·dom Reviewsterm·mth report relatively few healthcare interventionsterm·dom that are sustained beyond the initial implementationterm·dom phase or scaled to different populations or settings.

Acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key may influence scalabilityterm·dom and sustainabilityterm·key of a healthcare interventionterm·dom.

We have developed a testable conceptual framework that can be used to prospectively and iteratively guide the implementabilityterm·dom of healthcare interventionsterm·dom.

Prospective identification of factors that influence scalabilityterm·dom and sustainabilityterm·key of a healthcare interventionterm·dom is critical to avoid or reduce research wasteterm·key.

Background Implementation scienceterm·key aims to identify and address care gaps, support practice change and enhance quality and equity of health care. Building a robust and generalizable evidence base to inform implementationterm·dom practice is the objective of implementationterm·dom research. Implementationterm·dom research can also play a critical role in efforts to reduce research wasteterm·key, in that it can provide evidence about the strategies that are effective for translating the findings of clinical research into enhanced healthcare practice and thus improved health outcomes [ 1 – 3 ]. Identifying the factors important for translation of an effective intervention or innovation from the research setting to routine clinical practice can arguably contribute to reducing the estimated annual US$85 billion, globally, wasted in health research [ 1 , 2 , 4 ].

Most implementationterm·dom investigations focus on one of two approaches to achieve change. First, implementationterm·dom activities consist of either “top down” processes (e.g. governance arrangements, national policies and guidelines, continuing medical education, incentivisation systems) [ 5 – 7 ], or more granular “bottom-up” processes that consider views of healthcare workers: their perceived barriers and enablers to specific elements of practice change at the level of healthcare teams and individual clinicians [ 8 – 11 ]. The second approach considers features of healthcare contexts (including organisational factors and the wider health system context) that might interact with the implementationterm·dom activities to enable or impede practice change [ 12 – 16 ].

The current paper considers a third lever for achieving implementationterm·dom: the perceived features of healthcare interventionsterm·dom themselves (in addition to effectiveness). An early theory, Diffusion of Innovations [ 17 ], identified six features of innovations that make their adoption more or less likely, namely, relative advantage, compatibility with the existing system, complexity, trialabilityterm·dom, potential for reinvention and observed effects, where trialabilityterm·dom refers to being able to test the innovation or intervention on a small scale, such as a pilot study. The more recent Consolidated Framework of Implementation Researchterm (CFIRterm) proposed seven attributes of interventions, namely, intervention source, evidence strength and quality, relative advantage, adaptability, trialabilityterm·dom, complexity and design quality and packaging, which refers to the presentation of the intervention, such as how it is bundled and user accessibility [ 12 ]. A recently published reviewterm·mth identified 28 implementationterm·dom frameworks and models, including the CFIRterm, which were synthesised into a number of core phases and components [ 18 ]. The authors suggest there is a need for an overarching framework that can guide researchers from intervention development to sustainable practice change.

Uptake of an intervention by both providers and recipients also depends crucially on their perceptions of the intervention. The COVID pandemic of 2020–2021 exemplifies this point. Even though approved vaccine interventions have substantial evidence of a positive benefit-to-risk ratio, the speed of uptake in many countries has been dependent on the perceptions of politicians, service providers and members of the public regarding the necessity, urgency and benefits of vaccine programs. It seems that, independent of the objective features of an intervention, stakeholder perceptions about the intervention will radically influence implementationterm·dom and uptake at many levels. Furthermore, these perceptions may change over time and during roll-out of an intervention. We refer to these perceptions as views about the “implementabilityterm·dom” of an intervention. We define “implementabilityterm·dom” as the likelihood that an intervention will be adopted into routine practice and into health consumer behaviours across settings and over time. Several concepts related to implementabilityterm·dom of healthcare interventionsterm·dom are gaining traction in the implementation scienceterm·key literature and appear to be primarily focused on the earlier stages of intervention development or latter stages of evaluation. These are acceptabilityterm·dom, fidelityterm·dom, feasibilityterm·key, scalabilityterm·dom and sustainabilityterm·key. A search of the health sciences literature (conducted 25th February 2021) for studies published in the last 20 years containing one or more of these five concepts in the title, revealed that the annual frequency of usage has steadily increased. Figure 1 shows that, from a relatively low baseline in the year 2000, these concepts appeared in titles in 2020, respectively, > 900, > 450, > 4500, > 350 and > 650 times.

Annual frequency of the five key concepts in publications indexed to PubMed Where there was an explicit rationale in the included studies, authors noted that the concept under investigation was likely to influence engagement, adoption and ongoing use. For example, a systematic reviewterm·mth exploring videoconferencing in an orthopaedic setting [ 19 ] found that “acceptabilityterm·dom of service users (both patients and clinicians) is a key factor for the uptake of telemedicine in clinical practice” (p.184). Hence, it is plausible that these concepts, individually and collectively, influence intervention implementabilityterm·dom. To identify whether other implementation-related concepts were also appearing in the literature, we conducted a further illustrative search of literature published in the last 20 years (conducted on 29th August 2021) relating to specific interventions (using the phrase “of [intervention]”). We selected three healthcare interventionsterm·dom for which there were published reviewsterm·mth including one or more of the five concepts. The majority (> 90%) of reviewsterm·mth focused on clinical effectiveness or evaluation of outcomes. No additional concepts related to implementabilityterm·dom, other than the five considered in our proposed framework, were evident.

There seems to be little consensus about the nature of these concepts, appropriate measurement strategies and how they might be related to one another. Without consistent definitions or reliable measurement approaches, it is not possible to test assumptions or predictions about whether these features indeed influence the implementabilityterm·dom of healthcare interventionsterm·dom.

Implementabilityterm·dom has been previously explored in the published literature, but this has focused on the implementabilityterm·dom of clinical practice guidelines [ 20 – 23 ] or, more recently, the implementabilityterm·dom of late-phase clinical trials [ 24 ]. The current paper focuses more broadly on prospective implementabilityterm·dom of healthcare interventionsterm·dom, particularly at the early stages of development and evaluation, during scale-up, and over time.

The aim of this paper is to report the development of a testable conceptual framework of implementabilityterm·dom that includes acceptabilityterm·dom, fidelityterm·dom, feasibilityterm·key, scalabilityterm·dom and sustainabilityterm·key.

Methods A multifaceted approach was used to develop and refine the framework of implementabilityterm·dom of healthcare interventionsterm·dom. We use the World Health Organization definition of healthcare interventionsterm·dom as “an act performed for, with or on behalf of a person or population whose purpose is to assess, improve, maintain, promote or modify health, functioning or health conditions.” [ 25 ] Step 1: Overview of reviewsterm·mth A preliminary exploratory search indicated a large volume of systematic reviewsterm·mth on the aforementioned five concepts within published literature on healthcare interventionsterm·dom. We therefore decided to conduct an overview of reviewsterm·mth [ 26 ], to answer the following questions: Have the five concepts (acceptabilityterm·dom, fidelityterm·dom, feasibilityterm·key, scalabilityterm·dom and sustainabilityterm·key) been defined, operationalised and/or theorised in systematic reviewsterm·mth (SR) on healthcare interventionsterm·dom?

Have the five concepts been combined in any publications, frameworks or models used in published literature on healthcare interventionsterm·dom?

Search strategy Systematic reviewsterm·mth published from January 2000 to March 2021 were identified and retrieved by one author (MK). Searches were structured by combining relevant reviewterm·mth filters (Additional file 1 : search strategy), with the appearance of the truncated term for each concept (“acceptab*”, “fidelityterm·dom”, “feasib*”, “sustainab*” and “scalab*”) in article titles. Searching for these terms in article titles ensured that the reviewterm·mth had a primary focus on the concept. The term feasibilityterm·key was combined with “or process evaluat*” but papers were considered for inclusion only if the term feasibilityterm·key was in the abstract. We did not include a synonym search as we were specifically interested in the use of the particular concept terms in the literature.

Multiple databases were searched using OVID (Medline and Embase) and EBSCO (PsycINFO) and restricted to publications in English.

Screening citations Duplicates were removed using the deduplicate option in the OVID and EBSCO search engines and the remaining citations were imported into EndNote X9 [ 27 ] where further duplicates were manually identified and removed. Reviewsterm·mth were considered eligible for inclusion if they met the criteria detailed in Table 1 . Screening was a two-step process, commencing with an initial reviewterm·mth of the abstracts by one author (MK) to determine eligibility. If there was insufficient information in the abstract to make a decision regarding inclusion, the full paper was retrieved, and the methods section was screened. For example, drug-development reviewsterm·mth with acceptabilityterm·dom or feasibilityterm·key in the title but a focus on cost-effectiveness acceptabilityterm·dom curves were excluded.

Inclusion and exclusion criteria for the overview of reviewsterm·mth Full-text reviewterm·mth The full articles for all citations that met the inclusion criteria were retrieved by one author (MK) with an additional author (SK) independently reviewing a random selection of 10% of the retrieved papers. Data were extracted using a form developed by the research group and included if and how the concept was defined, how it was operationalised or measured, use or development of a framework and overall outcome of the reviewterm·mth, i.e. had the healthcare interventionterm·dom achieved acceptabilityterm·dom, fidelityterm·dom, feasibilityterm·key, scalabilityterm·dom or sustainabilityterm·key? A reviewterm·mth was considered to have used a framework if it was mentioned in the methods and data were synthesised using the components of the stated framework. Primary studies included in the reviewsterm·mth were not individually reviewed as this was considered to be outside of the scope of this overview. Data extracted from the reviewsterm·mth were summarised descriptively.

Assessment of quality Assessment of quality was not conducted as the aim of the study was to explore how the implementabilityterm·dom concepts were defined and conceptualised, rather than the quality of the information related to the health intervention being delivered.

Step 2: Development of the preliminary framework A preliminary framework (Fig. 2 ) was developed in parallel with the overview of reviewsterm·mth, integrating the five concepts that appeared to be mostly investigated in isolation in the published reviewsterm·mth on healthcare interventionsterm·dom, and based on the research experiences and theory-development expertise of two authors (JF and MK). This framework was then presented to the group of experts (co-authors), as described below.

Initial framework of implementabilityterm·dom of healthcare interventionsterm·dom Step 3: Modified Nominal Group Techniqueterm The Nominal Group Techniqueterm (NGTterm) is a facilitator-led, structured process for obtaining information and arriving at a decision with a target group who have some association or experience with the topic [ 28 ]. Various adaptations of the NGTterm have been used in conceptual studies that focus on framework development [ 29 – 33 ]. Recently, an additional pre-meeting, information-giving step has been suggested to enable more time for participants to consider their contribution to the topic [ 34 , 35 ]. The adapted NGTterm process utilised in this study was as follows: (i) identification of group members, to include experts with depth and diversity of experience [ 36 ]. All authors on this paper were invited by e-mail to attend an online group meeting. They were purposively identified at the start of this study for their knowledge and expertise in the fields of implementation scienceterm·key, theory development, biomedical informatics and clinical research across a broad range of fields; (ii) provision of information prior to the meeting, including a PowerPoint presentation, findings of the overview of reviewsterm·mth and objectives of the meeting. Five authors with extensive clinical research backgrounds were asked to prepare a clinical scenario on one concept for sharing at the group meeting. The intention of this exercise was to discuss the fit between a real-world example of a study that explored one of the concepts and the proposed framework; (iii) meeting conducted online and facilitated by one author (JF) who has extensive experience in consensus panel processes. Following presentation of the meeting materials, including the preliminary framework, group members were instructed to silently consider the framework and generate ideas and critiques; iv) round-robin process with participants sharing their ideas and critiques; v) clarification process where participants shared their clinical scenario on a concept and discussed the fit with components of the initial framework, and vi) voting and/or agreement on the preliminary framework.

Results Step 1: Overview of reviewsterm·mth The database searches initially identified a total of 839 references across all five concepts (acceptabilityterm·dom = 224, fidelityterm·dom = 281, feasibilityterm·key = 253, scalabilityterm·dom = 37 and sustainabilityterm·key = 44). Following removal of 317 duplicates and screening of titles and abstracts, 301 full texts were sought for retrieval. Two were not retrieved as they were not available in English. Of the remaining 299 reports assessed for eligibility, 43 were excluded due to being unrelated to the concept (e.g. fidelityterm·dom of DNA) and four were excluded as they focused on psychometric testing of a measure rather than a health intervention. The final number of publications included in this reviewterm·mth was 252, of which 22 papers discussed more than one concept. As we were considering the concepts separately, these 22 were treated as separate investigations, resulting in a total of 274 investigations (Additional file 2 : reviewsterm·mth included in the overview, consisting of acceptabilityterm·dom = 132, fidelityterm·dom = 41, feasibilityterm·key = 65, scalabilityterm·dom = 11 and sustainabilityterm·key = 25), with the stages of the search process presented in Fig. 3 .

PRISMA [ 37 ] flow chart of included reviewsterm·mth for search completed in March 2021 Characteristics of the included studies Of the 252 studies in the overview, 30% included a meta-analysis and 19% used a mixed-methods approach that incorporated both quantitative and qualitative data from empirical research. The healthcare interventionsterm·dom that were a focus of the reviewterm·mth were broad ranging and included psychological/psychiatric/psychosocial interventions (20%), technology-based interventions (5%), physical activities (6%), pharmacological and alternative interventions. The number of studies included in reviewsterm·mth ranged from 4 to 296, with the majority of studies (63%) having no setting exclusions (or not reported). Acceptabilityterm·dom and fidelityterm·dom were assessed using highly variable measurement approaches, so it was not appropriate to summarise the findings across reviewsterm·mth. The intervention under investigation was reported to be feasible in 32 (49%) reviewsterm·mth, sustainable in 1 (4%) reviewterm·mth and successfully scaled up in 1 (9%) reviewterm·mth (Additional file 3 ).

Definition and measurement (question a) and frameworks (question b) for the five concepts A total of 1096 items of information were extracted from the 274 investigations by the first reviewer (MK). The second reviewer (SK), double extracted information from 10% of the reviewsterm·mth (total 32 papers double reviewed), which were compared with the first reviewer to assess reliability.

Of these, 100% agreement was achieved for definition of the concept and outcome of the reviewterm·mth (e.g. acceptabilityterm·dom or fidelityterm·dom or feasibilityterm·key or scalabilityterm·dom or sustainabilityterm·key was/was not achieved). There was 96% agreement on the use of a framework and 89% agreement on the constructs measured. One reviewer failed to identify a framework in one of the studies resulting in 96% (out of 32 studies double reviewed) agreement. For constructs measured, reviewers did not identify the same constructs for two papers, resulting in 89% agreement. The two reviewers discussed these differences and agreed on a decision.

Twenty-two publications included two concepts in their reviewterm·mth, of which 20 considered acceptabilityterm·dom and feasibilityterm·key [ 38 – 57 ] in exploring implementationterm·dom of a healthcare interventionterm·dom, and two considered scalabilityterm·dom and sustainabilityterm·key [ 58 , 59 ].

Acceptabilityterm·dom Of the reviewsterm·mth exploring acceptabilityterm·dom of a healthcare interventionterm·dom, only 13 provided an a priori definition and they largely focused on whether those receiving a healthcare interventionterm·dom found it to be “appropriate” and “fair” and “reasonable” [ 42 , 48 , 60 – 66 ]. Four reviewsterm·mth considered acceptabilityterm·dom from the perspective of those delivering a healthcare interventionterm·dom [ 61 , 62 , 65 , 66 ].

The majority of reviewsterm·mth measured one variable in evaluating acceptabilityterm·dom, and this was predominantly either dropout rates (33%) or user perceptions of the intervention (30%). Twenty-three reviewsterm·mth measured three or more component variables, most commonly a combination of participant dropouts, recruitment rates, perceptions of users such as satisfaction measures, adherence to the study protocol and adverse events (3%).

Six reviewsterm·mth used a framework to define and measure acceptabilityterm·dom, as described in Table 2 . The Theoretical Framework of Acceptabilityterm·dom [ 67 ] was used in four of these reviewsterm·mth and defined acceptabilityterm·dom as “a multi-faceted construct that reflects the extent to which people delivering or receiving a healthcare interventionterm·dom consider it to be appropriate, based on anticipated or experiential cognitive and emotional responses to the intervention”. (p.8) This framework consists of seven constructs related to acceptabilityterm·dom including affective attitude, burden, perceived effectiveness, ethicality, intervention coherence, opportunity costs and self-efficacy.

Definitions, frameworks and commonly measured components in systematic reviewsterm·mth on acceptabilityterm·dom of healthcare interventionsterm·dom ( n = 132) Fidelityterm·dom Of the 41 reviewsterm·mth exploring fidelityterm·dom of a healthcare interventionterm·dom, 35 included an a priori definition. Almost all reflected a dictionary definition of fidelityterm·dom [ 68 ] with terms such as “integrity” and “delivered as intended” and “accuracy and consistency” included in their descriptions.

Thirty-six reviewsterm·mth measured four or more components as part of their assessment of fidelityterm·dom of a healthcare interventionterm·dom including adherence to the protocol (76%), dose delivered and received (76%) and provider training (49%). Thirty-four publications on fidelityterm·dom of a healthcare interventionterm·dom explicitly used a framework to guide their reviewterm·mth, as described in Table 3 . The framework used in 19 of these cases was from the National Institute of Health Behavior Change Consortium (NIHBCC) [ 69 ] which includes measures of study design, training, delivery of treatment, receipt of treatment and enactment of treatment. The NIHBCC describe fidelityterm·dom as “the methodological strategies used to monitor and enhance the reliability and validity of behaviour interventions”. (p.443) Definitions, framework and commonly measured components in systematic reviewsterm·mth on fidelityterm·dom of healthcare interventionsterm·dom ( n = 41) Feasibilityterm·key Thirteen of 65 reviewsterm·mth defined feasibilityterm·key using terms such as “practicality” and “ease of delivery” and “possible to undertake”. Seven of these 13 papers noted the importance of context and broader system factors when considering the “possibility of what could be done”, such as physical space, ongoing funding and political support [ 39 , 70 – 75 ].

The two most frequently measured constructs within the concept of feasibilityterm·key were adherence to the study protocol (34%) (i.e. the same as the operationalisation of fidelityterm·dom) and perceptions of key stakeholders (33%), including those providing and receiving the healthcare interventionterm·dom (23%). There was a significant overlap between feasibilityterm·key and acceptabilityterm·dom with a number (22%) of feasibilityterm·key reviewsterm·mth incorporating acceptabilityterm·dom as a construct to be measured within feasibilityterm·key.

Five reviewsterm·mth referred to a feasibilityterm·key framework (Table 4 ) with the most commonly used being Bowen et al’s publication on designing feasibilityterm·key studies [ 76 ]. This highly cited publication does not provide a definition of feasibilityterm·key but does identify eight areas of focus that should be addressed in feasibilityterm·key studies including acceptabilityterm·dom, demand, implementationterm·dom, practicality, adaptation, integration, expansion and limited efficacy testing. Bowen et al. define implementationterm·dom as the extent to which the intervention can be delivered as planned which is synonymous with fidelityterm·dom.

Definitions, frameworks and commonly measured constructs in systematic reviewsterm·mth on feasibilityterm·key of healthcare interventionsterm·dom ( n = 65) Almost a third (32%) of the reviewsterm·mth found the healthcare interventionterm·dom to be feasible, with the remaining being a mixture of not feasible or unable to establish feasibilityterm·key due to lack of information in the empirical studies.

Scalabilityterm·dom Healthcare interventionsterm·dom may be scaled up to different populations and/or settings. Eleven reviewsterm·mth explored scalabilityterm·dom of a healthcare interventionterm·dom with nine presenting an a priori definition including terms such as “deliberate efforts” and “expanding or increasing the impact”. Six of these definitions also included the need for a healthcare interventionterm·dom to be proven effective prior to scaling up.

Five reviewsterm·mth measured four or more constructs with organisation, community and sociocultural factors being the most frequently reported measures (45%) followed by resources, economic viability (18%) and adaptation of the intervention (18%). Only one of the reviewsterm·mth definitively found that the healthcare interventionterm·dom had been successfully expanded across different settings or populations [ 77 ]. The majority were unable to reach a conclusion due to lack of data in the included studies.

Four different scalabilityterm·dom frameworks (Table 5 ) were described and used in four of the 11 reviewsterm·mth [ 59 , 78 – 80 ]. These were the World Health Organisation ExpandNet Scaling-Up Framework [ 81 ], the Intervention Scalabilityterm·dom Assessment Tool (ISAT) [ 82 ], the Assess, Innovate, Develop, Engage, Devolve (AIDED) model [ 83 ] and the Non-adoption, Abandonment, Scale-up, Spread, Sustainabilityterm·key (NASSS) framework [ 84 ]. Commonalities across the four frameworks include the intervention, the strategic/political context to support scale-up and resources to support and sustain the scale-up process. Scalabilityterm·dom was defined in a similar way within these frameworks as the capacity or ease with which an intervention or innovation that had been proven effective could be expanded to other settings or populations [ 81 , 82 ].

Definition, frameworks and commonly measured constructs in systematic reviewsterm·mth on scalabilityterm·dom of healthcare interventionsterm·dom ( n = 11) Sustainabilityterm·key Of the 25 reviewsterm·mth on this concept, 15 included a definition with common use of terms such as “continuation” and “extended period of time”. Seven of these definitions also included the notion that sustainabilityterm·key is about the maintenance of the intervention or program after initial funding or implementationterm·dom efforts have ceased [ 58 , 85 – 90 ].

Constructs typically measured in reviewsterm·mth of sustainabilityterm·key of healthcare interventionsterm·dom included organisation- or community-specific factors (36%), continuation of the intervention beyond a specified period of time (24%), established collaborations or partnerships (12%) and resources (8%). Only one of the reviewsterm·mth [ 91 ] found that sustainabilityterm·key had been achieved for the healthcare interventionterm·dom under investigation. A number of other reviewsterm·mth [ 92 – 94 ] were unable to draw a definitive conclusion due to inconsistent definitions and measures of sustainabilityterm·key within the empirical literature in the reviewsterm·mth.

Seven different sustainabilityterm·key frameworks (Table 6 ) were referred to in nine of the systematic reviewsterm·mth. The two most frequently used frameworks included ongoing maintenance of benefits from the intervention, capacity building and integration of the intervention or program within the organisation [ 95 , 96 ]. Moore et al. developed a comprehensive definition of sustainabilityterm·key based on five constructs including continuation of a program or intervention of implementationterm·dom strategies or individual behaviour change, after a defined period of time, with or without adaptations but continuing to produce benefits for the individual and/or systems [ 95 ].

Definitions, frameworks and commonly measured constructs in systematic reviewsterm·mth on sustainabilityterm·key of healthcare interventionsterm·dom ( n = 25) In summary, two key findings emerged from the overview of reviewsterm·mth. First, the current literature suggests that the concepts are related, although there was some variation in the terms used. For example, reviewsterm·mth on feasibilityterm·key of a healthcare interventionterm·dom measured ‘implementationterm·dom’ of the intervention, defined as the extent to which the intervention can be delivered as planned, also known as fidelityterm·dom. Similarly, reviewsterm·mth on sustainabilityterm·key, also known as maintenance, measured resources, funding and organisational factors, which were all measures frequently included within reviewsterm·mth on feasibilityterm·key of a healthcare interventionterm·dom. The second key finding is that although acceptabilityterm·dom appeared to be an important factor contributing to fidelityterm·dom, additional factors were required, for example provider training. Similarly, although fidelityterm·dom was an important factor contributing to feasibilityterm·key, additional factors were required, for example funding and other resources such as physical space.

Step 2: Modified Nominal Group Techniqueterm The first group meeting took place online using ZOOM [ 97 ] in March 2021. Three themes were identified from the group discussion, as follows: Theme 1: It is plausible that the concepts influence implementabilityterm·dom of a healthcare interventionterm·dom.

Following presentation of the findings of the overview of reviewsterm·mth on each concept, participants agreed the theoretical plausibility of a framework of implementabilityterm·dom for healthcare interventionsterm·dom that includes all five concepts. This was further consolidated through sharing their own real-world research and clinical experiences as described in Tables 7 , 8 , 9 , 10 and 11 .

Scenario illustrating influence of acceptabilityterm·dom on implementabilityterm·dom Scenario illustrating influence of fidelityterm·dom on implementabilityterm·dom Scenario illustrating influence of feasibilityterm·key on implementabilityterm·dom Scenario illustrating influence of fidelityterm·dom and feasibilityterm·key on sustainabilityterm·key of a healthcare interventionterm·dom Hypothetical scenario illustrating influence of acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key on scalabilityterm·dom of a health intervention Theme 2: The concepts appear to be related to one another.

Participants were asked to consider the question “in your view, is it plausible that the concepts are related to each other?” All agreed that it was possible that the concepts were interdependent and should be considered together when developing and implementing a healthcare interventionterm·dom. They suggested that individual concepts were necessary but insufficient on their own to ensure implementabilityterm·dom of a healthcare interventionterm·dom. Participants also identified that in many of the clinical scenarios they shared, often more than one concept was involved in the final outcome. For example, it was identified that the hypothetical scalabilityterm·dom scenario included issues with feasibilityterm·key of a device, acceptabilityterm·dom of an online system and lack of procedural information relevant for fidelityterm·dom.

Theme 3: The preliminary implementabilityterm·dom framework could be amended to better represent the relationships between the concepts.

Although all participants agreed that the concepts were related, they were not in agreement that the preliminary framework adequately represented the nature of their interdependence. Some participants felt more detail was needed to explain how the concepts related whilst other participants felt that some concepts required greater representation than others. All participants were asked to further consider the framework over the following week and provide feedback and revisions.

Step 3: Revising the framework Following the first group discussion and feedback from the participants, the preliminary framework was revised, resulting in three options presented to consensus panel participants as three figures. These three draft frameworks included the revisions requested by the participants such as annotations on the first framework for option one, different graphics representing different relationships between the concepts on the second option and a combination of options one and two for the third framework option. Participants were asked to consider the three options and silently vote on their preferred framework and provide any further feedback via return email. The feedback was considered and integrated by two authors (MK and JF) resulting in a further version of the framework (Fig. 4 ). All participants agreed this final version.

Conceptual framework of implementabilityterm·dom of healthcare interventionsterm·dom The framework as depicted in Fig. 4 is designed to guide research activities, chronologically and iteratively, from left to right and from bottom to top. Commencing in the research context, it is proposed that acceptabilityterm·dom is the first concept to assess, during intervention development and work-up of supporting documentation and resources (intervention protocol, training manual, patient information leaflet, data and technology requirements, validation of digital components, etc.). If acceptabilityterm·dom is adequate to providers and potential recipients, it is appropriate to deliver the intervention to assess fidelityterm·dom as delivered and as received. Without adequate acceptabilityterm·dom, providers and recipients are unlikely to engage with the intervention, and hence, fidelityterm·dom will be low. Adequate fidelityterm·dom will also require other enabling factors such as provider training and confirmed information flow. Without adequate fidelityterm·dom, it would be wasteful to conduct a feasibilityterm·key study. Factors such as appropriate resources, workforce, technology, and management will be required for feasibilityterm·key. If feasibilityterm·key (supported by acceptabilityterm·dom and fidelityterm·dom) in the research context is adequate, it is appropriate to consider testing acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key in the healthcare context. If adopted consistently in one healthcare context, it is appropriate to consider scaling the intervention to other settings, provider groups and patient groups. In each new setting, it would be wise to re-assess acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key as described above because adequate feasibilityterm·key in one setting at one time, whilst a positive sign, is not a guarantee that the intervention will be feasible in other settings. Similarly, over time, the factors that support feasibilityterm·key may change, thus threatening sustainabilityterm·key. It would therefore be prudent to continue to assess the factors affecting feasibilityterm·key over time to detect any problems that need to be addressed.

Discussion Based on the findings from the overview of reviewsterm·mth and the group consensus process, we propose a framework of implementabilityterm·dom of healthcare interventionsterm·dom which includes five key concepts, namely, acceptabilityterm·dom, fidelityterm·dom, feasibilityterm·key, scalabilityterm·dom and sustainabilityterm·key. The framework illustrates the interrelationship between the concepts and chronology, with acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key requiring investigation during early stages of the development of a healthcare implementationterm·dom, including during proof-of-principle studies and pragmatic evaluations of intervention effectiveness at one point in time and in one specific context. Acceptabilityterm·dom is a necessary but not sufficient condition for fidelityterm·dom, and similarly, fidelityterm·dom is a necessary but not sufficient condition for feasibilityterm·key. All three concepts are context- and population-dependent and will require reinvestigation as the healthcare interventionterm·dom is scaled to different settings and populations and over time. We argue that there is an association between the concepts, with acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key influencing the scalabilityterm·dom and sustainabilityterm·key of a healthcare interventionterm·dom. We are not suggesting that there is a causal relationship between the concepts, rather, scalabilityterm·dom and sustainabilityterm·key depend on the pre-conditions acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key of the healthcare interventionterm·dom, and these concepts should be re-examined over time, and as the healthcare interventionterm·dom is implemented with different populations or in different settings. This argument is consistent with other frameworks on scalabilityterm·dom and sustainabilityterm·key, including the Intervention Scalabilityterm·dom Assessment Tool [ 82 ] and the Dynamic Sustainabilityterm·key Framework [ 98 ], both of which suggest that feasibilityterm·key, acceptabilityterm·dom and fidelityterm·dom must be considered in the planning for scaling up and sustainabilityterm·key of a healthcare interventionterm·dom. The Consolidated Framework for Implementationterm·dom Research [ 12 ] suggests that the outer setting, including the social and economic context, can influence implementationterm·dom. Our proposed framework encourages the researcher to prospectively assess acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key in both the inner and outer contexts as the key stakeholders are likely to be different.

From the 252 reviewsterm·mth identified in the overview, the majority did not provide a definition of the concept. Rather, the reviewsterm·mth used measurement approaches which implied a definition. For example, feasibilityterm·key was typically defined by measuring components such as compliance to the intervention, dropouts, recruitment rates and adverse events. There was conflating of terms, with feasibilityterm·key and safety used interchangeably in several reviewsterm·mth, particularly drug feasibilityterm·key reviewsterm·mth. Although frameworks were identified for all five concepts, they were not frequently used in the reviewsterm·mth identified in the overview. Most of the frameworks included conceptual definitions and operationalisation of the concept, but these varied between frameworks. It is difficult to test the influence of these concepts on the implementabilityterm·dom of a healthcare interventionterm·dom without consistent definitions, descriptions, operationalisation and measurement approaches for these concepts.

Of all the concepts explored in the systematic overview, scalabilityterm·dom and sustainabilityterm·key of healthcare interventionsterm·dom were not often achieved. These findings suggest that healthcare interventionsterm·dom may be found to be effective, acceptable and feasible in the development or pilot phase, but this does not guarantee successful scale-up or sustainment of the intervention over time. We propose that the framework of implementabilityterm·dom can provide a dynamic, longitudinal perspective of intervention development where researchers consider acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key during the earlier phases of intervention development and implementationterm·dom, and iteratively re-evaluate these factors as the healthcare interventionterm·dom is scaled to different settings and over time.

Other rigorous frameworks, such as RE-AIM and the Implementationterm·dom Outcomes Framework, propose that sustainable adoption and implementationterm·dom of healthcare interventionsterm·dom require consideration of many like concepts such as acceptabilityterm·dom, reach, effectiveness, adoption, implementationterm·dom and maintenance [ 99 , 100 ]. Whilst there are some similarities in the concepts contained in these frameworks and our proposed framework of implementabilityterm·dom, the latter is explicitly concerned with the prospective and ongoing identification of factors that will influence scalabilityterm·dom and sustainabilityterm·key of a healthcare interventionterm·dom. It should also be noted that prospective identification of factors that influence implementabilityterm·dom of healthcare interventionsterm·dom is receiving growing attention in the literature, particularly in relation to reducing avoidable research wasteterm·key [ 101 ]. A recent publication developed a framework to assist researchers to prospectively cost the different phases of healthcare interventionsterm·dom. The authors argue that implementationterm·dom costs are often underestimated or not included in cost-effectiveness analyses. This in turn contributes to research wasteterm·key which may have been avoided through a more systematic and earlier approach to identifying the factors that support the translation of effective interventions into real-world settings, and prospectively costing the implementationterm·dom of these.

It has been argued that involving end-users in the development and implementationterm·dom of healthcare interventionsterm·dom may improve outcomes through enhanced relevance, acceptabilityterm·dom and feasibilityterm·key [ 102 ]. We propose that our framework of implementabilityterm·dom could be used to test these assumptions. We recommend that researchers prospectively set criteria to inform the decision about whether to abandon, amend or proceed with the intervention, depending on the outcomes of the feasibilityterm·key study [ 103 ]. Table 12 illustrates how the framework of implementabilityterm·dom could be used to prospectively guide the ongoing implementationterm·dom activities of healthcare interventionsterm·dom. Enabling factors were identified from published frameworks and reviewsterm·mth included in step 1, though this is not an exhaustive list and is likely to be influenced by context. For example, scalabilityterm·dom of healthcare interventionsterm·dom to low- and middle-income countries may be more, or less, enabled by factors that differ from those in high-income countries [ 104 ]. The framework of implementabilityterm·dom of healthcare interventionsterm·dom may be particularly helpful in guiding effectiveness-implementation hybrid designs, which aim to simultaneously evaluate effectiveness of the intervention in the real-world context and the implementationterm·dom strategy [ 105 – 107 ].

Example plan for using the framework of implementabilityterm·dom We propose that empirical investigation of the framework of implementabilityterm·dom is required to answer the following questions: What is the nature of the relationship between the key concepts? (e.g. linear, curvilinear or threshold?) Do acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key predict scalabilityterm·dom and sustainabilityterm·key as proposed in the framework?

Can identified deficits in constructs of the framework be addressed to enhance the implementabilityterm·dom of effective interventions?

Strengths and limitations To our knowledge, this is the first reviewterm·mth that considers all five key concepts in published reviewsterm·mth on healthcare interventionsterm·dom. The overview collated and described important information on concepts that are increasingly being assumed to influence implementationterm·dom of healthcare interventionsterm·dom. The development of a framework utilising well-established consensus methods is another strength. Although one author was responsible for most of the screening, data extraction and coding, independent extraction by a second author of 10% of the reviewsterm·mth confirmed reliability of the extraction process.

In order to the make the overview of reviewsterm·mth feasible, we only focused on publications that had one or more of the concepts in the title and/or abstract. Therefore, it is possible we may not have identified some reviewsterm·mth that were relevant. It must also be noted that the framework of implementabilityterm·dom of healthcare interventionsterm·dom is untested. We propose that it articulates some of the untested assumptions in the current literature on implementation scienceterm·key and have suggested some approaches for empirical evaluation of the framework.

We do not propose that interventions with high implementabilityterm·dom will automatically result in high uptake. As we argued in the background to this paper, the features of interventions may interact with top-down and bottom-up implementationterm·dom activities, and with contextual factors, to achieve consistent uptake into routine practice. Our argument is that these implementationterm·dom activities are more likely to be effective if implementabilityterm·dom of the intervention is high.

Conclusions The framework developed in this study can inform research that aims to prospectively and iteratively identify the likely implementabilityterm·dom of evidence-based healthcare interventionsterm·dom. We suggest that the framework be tested empirically through studies that examine the actual uptake of interventions, across settings and over time, compared with prospective assessments of the independent variables (acceptabilityterm·dom, fidelityterm·dom and feasibilityterm·key) and the outcome variables (scalabilityterm·dom and sustainabilityterm·key) in the framework. We recommend that, to avoid research wasteterm·key, implementabilityterm·dom should be assessed, and enhancements made, during the clinical evaluation stages of the development of interventions [ 1 , 3 ]. This would potentially accelerate their uptake into clinical practice.

Supplementary Information Additional file 1. Search strategy.

Additional file 2. Papers included in the overview.

Additional file 3. Characteristics of studies included in the overview of reviewsterm·mth (n=252).

Acknowledgements Not applicable.

Abbreviations Assess, Innovate, Develop, Engage and Devolve Consolidated Framework of Implementation Researchterm Feasibilityterm·key, Appropriateness, Meaningfulness and Effectiveness Implementationterm·dom Outcomes Framework Intervention Scalabilityterm·dom Assessment Tool Implementationterm·dom of Treatment Integrity Procedures Scale Non-adoption, Abandonment, Scale-up, Spread and Sustainabilityterm·key Nominal Group Techniqueterm National Institute of Health Behavior Change Consortium Preferred Reporting Items for Systematic Reviewsterm·mth Reach, Effectiveness, Adoption, Implementationterm·dom and Maintenance Structured Assessment of Feasibilityterm·key Systematic reviewterm·mth World Health Organisation Authors’ contributions MK completed the systematic searches, synthesis of data, framework design and substantial writing in collaboration with JF. SK was the second reviewer for 10% of the literature included in the reviewterm·mth and commented on drafts. PH commented on drafts and provided a clinical scenario. WC commented on drafts and provided a clinical scenario. LD commented on drafts and provided a clinical scenario. DS commented on drafts and provided a clinical scenario. JF contributed the original idea for the framework, coordinated the authorship team and substantial writing in collaboration with MK. All authors participated in the consensus process and approved the final submitted version of the manuscript.

Funding Not applicable.

Availability of data and materials All data generated during this study are included either within the text or as an additional file.

Declarations Ethics approval and consent to participate Not applicable.

Consent for publication Not applicable.

Competing interests The authors declare that they have no competing interests.

Footnotes Contributor Information Marlena KlaicpersName, Email: marlena.klaicpersName@unimelb.edu.au.

Suzanne KapppersName, Email: suzanne.kapppersName@unimelb.edu.au.

Peter HudsonpersName, Email: phudson@unimelb.edu.au.

Wendy ChapmanpersName, Email: wendy.chapmanpersName@unimelb.edu.au.

Linda DenehypersName, Email: l.denehypersName@unimelb.edu.au.

David StorypersName, Email: dastory@unimelb.edu.au.

Jill J. FrancispersName, Email: jillian.francispersName@unimelb.edu.au References Associated Data Supplementary Materials Additional file 1. Search strategy.

Additional file 2. Papers included in the overview.

Additional file 3. Characteristics of studies included in the overview of reviewsterm·mth (n=252).

Data Availability Statement All data generated during this study are included either within the text or as an additional file.