What's in a mechanism? Development of a key concept in realist evaluation. — TEI-tagged view

Sonia Michelle DalkinpersName, Joanne GreenhalghpersName, Diana JonespersName, Bill CunninghampersName, Monique LhussierpersName (2015date). “What's in a mechanism? Development of a key concept in realist evaluation..” Implementation Sciencetitle·j.

Object of study — the concept of mechanismterm·key in realist evaluationterm·key

Research question — whether separating resourcesterm·key from reasoningterm·key, and treating activation as a continuum, clarifies mechanismsterm·dom

Contribution — disaggregating mechanismsterm·dom into resourcesterm·key and reasoningterm·key to tell mechanismterm·key from contextterm·key

Full text — tagged

The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.

What's in a mechanismterm·key? Development of a key concept in realist evaluationterm·key.

Abstract The idea that underlying, generative mechanismsterm·dom give rise to causal regularities has become a guiding principle across many social and natural science disciplines. A specific form of this enquiry, realist evaluationterm·key is gaining momentum in the evaluation of complex social interventions. It focuses on ‘what works, how, in which conditions and for whom’ using contextterm·key, mechanismterm·key and outcome configurations as opposed to asking whether an intervention ‘works’. Realist evaluationterm·key can be difficult to codify and requires considerable researcher reflection and creativity. As such there is often confusion when operationalising the method in practice. This article aims to clarify and further develop the concept of mechanismterm·key in realist evaluationterm·key and in doing so aid the learning of those operationalising the methodology.

Using a social science illustration, we argue that disaggregating the concept of mechanismterm·key into its constituent parts helps to understand the difference between the resourcesterm·key offered by the intervention and the ways in which this changes the reasoningterm·key of participants. This in turn helps to distinguish between a contextterm·key and mechanismterm·key. The notion of mechanismsterm·dom ‘firing’ in social science research is explored, with discussions surrounding how this may stifle researchers’ realist thinking. We underline the importance of conceptualising mechanismsterm·dom as operating on a continuum, rather than as an ‘on/off’ switch.

The discussions in this article will hopefully progress and operationalise realist methods. This development is likely to occur due to the infancy of the methodology and its recent increased profile and use in social science research. The arguments we present have been tested and are explained throughout the article using a social science illustration, evidencing their usability and value.

Background The idea that enquiry works by uncovering the underlying, generative mechanismsterm·dom that give rise to causal regularities has become a guiding principle across many social and natural science disciplines. This article aims to provide a brief description of social mechanismsterm·dom, mechanismsterm·dom within evaluation and then specifically mechanismsterm·dom in realist evaluationterm·key. The principles of Pawson and Tilleyterm·key’s [ 1 ] conceptualisation of mechanismterm·key will then be discussed and operationalised through a reconceptualisation of the Context-Mechanism-Outcome configurationterm (CMOcterm) and an understanding of mechanismsterm·dom on a continuum of activationterm·key.

Much ado about mechanismsterm·dom Social mechanismsterm·dom One of the key tenets of realism is the very basic idea that observational evidence alone cannot establish causal uniformities between variables. Rather, it is necessary to explain why the relationships come about; it is necessary to establish what goes on in the system that connects its various inputs and outputs. In this manner, physicists are able fully to understand the relationship between the properties of a gas (as measured by the variables—pressure, temperature and volume) using knowledge about the kinetic action of the constituent molecules. In pharmacology, the term ‘mechanism of actionterm·dom’ refers to the specific biochemical interaction through which a drug substance acts on the body to generate its curative effect. Programme evaluators do not suppose that CCTV (the intervention) causes a fall in crime rates (the outcome). It does so, when it does so, by persuading potential perpetrators of increased risks of detection (the mechanismterm·key). In all cases, science delves into the ‘black box’. In all cases, the mechanismterm·key is what generates the observed relationship.

Whilst it is possible to recognise the affinities in explanatory structure across these examples, they also demonstrate that the action of the generative mechanismsterm·dom is quite different, to such an extent indeed that that they defy a simple, unitary definition of their nature and content. Pawson expands on the applications of generative vs successive conceptualisations of causation elsewhere [ 2 ].

Readers of this journal will need no reminding that these paradigms have been debated for many years. Realists see physical and social reality as stratified and emergent. Things that cannot be cast as variables yet are vital to explanation (like kinetic forces, cultural norms and human interpretation or agency) are missing from correlational methods. Causal associations themselves are rarely universal; they are adaptive ‘demi-regularities’, which are always strongly influenced by setting and contextterm·key. The original sources for these arguments may be found in Hesse [ 3 ], Harré [ 4 ], Pawson [ 2 , 5 ], Sayer [ 6 , 7 ], Bhaskar [ 8 ], Boudon [ 9 ] and Stinchcombe [ 10 ].

We acknowledge the further cleft between ‘critical realismterm’ and ‘scientific realismterm’. The writings of Bhaskar [ 8 , 11 ] and Pawson [ 2 ] serve as a reasonable proxy for these two schools. They differ on the matter of whether social science can create ‘closed system’ investigations. For Bhaskar, the closed system, experimental control available to the natural scientist is not achievable in social research because of ever-present emergenceterm·dom, that is to say the unique and unceasing human capacity to change the circumstances in which they live. As a ‘substitute’ for closed system empirical enquiry, he thus proposes the usage of abstract, a priori reasoningterm·key and the admission of a moral lens through which to critically evaluate human actions ([ 11 ], p. 64). Pawson, by contrast, argues much more pragmatically that neither physical science nor social science investigation depends on the achievement of closed systems ([ 5 ], p. 67). There are no crucial experiments (most especially Randomised Controlled Trials) which alone furnish us with social laws. But equally, natural science only ever makes slow and imperfect progress in gathering knowledge of the potentially infinite number of contingencies that can shape a physical system. Investigatory closure is always partial. Again, we are presented with rather different visions, the only contradiction occurring when an investigation claims to be both normative and scientific.

For Archer [ 12 ], collective, constrained decision making is the underlying mechanismterm·key that creates all social outcomes. Society is made by but never under the control of human intentions. At any given time, peoples’ choices are conditioned by pre-existing social structures and organisations. We are thus externally constrained in our actions but always part of human agency is the choice to attempt to change the initial conditions that bear down on us. These adaptive choices, over time, go on to mould novel structures and changed institutions. Collectively, our present decisions congregate to form new systems, which in their own turn, constrain and enable the choices of the next generation. Society is thus patterned and re-patterned by wilful action, but as Archer reminds us, the causal outcomes never conform to anyone’s wishes—even the most powerful.

Most realists would affirm this broad account of the mechanismsterm·dom of social change, where structures shape actions, which shape structure, which shape actions, and so on. There are, however, some significant differences in where they locate the precise locus of that change. For Bhaskar [ 8 ], causal mechanismsterm·dom sit primarily within the structural component of the social world. They reside in the power and resourcesterm·key that lie with the great institutional forms of society. For other realists, such as Pawson and Tilleyterm·key [ 1 ], mechanismsterm·dom are identified at the level of human reasoningterm·key. Thus, mechanismsterm·dom can have different meanings depending on the scope of the intended explanation. Structural mechanismsterm·dom come to the fore if the social scientist is attempting to explain large-scale social transformations. If, however, the researcher is attempting to discover whether a particular fitness programme creates healthier participants, it can be assumed that key outcomes will result from the reasoningterm·key and responses of the participants.

Mechanismsterm·dom in evaluation This brings us to a consideration of mechanismsterm·dom in evaluation research; here the focus is on developing an explanation of how a particular programme works through changing the reasoningterm·key and responses of participants to bring about a set of intended outcomes. There have been a number of different conceptualisations of mechanismterm·key within evaluation. Chen and Rossi [ 13 ] were among the first researchers to use the term ‘mechanismterm·key’ and highlight its significance in theory-driven evaluation [ 14 ]. In 2005, Chen [ 15 ] broadened our understanding of causal mechanismsterm·dom by identifying two types: mediating and moderating. He defines these as follows: “A mediating causal mechanismterm·dom is a component of a program that intervenes in the relationship between two other components . . . [while] the second type of causal mechanismterm·key—moderating—represents a relationship between program components that is enabled, or conditioned, by a third factor.” (pp. 240–241) Weiss [ 16 ] also reflects on mechanismsterm·dom, in terms of programme theoryterm·key. She states that it is important to understand the difference between implementation theory and programme theoryterm·key. The earlier can be conceptualised as a logic model, whereas the latter: “. . . deals with the mechanismsterm·dom that intervene between the delivery of program service and the occurrence of outcomes of interest. It focuses on participants’ responses to program service. The mechanismterm·key of change is not the program service per se but the response that the activities generate.” (p. 46) As Weiss [ 16 ] states, mechanismsterm·dom are not the programme service but the response it triggers from stakeholders and resulting outcome. For example, Vassilev et al.’s [ 17 ] metasynthesis investigated how social networks can make a considerable contribution to improving health outcomes for people with long-term conditions (specifically, type 2 diabetes). They identified three themes which translated into three ‘network mechanismsterm·dom’: network navigation (identifying and connecting with relevant existing resourcesterm·key in a network), negotiation within networks (re-shaping relationships, roles, expectations, means of engagement and communication between network members) and collective efficacy (developing a shared perception and capacity to successfully perform behaviour through shared effort, beliefs, influence, perseverance, and objectives). The authors highlight not only resourcesterm·key in these mechanismsterm·dom but also reasoningterm·key; these mechanismsterm·dom convey the close interdependence between social and psychological processes in long-term conditions management. Furthermore, these network mechanismsterm·dom are subject to contextterm·key, as the authors state: “they are shaped by the environments in which they take place which can be enabling or disabling depending on the capacities they offer for carrying out illness management work and supporting behaviours beneficial for people’s health.” (p. 10) Despite the many different conceptualisations, e.g. [ 9 , 13 - 16 , 18 ], and applications of mechanismsterm·dom, e.g. [ 17 , 19 , 20 ], most in some way have been influenced by the critical realismterm and scientific realismterm accounts of causation, e.g. [ 1 , 21 , 22 ], discussed above. In these schools of thought, mechanismsterm·dom are usually hidden, sensitive to variations in contextterm·key and generate outcomes. As Astbury and Leeuw [ 14 ] state, mechanismsterm·dom in realism are: “underlying entities, processes, or structures which operate in particular contextsterm·key to generate outcomes of interest.” (p. 368) We survey this broader terrain as a prelude to focussing on the more specific version of mechanismterm·key thinking referred to by Pawson and Tilleyterm·key that has come to play a key role in the evaluation of social interventions, namely realist evaluationterm·key [ 1 ], which is the main focus of this article.

Mechanismsterm·dom in realist evaluationterm·key Within the scientific realismterm approach, Pawson and Tilleyterm·key [ 1 ] have provided their own conceptualisation of mechanismsterm·dom; mechanismsterm·dom are a combination of resourcesterm·key offered by the social programme under study and stakeholders’ reasoningterm·key in response [ 1 ]. They state that mechanismsterm·dom will only activate in the right conditions, providing a contextterm·key + mechanismterm·key = outcome formula as a guiding principle to realist enquiry [ 1 ]. This article sits within the empirical application of realism in the form of realist evaluationterm·key and the usage of mechanismsterm·dom therein. In particular, we make a case for the explicit disaggregation of resourcesterm·key and reasoningterm·key in implementation endeavours, to which task we now turn.

The units of analysis within realist evaluationterm·key are programme theoriesterm·dom—the ideas and assumptions underlying how, why and in what circumstances complex social interventions work. Many readers will by now be very familiar with programme theoriesterm·dom expressed as CMOcterm and with the fact that data collection and analysis in realist evaluationterm·key centres on the process of developing, testing and refining CMOcterm. In the next section of the paper, we propose a development of this formula, which aims to facilitate the study of implementation processes and interventions.

A social science illustrative case study In order to illustrate our argument in this article and maximise explanatory reach, we draw on empirical data from our realist evaluationterm·key of a palliative careterm·key Integrated Care Pathwayterm·dom (ICPterm·dom). The ICPterm·dom aimed to improve the co-ordination of care for people in the final year of life by identifying individuals approaching end-of-life, assessing and agreeing how needs and preferences of patients could be met, providing support for families and carers and using Advance Care Planning (ACP) to manage the patients’ final illness in order to achieve a ‘good’ (preference based) death. The ICPterm·dom comprised a variety of interventions including palliative careterm·key registrations, ACP and multidisciplinary team meetings in order to anticipate and plan care for patients with palliative careterm·key needs. We evaluated the implementation of the ICPterm·dom across 14 GP practices in one UK locality using realist evaluationterm·key. Five initial programme theoriesterm·dom, generated from immersion in the field and literature on ICPsterm·dom, were tested: (1) the embeddedness of the ICPterm·dom into GP practices, (2) the registration of palliative careterm·key patients, (3) preference discussions and ACP, (4) facilitating difficult conversations and (5) facilitating home deaths. The five refined programme theoriesterm·dom were combined to create one overall programme theoryterm·key of the whole ICPterm·dom. This encapsulated the ICPterm·dom as a translational tool of national policy drivers (such as shared decision making, patient-centred care and proactive care) into local practice.

Using realist evaluationterm·key to shed light on how such a complex intervention could work in practice made intuitive sense but proved not to be without operational challenges. These have been echoed by other realist researchers [ 23 - 25 ] and have prompted the writing of this paper.

This paper has two main aims: To make a case for the explicit disaggregation of resourcesterm·key and reasoningterm·key within mechanismsterm·dom; To reiterate the need for nuance in considering whether mechanismsterm·dom fire in a dual on/off mode.

Discussion Disaggregating mechanismsterm·dom into resourcesterm·key and reasoningterm·key 1 The concern Realists posit that exposing not only the mechanismsterm·dom of change in an intervention but more importantly their relationship to the contextterm·key of their implementation is key to the evaluation of complex programmes [ 20 , 26 ]. However, deciding whether aspects within an intervention implementation process in a realist project contribute contextually or mechanistically to the overall explanatory endeavour has become the realist researcher’s quandary [ 14 , 23 , 27 ]. Like these authors, we encountered challenges in distinguishing between contextterm·key and mechanismterm·key in our evaluation of the ICPterm·dom and were cognisant of the need not to conflate programme strategy (the intervention) with mechanismterm·key. We concur with Jagosh et al. [ 23 ], who note how it is not always as straightforward as might be assumed to map the complexities of the transformation process and the multiple systems within which it operates onto the C + M = O formula. Arguably, outcomes can be identified with most ease; they are observed or measured or at least aimed at with a degree of clarity. Although the distinction between resourcesterm·key and reasoningterm·key is used in Pawson and Tilleyterm·key’s seminal work [ 1 ], their relative importance in understanding mechanismsterm·dom is often understated. Consequently, researchers often emphasise one at the expense of the other, under the banner of mechanismterm·key [ 25 ]. To address this, we offer the solution below.

2 Our way forward Building on the original work of Pawson and Tilleyterm·key [ 1 ], we would like to propose an alternative operationalisation of the CMOcterm formula: Intervention resourcesterm·key are introduced in a contextterm·key, in a way that enhances a change in reasoningterm·key. This alters the behaviour of participants, which leads to outcomes.

The revised formula therefore reads: M ( Resourcesterm·key ) + C → M Reasoningterm·key = O Resourcesterm·key and reasoningterm·key are mutually constitutive of a mechanismterm·key, but explicitly disaggregating them can help operationalise the difference between a mechanismterm·key and a contextterm·key. Although resourceterm·key and reasoningterm·key are made explicit in the seminal work of Pawson and Tilleyterm·key [ 1 ], they have often not been referred to explicitly in subsequent research. In our own study, through using this formula, it became clearer whether data contributed contextually or mechanistically, as we could identify mechanismterm·key components (resourceterm·key and reasoningterm·key) which are different to contextsterm·key. Figure 1 illustrates how we have presented the new formula diagrammatically in the ICPterm·dom study. Through trial and error, it became clear that the original formula could be built upon, hence the new formula which disaggregates resourceterm·key and reasoningterm·key, placing ‘contextterm·key’ in between. However, this is not to be confused with just using resourcesterm·key without reasoningterm·key—they must always come as a pair. It is important to note here that this new formula is only an extension of the original heuristic developed by Pawson and Tilleyterm·key [ 1 ]. This new formula does not aim to re-draw the full sequence of causation but to modify the basic heuristic to aid operationalisation of realist approaches. Figure 1 A CMOcterm framework.

Differentiating between resourceterm·key (the component introduced in a contextterm·key) and reasoningterm·key therefore helps distinguish between relevant contextterm·key and mechanismterm·key. Identifying the resourceterm·key is contingent on the purpose of the study, and identifying the reasoningterm·key avoids the issue of conflating programme strategy (resourceterm·key) with mechanismterm·key.

3 The social science illustration In the palliative careterm·key ICPterm·dom study, an outcome pattern was observed that practices identified and placed fewer palliative patients with non-cancer illnesses on their palliative careterm·key registers, in comparison to those with cancer illnesses. This was common across all 14 practices studied and was particularly noticeable for patients residing in care homes, where many older adults have non-cancer illnesses. Patients with non-cancer illnesses have unpredictable illness trajectories, meaning that registering this patient group is challenging for health care professionals, as a period of significant decline can be followed by substantial improvement, despite a downward trend in wellness [ 28 , 29 ]. Comparatively, this is not the case with cancer diagnoses as often there is a specific diagnosis and steady illness trajectory. We aimed to generate a CMOcterm to explain why there were less palliative careterm·key registrations of patients with non-cancer illnesses than cancer patients (outcome). In attempting to formulate the configuration, we were uncertain whether the contextterm·key was the unpredictable illness trajectories of older adults without a cancer diagnosis, or care homes in general or the palliative careterm·key register being difficult to use with non-cancer patients. Breaking down the C + M = O formula to include resourceterm·key and reasoningterm·key using the new formula, M (resourceterm·key) + C → M (reasoningterm·key) = O, helped in deciphering the contextterm·key from the mechanismterm·key. The use of the new formula diagram (Figure 2 ) also helped in configuring the whole CMOcterm. Figure 2 displays the novel way in which the new formula should be represented diagrammatically. Through using the new formula and associated diagram, it became clear that the resourceterm·key was the palliative careterm·key register which, when used with older adults who had unpredictable illness trajectories (contextterm·key), resulted in anxiety in registering these patients (reasoningterm·key), which meant that less older patients in care homes were registered (outcome) (Figure 2 ). Through understanding that resourcesterm·key were introduced into pre-existing contextsterm·key in a way that altered the participants’ reasoningterm·key, it becomes easier to explain the differential registration numbers (outcome). Figure 2 Refined CMOcterm for patients in care homes receiving the ICPterm·dom.

Disaggregating resourcesterm·key and reasoningterm·key encourages researchers to consider both concepts, rather than privileging one at the expense of the other and will contribute significantly to the explanatory endeavour of the realist researcher. It is important to understand the new formula (M (Resourceterm·key) + C → M (Reasoningterm·key) = O) highlights that resourcesterm·key must be introduced into a pre-existing contextterm·key, which in collaboration induces an individual’s reasoningterm·key, leading to an outcome. Distinguishing the resourcesterm·key that are introduced into contextsterm·key from the reasoningterm·key this generates can provide both an operational and a conceptual clarification of mechanismterm·key. It can enable researchers to clearly understand the role of contextterm·key in triggering mechanismsterm·dom, thus developing their explanation of how interventions work. We now turn to interrogate the notion of mechanismsterm·dom being ‘triggered’ in the next section of the paper.

A case for continuums of activation in reasoningterm·key 1 The concern A separate but related difficulty encountered when using mechanismsterm·dom in social science research is the notion that mechanismsterm·dom are often said to ‘fire’, ‘trigger’ or ‘modify’ in contextterm·key to create an outcome [ 1 , 30 - 32 ]. Pawson and Tilleyterm·key [ 1 ] use the much referenced gun powder analogy to explain this. When a spark is introduced to gun powder, the chemical composition of gun powder (mechanismterm·key) results in an explosion (outcome). However, there are no explosions if the contextterm·key is not right—damp conditions, insufficient powder, not adequately compact, no oxygen present, duration of heat applied is too short (contextterm·key). Thus it purports that causal outcomes follow from mechanismsterm·dom acting in contextsterm·key; this is the base from which all realist explanation builds. Most complex social interventions involve stakeholders’ volition (reasoningterm·key). As Pawson [ 33 ] states, “much more than in any other type of social programme, interpersonal relationships between stakeholders embody the intervention” [ 33 ]. We found it difficult to apply the firing analogy to interventions where human volition is entwined in the intervention. Reasoningterm·key in these cases is rarely activated via an on/off switch, triggered in favourable contextual conditions. Instead, activation operates along a continuum similar to the light created by a ‘dimmer switch’, where intensity varies in line with an ever evolving contextterm·key. Our experience suggests that researchers are often enabled to develop their realist thinking further when this myth of on/off reasoningterm·key is dispelled. The metaphor of the dimmer switch accommodates the activation of new volition as well as the idea of continuums of activation.

2 Our way forward Conceptualising volition as happening in a binary ‘firing’/‘not firing’ fashion masks a continuum of activationterm·key which can have more explanatory value in understanding how interventions work. There are varying degrees to which an individual can feel confident, angry or mistrustful, leading in turn to a gradation of outcomes.

3 The social science illustration In our evaluation of the ICPterm·dom, we observed that the volition of health care professionals was always on a continuum. Health care professionals felt anxious when registering older adults with an illness other than cancer, as the trajectory of such illnesses is so unpredictable (Figure 2 ). Health care professionals could not predict patients’ decline, did not wish to over populate their palliative careterm·key registers and were worried about registering patients who seemed relatively well but could decline quickly. Furthermore, once a decline in health begins in older adults with non-cancer illnesses, it can be very rapid and thus end-of-life care is implemented quickly and is often unplanned, which can result in a death that does not adhere to patient preferences. The anxiety of health care professionals working with palliative non-cancer patients was evident, yet this anxiety did not switch on and off, it developed over time, as patients’ illnesses progressed. It also differed between health care professionals; those with more experience of working with patients with non-cancer disease had less anxiety about registering them. Thus the reasoningterm·key of having anxiety was on a continuum for health care professionals using the palliative careterm·key register. There is a variation in the amount of anxiety a health care professional will feel when registering a patient with a non-cancer illness, it is not dichotomised; the degree to which this is felt is combined with a facilitative contextterm·key and appropriate resourceterm·key. This should lead to a more appropriate use of the palliative careterm·key register.

Summary This paper aimed to help the operationalisation of the C + M = O formula, through (1) a disaggregation of the mechanismterm·key resourceterm·key and mechanismterm·key reasoningterm·key and (2) a conceptualisation of activation continuums, rather than a binary trigger. The solutions proposed in this article will enable a clearer application of realist evaluationterm·key to understanding how complex interventions are implemented. We have already found some evidence to support this argument by applying it in our own teaching and workshops. For example, the ‘workability’ of this framework has been tested with researchers at the beginning of their realist journey in a realist summer school at the Centre for Advancement in Realist Evaluationterm·key and Synthesis (CARES), University of Liverpool. Course participants found it useful to guide their realist learning, understand the method further and clarify the differences between mechanismterm·key and contextterm·key, and resourcesterm·key and reasoningterm·key.

We hope that this article furthers the discussions on the operationalisation of realist theory development in a way that, in particular, helps novice realist researchers to embrace and in turn develop the methodology. The authors would welcome testing of the methodological refinements discussed throughout this article by other researchers across a wide range of fields, with such testing aiding further developments.