Evidence-based mechanistic reasoning

An evidential role for mechanisms

Systematic reviews of high quality randomized trials generally count as the ‘best evidence’.¹ However, well-conducted randomized trials are sometimes unavailable,^2,3 unfeasible,⁴ unethical⁵ or unnecessary.^6,7 In such cases other forms of evidence must be considered. Many EBM proponents accept mechanistic reasoning (‘pathophysiologic rationale’) for generalizability,^1,8 hypothesis generation,⁹ ruling out implausible hypotheses,^10,11 and for supporting efficacy in the absence of other ‘stronger’ forms of evidence. Yet because mechanistic reasoning has often led us astray,^12,13 most EBM proponents are justifiably sceptical about using mechanistic reasoning as evidence for efficacy.

We suggest that the scepticism about the value of mechanistic reasoning should not extend to high quality mechanistic reasoning. Just as poor quality randomized trials (that are unblinded,^14–16 underpowered or biased,¹⁷ that employ unconcealed allocation,^15,16 or otherwise biased) will not provide high quality evidence for efficacy, so poor quality mechanistic reasoning will be unreliable. In this theoretical exploration we suggest that mechanistic reasoning involving a not incomplete inferential chain and that takes potential complexity into account can and should be used as evidence of efficacy. We support our rules for mechanistic evidence with three examples.

Comparative clinical studies and mechanistic reasoning

Comparative clinical studies (either randomized trials or observational studies) contrast an experimental therapy with a comparator, and provide direct evidence of a relationship between the intervention and the clinically relevant outcome. For example, a randomized trial of antiarrhythmic drugs versus placebo suggested that the drugs unexpectedly increased mortality by 3.3%.¹⁹ This conclusion did not rely on an explanation of how they did so – that remained a ‘black box’ ( Figure 1).

Figure 1.

The ‘black box’ in a comparative clinical study

Mechanistic reasoning involves looking inside the ‘black box’, and relies on knowledge of the underlying mechanisms to predict what the relevant effect of a therapy will be. For example, it was known that myocardial infarction damages cardiac muscle and conducting tissues, leaving the heart susceptible to arrhythmias. One type of arrhythmia, ventricular extra beats (VEBs) can degenerate into ventricular tachycardia or fibrillation, followed by death in the absence of electric shock. Large-scale epidemiological studies suggested that 25–50% of sudden cardiac deaths were associated with arrhythmias.²⁰ Based on this knowledge of the underlying mechanisms, it seemed rational to assume that reducing VEBs would reduce mortality. As a result, many drugs were designed and subsequently prescribed to regulate VEBs.

Mechanisms and mechanistic reasoning

Before describing the problems with mechanisms as evidence, it is useful to distinguish between mechanisms and mechanistic reasoning.

Mechanisms are arrangements of parts/features that (allegedly) ensure a stable relationship between ‘inputs’ and ‘outputs’.

The heart (as a pump), the brain (as a ‘control centre’), and the liver (as a detoxifying agent, among other things) are all mechanisms in this sense. Previous terms have included ‘apriority’²¹ and ‘theory’,²² but the term ‘mechanisms’ has gained purchase in the philosophical literature,²³ and is commonly used in scientific discussions.

However, described mechanisms do not amount to evidence. Our knowledge of pathology and physiology is rarely sufficiently complete to infer precisely how an intervention will affect mortality or morbidity. For example, the proposed mechanism predicting that antiarrhythmic drugs would reduce mortality in patients with asymptomatic cardiac arrhythmias after myocardial infarction was mistaken (Figure 2). Although the drugs reduced VEBs they also had a proarrhythmic effect in some patients; moreover, the drugs might have affected other mechanisms that, in turn, increased mortality. For mechanisms to be evidential, we must make an inference from (alleged) knowledge of the relevant mechanisms to claims about treatment effects.

Figure 2.

An example of poor mechanistic reasoning; the original hypothesis proposed that antiarrhythmic drugs would reduce mortality (middle column); in fact they increased it (right hand column)

Mechanistic reasoning is the inference from mechanisms to claims that an intervention produced a patient-relevant outcome. Such reasoning will involve an inferential chain linking the intervention (such as antiarrhythmic drugs) with the outcome (such as mortality).

For example, antiarrhythmic drugs are orally administered, then delivered to the site of absorption in the gut by a combination of pharmaceutical mechanisms (related to the drug formulation) and physiological mechanisms (such as swallowing and gastric emptying). Next, the drugs are absorbed into the bloodstream, and they (or their metabolites) eventually reach their pharmacological targets in the heart via metabolizing, circulatory and binding mechanisms. Then antiarrhythmic drugs reduce the frequency of VEBs by modifying the heart's conducting mechanism. Finally, a reduction in VEBs (it was supposed) reduces the risk of sudden death (Figure 2).

It is almost always possible to describe a mechanism at a more detailed level. For instance, we might have included cardiac molecular mechanisms (e.g. actions on potassium channels) in our description of antiarrhythmic drug action. The essential feature of mechanistic reasoning as evidence is that it involves a coherent inferential chain linking the intervention with the patient-relevant outcome.

A framework for analysing mechanistic claims

We propose to analyse mechanistic reasoning in a spectrum involving four often overlapping phases²⁴ that characterize levels of mechanistic functioning (Table 1):

1. Phase I – Set-up: In order for any mechanism to be ‘activated’, various background, or ‘set-up’ factors and conditions must obtain. These include correct diagnosis, the availability of a suitable formulation (of a drug), the successful administration of anaesthesia (for surgery), the use of comfortable clothing (for physiotherapy), or the establishment of a quiet, calm environment (for psychological therapy);

2. Phase II – Delivery: Various mechanisms are involved in delivering the intervention. For the ‘pharmaceutical’ and ‘pharmacokinetic’ phases of orally-administered drugs these mechanisms include swallowing, gastric emptying, and absorptive and distributory mechanisms. For a surgical intervention such as a hip replacement, the relevant mechanisms might include cutting tissues and tying blood vessels, in order to access the hip-bone. For a psychological intervention such as cognitive behavioural therapy, they might include introductions, history-taking, and other preliminaries, and all the mechanisms required for speaking, listening and thinking;

3. Phase III – Action: Most therapies have a particular site of action. For antiarrhythmic drug therapy this (the ‘pharmacodynamic’ phase) would be the pharmacological effect on cardiac mechanisms. In surgery it would be repair or re-building of the relevant body part (such as replacing a hip). In psychological therapy it might involve providing insight (psychoanalysis) or suggesting a new behavioural pattern (Cognitive Behavioural Therapy);

4. Phase IV – Outcome: Whether rapid or delayed, the action of the intervention should produce a change in patient-relevant outcomes. Whenever the intended therapeutic effect is to reduce mortality, the relevant mechanism would be established by whatever definition of death is current (such as irreversible cessation of circulatory and respiratory functions, or irreversible cessation of all functions of the entire brain).^25,26 For many surgical interventions or physiotherapy the relevant outcomes might be improved function or quality-of-life measurements. For psychological interventions, they might include reduced depression, other patient-reported outcomes, and quality of life.

Table 1.

Different phases of mechanisms

	Examples of phase-specific mechanisms (or conditions, in the case of Phase I)
	Pharmacological therapy	Surgery (e.g. hip replacement)	Physiotherapy	‘Talking’ psychological therapies (e.g. CBT for depression)
Phase I – Set-up Conditions that must obtain in order to deliver the intervention	Correct diagnosis (implicit), availability of an appropriate formulation, etc.	Correct diagnosis (implicit), delivery of anaesthesia, clean instruments, etc.	Correct diagnosis (implicit), a comfortable environment in which the physiotherapeutic exercises can be performed, comfortable loose clothing, etc.	Correct diagnosis (implicit), establishing an appropriate (quiet, calm) environment, etc.
Phase II – Delivery The time after the ‘set-up’ and before the technology reaches the proximate site of action 1.1.1	Pharmaceutical and pharmacokinetic mechanisms (ADME)	(When the proximate site of action is accessed): skin, subcutaneous tissues, fat, muscle, blood vessels, etc.	(Such as when a preliminary strength exercise or stretch is performed): musculoskeletal system, nervous system, etc.	(The phase in which questions are asked, and problems are revealed): ‘mechanisms’ of speech, hearing, and understanding
Phase III – Action When the technology achieves proximate actions 1.1.2	Pharmacodynamic mechanisms	(When the problem is repaired): musculoskeletal mechanisms	(Such as a strength or flexibility exercise that improves neuromuscular functions): musculoskeletal, etc.	(When changes in understanding/ insight/ behaviour are achieved): cognitive mechanisms, etc.
Phase IV – Outcome The patient-relevant outcome that results from proximate actions and outcomes	Mechanisms involved in death (when the outcome is mortality); any other mechanisms involved in quality of life and other patient relevant outcomes; palliation, cure, prevention	Mechanisms involved in death (when the outcome is mortality); any other mechanisms involved in quality of life and other patient relevant outcomes, etc.	Mechanisms involved in death (when the outcome is mortality); any other mechanisms involved in quality of life and other patient relevant outcomes, etc.	Mechanisms involved in quality of life and other patient relevant outcomes

ADME = absorption, distribution, metabolism and excretion mechanisms

First problem with mechanistic reasoning: ‘empty’ and ‘partial’ mechanisms

In the spirit of Fisher's hypothesis tests²⁷ and Popper's falsification,²⁸ we propose that mechanistic reasoning should be ranked according to the extent to which it overcomes obvious flaws (Table 2).

Table 2.

Problems with mechanistic reasoning

Problem		Example
1. Problems with the mechanism	1a. Mechanism derived from a fanciful theory	Blood-letting
	1b. Mechanism sounds plausible but has no supporting evidence	Dr Spock's advice to place babies to sleep on their stomachs to reduce SIDS
	1c. Mechanism is partially supported by evidence, but some factors are ignored	Antiarrhythmic drug example (see text)
2. Problems with the inference from the mechanism to the conclusion of efficacy	2a. Failure to consider the probabilistic nature of the mechanism	Antiarrhythmic drug mechanism
	2b. Failure to consider the complexity of the mechanisms (including failure to consider mechanisms that might produce adverse events)	Paradoxical responses

Some mechanistic reasoning is flawed because the mechanisms on which it relies are ‘empty’ (they have little evidential basis). Many medical therapies, including leeching, blood-letting, cocaine as a non-addictive panacea,²⁹ and psychosurgery,³⁰ were based on assumptions about mechanisms that lacked an empirical basis. Dr Spock's advice to put babies to sleep prone in order to reduce the risk of Sudden Infant Death Syndrome (SIDS) was also empty. His reasoning, ‘if [an infant] vomits, he's more likely to choke on the vomitus’,³¹ was seductive but unsupported by evidence. Even if choking on vomitus caused SIDS, Dr Spock's advice was not evidence-based. Healthy babies, unlike drunk or drugged adults, are skilled at swallowing and spitting, and well-conducted comparative clinical studies^32,33 showed that putting babies to sleep on their stomachs increased the risk of SIDS. It should hardly be surprising that inferences from ‘empty’ mechanisms will not provide reliable evicence for efficacy.

The problem with mechanistic reasoning of this kind extends to ‘partial mechanisms’ that have some obvious gaps in the inferential chain linking the intervention to the clinically relevant outcome. Ironically, mechanistic reasoning based on partial mechanisms often causes more harm than reasoning based on empty mechanisms. A partial empirical basis for a mechanism can lend an air of authority, which in turn leads to greater use of a harmful treatment. For example, the mechanisms linking antiarrhythmic drugs with a reduced risk of arrhythmias was partially supported by strong evidence, but the link between VEBs and mortality was merely an association. Worldwide it has been estimated that antiarrhythmic drugs killed more people than were killed in action during the whole of the Vietnam War.³⁴

Surrogates (such as a reduction in VEBs) for the desired outcome (such as mortality) are common examples of mechanistic reasoning based on ‘partial’ mechanisms. The evidence linking the surrogate and the clinically relevant outcome is often lacking.^35–37

All mechanisms are, at least to some extent, ‘partial’, in the sense that they are not completely understood.

Second problem with mechanistic reasoning: the probabilistic and complex nature of mechanisms

While it is generally accepted that most associations are probabilistic (nobody believes that all smokers develop lung cancer), the implication of the probabilistic and complex nature of mechanisms often goes unacknowledged. For example, antiarrhythmic drugs did not always suppress VEBs – they worked in about 90% of patients (Figure 3).³² Links in a mechanistic chain are rarely deterministic or even stochastically multiplicative. If one ‘output’ occurs on only 10% of the occasions that an ‘input’ is delivered, then even if the remaining links are strong, the overall correlation between intervention and outcome cannot exceed 10% (assuming independence). Or, if a mechanism consists of five ‘strong’ links (each representing, say, 90% dependency), then (again assuming independence) overall correlation cannot exceed 0.9⁵ = 59% (Figure 3). In reality, of course, the links are almost always far weaker, and the extent to which independence holds is unknown, making any estimate of the treatment benefit likely to be inaccurate.

Figure 3.

The probabilistic nature of mechanistic reasoning

But even if the failure to assume independence makes it difficult to estimate the precise effect size, it might seem reasonable to infer some association, given complete mechanisms. In fact, this assumption is unwarranted. Even if one causal pathway between the intervention and a particular outcome (via the correctly identified mechanisms) is identified, it remains possible that the intervention (or some other component of the mechanisms) instigates another series of events that negates the impact of the outcome of the initial causal pathway (Figure 2). A rare but illustrative type of complexity involves a ‘paradoxical’ response, whereby one outcome occurs in some and the opposite outcome in others.^38,39 For example, there was a proarrhythmic effect of antiarrhythmic drugs in about 7% of patients.³⁴

Paradoxical responses and unexpected adverse effects are not the only type of complexity. The same phenomenon can be produced by many different causes. Hypertension, depression, cancers, and many other ailments have more than one cause, and most (if not all) medical interventions are far more complex than is generally assumed. Viagra therapy, for example, involves not only the pharmacological action of sildenafil, but also, among other things, the potentially relevant effects of tablet excipients (e.g. bulking agents), the liquid with which the tablet is swallowed, and the patient's beliefs associated with Viagra.^14,40,41 These further treatments can, and sometimes do, affect further relevant mechanisms.

The complexity of both individual mechanisms and of the interactions between the various mechanisms involved in any treatment makes reasoning from knowledge of what happens via (some of the) mechanisms under intervention to a prediction of a clinically relevant outcome highly uncertain.

To recap, mechanisms are never completely understood, and they are all potentially complex in unsuspected ways. But the problem with mechanistic reasoning must be contextualized. For one, incomplete knowledge is the norm in science: even the best randomized trial, for example, could have some potentially confounding difference between comparison groups. Moreover, it is unreasonable to assume that all mechanistic reasoning suffers from the problems described above to the same degree .

A proposal for redemption: mechanistic reasoning based on mechanisms without obvious gaps is useful evidence

We propose that whenever mechanistic reasoning is used, (a) the links in the mechanistic chain should be made explicit; and (b) the evidence for the links should be provided (Figure 4). The quality of mechanistic reasoning can be evaluated according to the extent to which it overcomes the problems listed above, and can, therefore, be rated according to the extent to which it satisfies the following two factors:

1. The knowledge of mechanisms upon which the mechanistic reasoning is based, is not incomplete. The input–output relationship for each ‘link’ in the mechanistic chain is evidence-based (for example, based on randomized trials);

2. The probabilistic and complex nature of the mechanisms are explicitly taken into account when inferring from the mechanism to any claims that an intervention has a patient-relevant benefit.

Figure 4.

Evidence-based mechanistic reasoning, without obvious missing links, supporting the effectiveness of an accelerated hepatitis B immunization regimen*

Examples of ‘high quality’ mechanistic reasoning

The following are examples of what we believe to be ‘high-quality’ mechanistic reasoning.

Example 1: Accelerated hepatitis B immunization

Conventional hepatitis B immunization schedules involve injections at 0, 1, and 6 months. This is inconvenient for travellers who have to go to a hepatitis B virus endemic area at short notice. An accelerated regimen of injections at 0, 10, and 21 days, has been studied in randomized trials and shown to result in the same high seroproduction rates as the regular regimen.^42,43 Similar seroproduction rates indicated that the accelerated regimen produced the same immunity as the longer regimen. The simplified mechanism here is: accelerated regimen → seroprotection → immunity (Figure 4). The trial provided strong evidence for immunization → seroprotection (with no short-term paradoxical or serious adverse events), while strong background evidence links seroprotection with immunity: it has been known for several decades that hepatitis B is caused by the hepatitis B virus (HBV),⁴⁴ that the antibodies in the vaccine neutralize the virus,^45–47 and that seroprotection is related to immunity.^48–50 In this example our knowledge of the action of vaccines permits us to infer that even if the therapy is delivered differently, the effect is likely to remain the same. In this case there are no missing links in the mechanistic chain, and no studies have revealed unexpected paradoxical or harmful effects (Figure 4).

Concerns might still arise, of course, about the duration of the seroprotection achieved with an accelerated regimen, and even about whether seroprotection is in fact the cause of immunity. However, in the absence of randomized trials linking the accelerated immunization regimen with immunity, it is reasonable to use the accelerated regimen for people who are travelling to a hepatitis B endemic area at short notice, based on the mechanistic evidence.

Example 2: Obstruction by a goitre

Large nodular goitres obstruct the airway and impair respiratory function. At the same time, there is strong evidence that radiotherapy shrinks a goitre (and does not cause any serious adverse effects),⁵¹ although radiotherapy can also cause short-term thyroid swelling.⁵² Mechanistic reasoning allows us to conclude that radiotherapy will improve respiratory function in the longer term. One might even question whether the trial that tested the hypothesis was justified, given the quality of the available mechanistic reasoning.

Example 3: Reminder packaging

There is evidence for a link between blood glucose concentration, blood pressure control, and the prevention of diabetic complications.^53,54 However, controlling the glucose concentration and blood pressure often results in complex therapy involving 10–15 tablets per day. Adherence to such therapy is inversely proportional to the complexity of the administration regimen⁵⁵ and special packaging can improve patient adherence.⁵⁶ If we assume that the medication itself causes the reduction in blood pressure, we can appeal to mechanistic reasoning to infer that special packaging will help prevent diabetic complications by increasing adherence. Indeed a randomized trial showed that calendar blister packs significantly improved glucose concentrations and blood pressure.⁵⁷ Although the direction of the effect was unsurprising, the size of the effect of calendar packaging could not have been predicted from mechanistic reasoning – that is not its purpose.

Using mechanistic reasoning to deny that a treatment has an effect

Low quality mechanistic reasoning is equally unreliable as evidence that an intervention is ineffective. The introduction of many useful treatments, such as antisepsis⁵⁸ and antibiotics for peptic ulcers⁵⁹ was delayed because of failure to consider mechanisms properly.⁴ Warren and Marshall, for example, were ridiculed for presenting empirical evidence that Helicobacter pylori caused peptic ulceration. Sceptics wrongly inferred (from low quality mechanistic reasoning) that bacteria could not live in the hostile environment of the stomach.⁵⁹

Conclusion

Our modest aims in this paper were to point out that not all mechanistic reasoning was created equal, and set out a preliminary set of standards that mechanistic reasoning can and should be held up to when used to support claims about treatment effects. Whereas comparative clinical studies have been held up to explicit standards,⁶⁰ according to the strength of evidential support they provide, all forms of mechanistic reasoning as evidence for efficacy have hitherto been lumped together and generally denigrated. Whenever mechanistic reasoning is used to justify a therapeutic intervention, the stages and chain of reasoning should be shown, accompanied by the evidence that supports each link in the chain (Figure 4). Mechanistic reasoning based on empty or partial mechanisms should be disregarded. Yet, our analysis and examples suggest that high quality mechanistic reasoning involving an evidentially justified chain without any obvious missing links and that takes potential complexities into account can and should be used to support hypotheses of therapeutic efficacy. Further research will determine how high quality mechanistic reasoning fits into current evidence ranking schemes.^60,61

Footnotes

DECLARATIONS —

Competing interests None declared

Funding None

Ethical approval Not applicable

Guarantor JH

Contributorship JH prepared the first manuscript based on his philosophical research on mechanisms. PG provided the examples of ‘high quality’ mechanistic reasoning. JKA introduced the idea of mechanistic ‘stages’ and provided the details of the particular medical mechanisms discussed in the paper. All three authors edited the manuscript