A Systematic Review of Immunosupressant Adherence Interventions In Transplant Recipients: Decoding the Streetlight Effect

Abstract

Nonadherence to immunosuppressant medications is an important risk factor for graft dysfunction. To evaluate the effectiveness of adherence-enhancing interventions, we reviewed adherence intervention studies in solid organ transplant recipients (all ages). Using the following databases: PsychINFO, PubMed, Scopus, and ScienceDirect, we identified 41 eligible studies. Only 3 nonrandomized trials showed a possible positive effect on objective indicators of transplant outcomes (such as rejection, liver enzyme levels, kidney function). None of the 21 randomized-controlled trials (RCTs) showed an improvement in transplant outcomes. Three studies showed a higher rate of adverse events in the intervention group as compared with controls, although this may be related to ascertainment bias. Improvement in adherence as measured indirectly (e.g. with electronic monitoring devices) was not aligned with effects on transplant outcomes. We conclude that adherence interventions, to date, have largely been ineffective in improving transplant outcomes. To improve this track record, intervention efforts may wish to concentrate on nonadherent patients (rather than use convenience sampling, which excludes many of the patients who need the intervention), use direct measures of adherence to guide the interventions, and employ strategies that are intensive and yet engaging enough to ensure that nonadherent patients are able to participate.

Introduction

A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost a coin. They both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost the coin here, and the drunkard replies, no, and that he lost it in the park. The policeman asks why he is searching here, and the drunkard replies, “This is where the light is.” ¹

This drunkard is clearly searching for the coin in the wrong place. “The streetlight effect”, or “the drunkard search”², describes a type of bias in clinical research that occurs when investigators who are searching for something look only where it is easiest to look for an effect (but not where it is important to look for that effect). “The streetlight effect” implies either or both of the following: the research is being conducted on the wrong set of patients (the researchers focus on a sample that is easiest to recruit but is not representative of the population that is supposed to be treated); or the intervention examines the wrong set of outcome measures (typically, this means studies that use “process” measures – measures that look at some aspect of the intervention’s purported mechanism of action but not at the actual outcomes). In our view, the “streetlight effect” is exceptionally relevant in adherence intervention research in transplant medicine; the following review illustrates that point.

Nonadherence to medications in children, adolescents, and adults who had a solid organ transplant has been identified as the most important risk factor for Late Acute Rejection (LAR) and other adverse outcomes^3–15. Nonadherence is quite prevalent, affecting about 15–40% of recipients.^16,17 The reported rate depends on the method used to assess adherence, and on the threshold assigned by investigators for the determination of whether a patient is adherent or not. Since methods are quite disparate, and there is no consensus on what constitutes the threshold beyond which a patient should be considered “nonadherent”, it is difficult to quantify the exact scope of the problem and compare adherence rates between different groups of organ transplant recipients.

Nevertheless, nonadherence is certainly a prevalent risk factor. Since it is considered by many to be modifiable³, professional societies recommend that adherence monitoring should become standard care¹⁸, and identifying empirically-driven ways to improve adherence in transplant care is a priority.^18–20 Therefore, researchers have been engaged in trying to measure, monitor, and improve adherence in transplant settings for at least 2 decades, but largely without any proof that those efforts can in fact improve transplant outcomes. Of course, nonadherence is not just an issue in transplant medicine; comprehensive reviews show that adherence interventions seldom lead to improved medical outcomes²¹. We believe that those disappointing results are mainly due to the consequences of the “Streetlight Effect”: it is very hard to engage nonadherent patients in research (in the same way that it is hard to search for the coin in the dark park, where it was lost). As a result, these patients are systematically excluded, and research is conducted on a biased sample of patients who are adherent and, therefore, do not really need the intervention²² (searching under the streetlight is the wrong place to look for the coin). In addition, process outcomes (adherence behavior) rather than medical outcomes are often used as the primary endpoint, which may lead to conclusions that the intervention was successful even if there was no improvement in clinical outcomes (e.g., rejection, graft loss).

Since much has been done already in this area in transplant medicine, we believe that a comprehensive review of intervention efforts is indicated and could inform the next round of clinical and research efforts. Below, we describe and then discuss the results of a systematic review of adherence intervention research. Our primary aim was to evaluate the extent to which published interventions that attempted to improve adherence in solid organ transplant recipients have been able to demonstrate a statistically significant effect on transplant (medical) outcomes. We define “transplant outcomes” as robust, objective evidence of graft function (or dysfunction), including, for example, rejection, organ loss (retransplantation, death), liver enzyme levels (as evidence of graft function of liver transplant recipients), glomerular filtration rate, GFR (renal transplant function), and Forced Expiratory Volume in one second, and FEV₁ (lung transplant function). As a secondary objective, we evaluated the extent to which those interventions reported an improvement in adherence behavior (and other outcomes), and whether such improvements were or were not associated with changes in transplant outcomes. Adherence is worse in adolescents²³, but since many intervention studies involved adults, our comprehensive review included all ages. In our synthesis, we examine the potential role of the “streetlight effect” in the reviewed articles, and present our ideas on how to take the “drunkard’s search” back to the park, as a potential way to improve our track-record in research and clinical practice.

Methods

A comprehensive review is, in our opinion, superior to efforts to meta-analyze data at this point, since adherence measurement schemes vary widely limiting meta-analytic approaches. We therefore conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines²⁴. The procedures that were followed and the review protocol are described below and outlined in Figure 1.

FIGURE 1.

PRISMA statement diagram

Eligibility criteria

Since immunosuppressant adherence is the most common form of adherence targeted in adherence interventions in transplant recipients, and is the only form of adherence that has conclusively been shown to be related to poor outcomes, we reviewed published reports of interventions to address solid organ transplant patients’ nonadherence to immunosuppressant medications. Study designs included in the review were randomized controlled trials (RCTs), controlled trials (not randomized), cohort studies, observational cohort studies, pre-post comparisons, and retrospective chart reviews. Participants were humans of any age that had received a solid organ transplant; animal studies were excluded. Eligible interventions were those that directly or indirectly aimed to improve patients’ adherence to their immunosuppressant medication regimens and include interventions delivered in-person or remotely. Interventions targeting adherence to other medical recommendations that did not include at least a component targeting immunosuppressant adherence (for example, adherence to a spirometry monitoring program for lung transplant patients) were excluded. Articles that were peer-reviewed and published in English were eligible for inclusion.

Information sources

Four electronic databases were used to identify relevant articles published before May 2017: PsycINFO, PubMed, ScienceDirect, and SCOPUS. Articles were identified through the database search, as well as an additional scrutiny of articles that reviewed medication adherence interventions in the past and their reference sections. We used only information provided in the published articles and did not use any other external sources (i.e., we did not look at grant proposals or registration materials and did not contact the authors for details that were not presented in published manuscripts).

Search

Sample electronic database search strategy: PubMed title/abstract: “Adherence OR compliance” AND “transplant*” AND “interven*.” Filters activated: Humans, English.

Study selection

One author (SD) screened article titles produced by the search strategy described above. First, article titles were reviewed, and abstracts were then examined for articles that did not meet the exclusion criteria based on the title. Full-text articles were reviewed for studies that appeared to meet eligibility criteria based on the abstract. A second author (ES) independently reviewed the selected studies to confirm eligibility.

Data collection process

One author (ES) extracted the data and classified it using a standardized form. A second author (CD) independently reviewed the classifications using the same source materials; disagreements were discussed between the two authors and adjudicated by a consensus.

Data items

The following data were collected from each article: sample size, demographic data, study design, intervention description, primary and (if applicable) secondary measures of adherence, primary outcome, secondary outcomes, and transplant outcomes. We defined “transplant outcomes” as outcomes that directly evaluated the function or physiology of the transplanted organ, such as (depending on the transplant) rejection, organ loss (including death), liver function tests, lung or cardiac function tests, and glomerular filtration rate (GFR), FEV₁. Admission rates, quality of life, cost assessments, etc. all may be important outcomes, but were not considered transplant outcomes for the purposes of the present review.

Classification of outcomes

When evaluating study outcomes, we used the authors’ definitions of adherence or transplant-related endpoints, whenever stated, and decided, based on the reported results, whether those endpoints were met or not. Whenever possible, we examined differences between control and intervention arms. We looked at the data as reported rather than at the interpretations of the data; in some cases, our classification was different from the authors’. For example, a non-significant difference in an outcome measure between an intervention and a control group would be classified by us as “no difference”; this may have been stated as a “non-significant trend” or “non-significant improvement” in the article. As another example of how we rated transplant outcomes, in one study, rejection rates improved nonsignificantly and liver enzyme levels improved significantly; since no specific transplant outcome was defined a-priori as primary, we rated this outcome as “unclear” although the authors contended that transplant outcomes “nonsignificantly improved” in that study. Additionally, we classified a significant improvement or worsening, as identified by the study’s primary analytic technique, as “improvement” or “worsening” even if another analysis had been added, post-hoc, that found different results.

Assessment of bias

Sources of bias in our approach could include: a publication bias (in which positive findings are more likely to be published than negative ones), and selective reporting within studies. We were unable to access any database (and we are not aware that one exists) that maintains unpublished study results. As such, our results are subject to publication bias. Presumably, published results (that are available to us) are more positive than real-world data. Our approach mitigated, in part, the issue of selective reporting in that we formed our own interpretation of the data, using all available information, rather than rely on authors’ preferred presentation.

Results

Table 1 summarizes the search. We identified 41 intervention studies^25–65, of which 29 were controlled, and 21 of those were RCT’s^45–65. The specified databases enhanced and by a scrutiny of published review manuscripts^66–71 were used to identify these studies. Studies were conducted in liver (n=8), kidney (n=25), heart or heart/lung (n=2), and lung transplant recipients (n=4), as well as mixed groups (n=2). Of the RCTs, 11 did not report on transplant outcomes,^{45–54, 57} and 10 examined them and found no effect of the intervention.^{55,56,58–65}. Table 2 is a detailed review of the 10 RCTs that looked at transplant outcomes (and found no intervention effect), presenting other outcomes that were examined in those studies.

TABLE 1.

Studies examining interventions to address medication adherence in transplant recipients

Ref #1	Age2	Tx3	N4	Design5	Recruitment strategy6	Intervention7	Primary adherence measure8	Adherence improved?9	Blood levels examined? If yes – improved?10	Transplant outcomes examined?	Transplant outcomes improved?
25	P	LI	41	PRE-POST	CON	TEXTS	MLVI	YES	EXAMINED, IMPROVED	YES	YES
26	P	LI	20	PRE-POST	CON	ED	MLVI	YES	EXAMINED, IMPROVED	YES	YES
27	P	LI	23	PRE-POST	CON/NA	ED-BEH	MLVI	NO	NOT IMPROVED	YES	UNCLEAR
28	A	K	36	PRE-POST	CON	PHARM CARE	LEVELS	YES	EXAMINED, IMPROVED	YES	YES
29	P	K	22	CO	CON	ED/TC	MLVI	NO	NOT IMPROVED	YES	NO
30	P	LI	22	CO	CON	PM	MLVI	NO	NOT IMPROVED	NO	UNKNOWN
31	P	LI	34	CO	CON	TC	MLVI	YES	EXAMINED, IMPROVED	NO	UNKNOWN
32	P	K	29	CO	CON	ED-BEH	PILL COUNTS	UNCLEAR	NOT IMPROVED	NO	UNKNOWN
33	A	K	18	PRE-POST/RCT	ONA	ED-BEH	EM	NO	NOT EXAMINED	NO	UNKNOWN
34	A	H	20	PRE-POST	CON	ED/INTERNET	Self-report	NO	NOT EXAMINED	NO	UNKNOWN
35	A	K	18	OBS	COHORT	FREE MEDICATION	RR	NO	NOT IMPROVED	NO	UNKNOWN
36	P	K	21	PRE- POST	CON	PHARM COUNSELING	PILL COUNTS	NO	NOT EXAMINED	NO	UNKNOWN
37	A	M	97	OBS	CON	TSP	WRBL	Unclear	EXAMINED, UNCLEAR	NO	UNKNOWN
38	P	LI	45	OBS	CON	COUNSELING	MULTIPLE	YES	EXAMINED, UNCLEAR	YES	NO
39	A	K	519	RETRO	RETRO	TSP	MPR	YES	NOT EXAMINED	YES	NO
40	A	K	290	CO	COHORT	TSP	MPR	UNCLEAR	NOT EXAMINED	YES	NO
41	P	K	50	CO	CON	ED	SELF REPORT	UNCLEAR	NOT EXAMINED	YES	NO
42	A	K	59	CO	CON	ED	LEVELS	NO	NOT IMPROVED	NO	UNKNOWN
43	A	H, LU	36	CO	CON	SMAP	SELF REPORT	NO	NOT EXAMINED	NO	UNKNOWN
44	A	LI	119	PRE-POST	CON	ODM	SELF REPORT	YES	EXAMINED, UNCLEAR	YES	NO, SAE***
45	A	K	219	RCT	CON	ODM	EM	NO	NOT EXAMINED	NO	UNKNOWN
46	A	K	20	RCT	CON/NA	mHEALTH REMINDERS	EM	YES	NOT EXAMINED	NO	UNKNOWN
47	A	K	18	RCT	CON/NA	mHEALTH REMINDERS	None	?	NOT EXAMINED	NO	UNKNOWN
48	A	K	33	RCT	CON	CBT	Self-report*	YES	NOT IMPROVED	NO	UNKNOWN
49	A	K	13	RCT	ONA	NURSE SESSIONS	EM	YES	NOT EXAMINED	NO	UNKNOWN
50	A	LU	30	RCT	CON	POCKET-PATH	Self-report	YES	NOT EXAMINED	NO	UNKNOWN
51	A	K	159	RCT	CON	ED	DIARY	YES	NOT EXAMINED	NO	UNKNOWN
52	A	K	120	RCT	CON	REMINDERS/NOTIFY	EM	YES	NOT IMPROVED	NO	UNKNOWN
53	A	K	24	RCT	CON	PHARM COUNSELING	RR	YES	EXAMINED, IMPROVED	NO	UNKNOWN
54	A	K	48	RCT	CON	E-FEEDBACK	EM	NO	NOT EXAMINED	NO	UNKNOWN
55	A	K	111	RCT	CON	COUNSELING	Self-report	YES	NOT IMPROVED	YES	NO
56	A	K	67	RCT	CON	PHARM COUNSELING	EM	YES	NOT EXAMINED	YES	NO
57	A	K	150	RCT	CON	PHARM CONTRACT	MPR	YES	NOT EXAMINED	NO	UNKNOWN
58	A	LI	50	RCT	CON	PHARM CARE	EM	YES	EXAMINED**	YES	NO
59	A	M	205	RCT	CON	BEH/REMINDERS	EM	YES	NOT EXAMINED	YES	NO
60	A	LU	201	RCT	CON	POCKET - PATH	Self -report/collateral	YES	NOT EXAMINED	YES	NO
61	A	LU	182	RCT	CON	POCKET-PATH	Not examined		NOT EXAMINED	YES	NO
62	A	K	46	RCT	CON	TSCM	CAS	YES	EXAMINED, UNCLEAR	YES	NO
63	A	K	110	RCT	CON	PSYCHOED	Self-report	YES	NOT EXAMINED	YES	NO – WORSE
64	A	LU	64	RCT	CON	TABLET EDUCATION	LEVELS	NO	NOT IMPROVED	YES	NO, SAE***
65	A	K	80	RCT	CON	AUTOMATED REMINDERS	EM	UNCLEAR	NOT IMPROVED	YES	NO, SAE***

ITALICS = Improved transplant outcomes.

BOLD = Randomized Controlled Trial

¹Ref #: Reference number

²Age: P = Pediatric, adolescent, and/or young adult. A = Adults only

³Tx: Transplant type: K = Kidney; LI = Liver; LU = Lung; H = Heart; M = Mixed

⁴N: Number of subjects in primary analysis, all arms

⁵Design: RCT = Randomized Controlled Trial; PRE-POST = cohort study, pre-post comparisons; OBS = observational cohort study; CO = controlled, not randomized; RETRO = Retrospective chart review, matched controls

⁶Recruitment strategy: CON = Convenience Sample; CON/NA = Convenience sampling, intervention given to nonadherent patients primarily; ONA = Only nonadherent patients received the intervention

⁷Intervention: CBT = Cognitive Behavioral Therapy; PSYCHOED = Psychoeducation; PHARM COUNSELING = Counseling by a pharmacist; TABLET EDUCATION = Education delivered via an electronic tablet (compared with standard education); TC = Transition Coordinator; POCKET-PATH = Pocket Personal Assistant for Tracking Health; ED = Education; BEH = Behavioral Intervention; TSP = Transplantation Specialty Pharmacy services; ODM = Once Daily Medication; mHEALTH = Mobile Health; TC = Transition Coordinator; SMAP = Self Medication Administration Program; TSCM = Telemedically Supported Case Management; PM = Peer Mentorship

⁸Primary Adherence Measure: MLVI = Medication Level Variability Index; EM = Electronic Monitoring; MPR = Medication Possession Ratio; RR = Refill Rate; WRBL = Within-range Blood Levels; CAS = Composite Assessment Score

⁹As measured by the primary method used to assess adherence

^*Pill counts done by patients and reported to the investigators via the phone

^**Blood levels classified into “within range” and “out of range”, but that classification does not seem to have been individualized or reflective of the ranges used for clinical decisions

^***Serious adverse events (death, organ loss, cerebrovascular accidents), which were not part of the primary outcome measure, were reported in intervention group but not in controls

TABLE 2.

Randomized controlled trials examining effects of medication adherence interventions on transplant outcomes

Ref#1	Tx2	N3	INTERVENTION4	ADHERENCE ASSESSMENT (D=Direct/I=Indirect). Improved?	Transplant Outcome Measures (No improvement)5	Other examined measures/Improved?
55	K	111	EDUCATION/counseling: 10 sessions, weekly, 30 minutes each.	Self-report (I), improved.	Renal Function (creatinine clearance/mean creatinine concentration), Acute Rejection, Graft Loss, Death.	Mean Serum Medication Blood Levels, not improved.
56	K	67	PHARM COUNSELING: at least 3 face-to-face sessions with a pharmacist.	EM* (I), improved.	GFR, Rejection.	Self-reported adherence, not improved. HRQOL**, not improved. Anxiety and depression score, not improved.
58	LI	50	PHARM CARE: 3–4 sessions with a pharmacist in-house, and quarterly or more thereafter.	EM* (I), improved.	Rejection.	Pill counts, improved. Self-reported adherence, not improved. Within-range blood levels, improved (but definition unclear).
59	M	205	BEH/REMINDERS: Behavioral change treatment (5 face-to-face sessions).	EM* (I), improved.	Composite of death and retransplantation.
60	LU	201	POCKET – PATH: a mobile device providing reminders and education, and monitoring behavior.	Self-report/collateral (I), improved.	Survival.	Self-monitoring (per device), improved. Self-care agency, not improved. Rehospitalization, not improved. Hospital stay, not improved.
61	LU	182	POCKET-PATH: a mobile device providing reminders and education, and monitoring behavior.	Not examined.	Long term mortality, Bronchiolitis Obliterans.
62	K	46	TSCM: Telemonitoring plus case management.	MIXED – Patient report, provider report, and blood levels, improved.	Rejection, GFR.	Unplanned admission rate, improved. Length of stay in unplanned admissions, improved. Ambulatory visits, no improvement. QOL, several subscales improved.
63	K	110	PSYCHOED: 8 weekly sessions.	Self-report (I), improved.	Allograft survival – no effect initially, and at 10 years, intervention worse than controls.
64	LU	64	TABLET EDUCATION	MEDICATION LEVELS (D), not improved.	GFR	Two deaths in intervention group, none in controls. Knowledge about transplant, self-reported and physician reported adherence, none improved.
65	K	80	AUTOMATED REMINDERS via electronic medication dispenser.	EM* (I), improved.	Creatinine, Rejection.	6 patients in the intervention group withdrew from the study, none from the controls. One death and one stroke in the intervention group.

¹Reference number

²Transplant type: K = Kidney, LI = Liver, M=Mixed, LU = Lung or Heart and Lung

³N = number of enrolled patients

⁴Intervention type: PSYCHOED = Psychoeducation; PHARM COUNSELING = Counseling by a pharmacist; TABLET EDUCATION = Education delivered via an electronic tablet (compared with standard education); POCKET-PATH = Pocket Personal Assistant for Tracking Health; ED = Education; BEH = Behavioral Intervention; TSCM = Telemedically Supported Case Management

⁵Transplant outcomes that were measured (but did not improve) in those studies. GFR = Glomerular Filtration Rate.

^*EM: Electronic Monitor

^**HRQOL: Health Related Quality of Life

Only 3 very small, nonrandomized studies^25,26,28 – 2 of which were in pediatric populations – found an improvement in transplant outcomes (rejection rates). Another pediatric intervention study²⁷ was inconclusive (liver enzyme levels improved but rejection rates did not). More important, perhaps, is our finding that of the 21 RCTs that we reviewed, none demonstrated improvement in transplant outcomes: 11 did not measure those outcomes, 7 measured such outcomes (GFR, rejections, graft loss, death) and found no intervention effects, one study showed significantly more organ loss in the intervention group, and two studies reported serious adverse events (death, cerebrovascular accidents) in the intervention but not in the control arm. Studies reported a range of other secondary outcomes that might have improved (such as quality of life, hospitalization rates, service utilization) but those outcomes (Table 2) were not pre-defined as single hypotheses, suggesting that some of those putative effects might have been spurious results.

Discussion

Strategies to monitor and improve adherence are frequently recommended as a part of post-transplant care¹⁸, which is common sense, but our results do not support those recommendations. Through this review, we identified 41 published intervention studies. Of note, previous systematic reviews^68,71 identified only a dozen or so articles, probably reflecting the substantial work that has been done in this field relatively recently, as well as (possibly) less restrictive criteria in our search, and our ability to look at multiple databases.

Our conclusion is that adherence interventions, as currently conceived, are ineffective inasmuch as transplant outcomes are concerned, and may even be associated with negative outcomes. One RCT found a significantly higher rate of death in the intervention arm⁶³, and 2 other RCTs reported serious adverse events in the intervention but not in the control arm^64,65 (those adverse events ranged from a cerebrovascular accident and death, to milder events). We caution, however, that the increased rate of reported adverse events in the intervention groups might be an ascertainment bias: it might be related to the fact that intervention group patients were under better surveillance (and so, were more likely to report such events to the research group). In addition, those events were uncommon and could be a chance finding, in a population that is already prone to adversity. Hence, we do not conclude that interventions were harmful. A better interpretation of the data, in our view, is simply that interventions were not beneficial. The findings do suggest that future intervention research in this area should pay attention to the definition and monitoring of adverse events in this population, both in the intervention and in the control arms. In concluding that interventions did not lead to improved outcomes, transplant settings seem to have a similarly disappointing record as compared with adherence interventions in other fields of medicine. For example, a recent analysis²¹ concluded that, “among 57 studies measuring clinical outcomes, only 8 reported a significant improvement in clinical outcome.”

Characteristics Of Effective Interventions

The few intervention studies that did show improved transplant outcomes^25,26,28 had the following common characteristics: they used a direct measure of adherence, and most (2 out of 3, plus the one inconclusive study) were conducted on children and adolescents. It may be easier to engage a nonadherent child or adolescent whose parents may provide support, as compared with adults who do not have such a built-in support system. In some pediatric cases, the parents are the ones responsible for the child’s adherence, and this might make the parents the preferred (and more responsive) intervention target. Given those insights, it appears that one target of adherence interventions could be adolescents – both because poor outcome ( likely in part related to nonadherence) is a major problem in that group, and also because this seems to be a promising time window in which to attempt an intervention. We do not, however, suggest that adults should not be enrolled in intervention efforts. The very small sample sizes and nonrandomized, weak study design of those studies prohibit us from offering any specific recommendation on the targeting of specific populations.

RCT Results

None of the RCT’s demonstrated improved transplant outcomes, but we would highlight that authors generally stated that their intervention was successful even when it did not lead to improvements in transplant function or survival. Authors cited three reasons for this optimistic view: that the study was not powered to show a difference in transplant outcomes, that other outcomes seemed to have improved, and that measured adherence seemed to have improved (although transplant outcomes did not). We address those attempts to explain the lack of intervention effect on transplant outcomes below.

Power

The contention that sample sizes were insufficient to show a difference in transplant outcomes is not consistent with our finding that the studies that did show an improvement in those outcomes^25,26,28 were amongst the smallest studies in this review. Sample size, therefore, did not appear to be the source of mixed findings. With regards to the frequency of the measure used to determine transplant outcomes (another consideration for power determinations): while death or rejection could be infrequent events, one could evaluate GFR or liver enzymes which are continuous variables (when those were evaluated, they did not improve, except in one non-randomized small pediatric study²⁷). We believe that the correct interpretation of the observation that RCTs consistently and repeatedly found no intervention effects on transplant outcomes is simple: interventions that did not show an effect on transplant outcomes were not effective in improving such outcomes, either because the intervention was not effective or because it was not applied to the correct group of patients. We suggest that future studies evaluate the intervention effect on transplant outcomes as a primary aim and that this aim be used as the basis for power calculations before the study commences.

Masking, and a Placebo Effect

We have reservations about the presentation of results as “promising” based on improvements in outcomes other than transplant outcomes. An important issue here is that of masking; a double blinded, placebo controlled trial is not feasible for the type of interventions that are considered here, and authors indeed generally acknowledged a lack of masking as a limitation. This means that a placebo effect is an important consideration in those studies. Therefore, patient (or caretaker) self-reports are very likely to be skewed by patient (or the caretaker) knowledge of the study allocation. Hence, self-reported adherence or other self-reported constructs such as quality-of-life questionnaires would be expected to show a placebo effect that might be misconstrued as proof that the intervention was helpful. Constructs such as admission rates could be affected, as well, by the fact that physicians know whether the patient is in the “intervention” or “control” arms (this is the reason that “gold standard” medication trials are generally double-blinded and not single-blinded). For example: a clinician who knows that a patient is enrolled in an intensive case-management paradigm might be less likely to admit this patient because of the clinicians’ belief that the case-management intervention will help stabilize the patient without a need for admission. While it is indeed impossible to mask the intervention, one way to mitigate the placebo effect is by blinding the assessors (a routine procedure in behavior intervention research^72,73), and by choosing robust biological outcome measures. An example of such a robust outcome measure tied to a masked assessor is a masked pathology reading of a biopsy result (by a central pathology lab that does not know the patient or the treatment allocation) as the standard for determination of a rejection event²³. Biological outcomes such as liver function tests or GFR could also be used. The fact that all the RCTs that we reviewed consistently showed no intervention effects on robust biological markers of transplant function (such as rejection, liver function tests, GFR), while occasionally showing putative effects when self-reported measures or unmasked clinical decisions were involved, suggests that observed intervention effects on unmasked constructs could have been a placebo effect.

Adherence Improves While Transplant Outcomes Do Not: the Importance of Assessment Methods and Selection Bias

Another recurrent, and surprising, finding (that was frequently cited as evidence that the intervention worked) is that many interventions showed a statistically significant improvement in adherence behavior, as measured in those studies. Those improvements, however, did not lead to any effect on transplant outcomes as discussed above (Table 2). This lack of correlation between adherence improvement (as measured by medication refill rates, self-reports, or, most frequently, electronic monitors) and transplant outcomes deserves a closer look. Why would a significant, and at times substantial, increase in measured adherence not be associated with any benefit to organ function or survival?

There are two observations we would like to make. First, this “uncoupling” is true for studies that used indirect adherence measures^{38–40, 56, 58–60, 62, 63, 65} (many used electronic monitors^{56, 58, 59, 65}), but was not true in studies that used direct measures that relied on medication blood levels^{25–28, 55, 64, 65}. As shown in Table 1, when medication blood levels (or, the degree of fluctuation between levels) improved^25,26,28, transplant outcomes improved as well; when levels did not improve^27,55,64,65, transplant outcomes did not improve, either. We believe that this suggests that direct measures of adherence are better measures of medication intake than indirect measures (indeed, by definition, indirect measures do not actually measure intake).

Second, most of the studies used convenience sampling, which may lead to heavily skewed samples, consisting primarily of very adherent patients. In such circumstances, improvement in adherence (from excellent adherence at baseline to even better than excellent at the end of the study) may be clinically irrelevant. For example, an improvement in indirectly measured adherence from 83% to 89% may be statistically significant⁵⁷, but is not likely to result in clinical benefits. It is rarely possible to examine the extent of the (inadvertent) selection bias that results from convenience sampling in adherence research, as researchers do not usually know the adherence level of those who are not enrolled. In one notable instance, however, the average rate of adherence was examined prior to the intervention study. The results were startling: the same group estimated that the average rate of adherence in the clinic was 54%⁷⁴, whereas the average rate of adherence at baseline in their intervention study⁵⁷ was 83%.

Stated differently, it is hard to recruit nonadherent patients to clinical trials; it is much easier to recruit, and follow, adherent patients. Studies that primarily enroll very adherent patients can improve adherence (from very good to excellent) relatively easily because such patients are able to adhere to study procedures. But, those studies may not demonstrate improved transplant outcomes because the patients are already sufficiently stable before the intervention even takes place. In the same way, it is much easier to search for the coin under the streetlight (not where it was lost), but it will never be found there.

Recommendations

Over the last several decades and more, a substantial amount of evidence has been accumulated on adherence interventions; however, the preponderance of research unfortunately has failed to demonstrate, again and again, that adherence interventions improve transplant outcomes. We should pay attention to those findings rather than dismiss them: a paradigm shift in the way adherence interventions are studied is essential. Using the “drunkard’s search” framework, we suggest the following:

1. Move the search back into the park (target nonadherent patients, avoid selection bias), where there is an actual chance to find the “coin” (improve transplant outcomes). In order to mitigate selection bias, we propose the following strategies:

   1. Design studies that will specifically try to prevent the inadvertent exclusion of nonadherent patients. For example: reduce patient burden by using non-intrusive monitoring strategies rather than electronic monitors, reduce the number of questionnaires, assessments, or intervention sessions. Enhance convenience sampling by insisting that all patients in a clinic will be approached, or engage in probabilistic sampling methods. Alternatively - target only nonadherent patients, who are identified via a robust measure of adherence, for intervention.

   2. Always try to evaluate the possibility of selection bias in the final cohort: to quantify the degree of bias, investigators should try to compare between patients who do or do not participate in the research, whenever possible.

2. Because of the “streetlight effect”, clinicians, researchers, funding agencies, patients, and journal editors and reviewers should insist that “finding the coin” (improving transplant outcomes) is ultimately the important outcome measure. Transplant outcomes (for example: GFR, liver function tests, pulmonary function, rejection, graft loss, death) should always be examined; if adherence “improves” but transplant outcomes do not, it should be presented as a negative study (the intervention did not work). It has been shown consistently that an “improvement” in measured adherence does not necessarily lead to any clinical benefits. Therefore, an improvement in measured adherence should not be considered a successful outcome in and of itself, in the same way that if the coin is not found, the search should not be considered successful. In addition, if other outcomes, which were not pre-defined as primary, improve, this should not be presented as proof that the intervention was successful (in the same way: if the drunkard finds a dollar bill under the light, this should not be considered a successful search for the coin which he lost in the park).

3. A flashlight should be used in the park. The drunkard should not use the same method of search on the lighted sidewalk as compared with the dark park. Similarly, we must not assume that interventions deemed “effective” among a skewed population of very adherent patients would also improve adherence (and outcomes) in the more important population of nonadherent patients. We should also not assume that such interventions will improve clinical transplant outcomes, even if larger samples (of very adherent patients) are engaged. For example: intervention strategies that require excessive adherence to elaborate procedures (eg, attending multiple face-to-face meetings³³, consistently using an electronic device that beeps whenever a cap is not opened,⁶⁵) are likely to be followed only by patients who are very adherent anyway. Using such strategies is likely to result in exclusion of the nonadherent patients that need the intervention the most, but cannot adhere to its requirements⁶⁵. Intervention strategies that are likely to improve transplant outcomes in nonadherent patients probably have to be more comprehensive than simple reminders or automated strategies, yet also unobtrusive enough to avoid posing an excessive burden on patients. A promising example involves the use of intensive case management delivered via a telemedicine paradigm⁶².

Limitations

This review is limited, by design, by its inclusion criteria (for example: we only reviewed English-language literature). Another limitation is that we could not review ancillary data sources (because they do not exist), that might have shed more light on transplant outcomes that may have been evaluated in studies that did not report such outcomes. It is possible that in some instances, transplant outcomes may have been obtained by researchers but not reported, resulting in a reporting bias that we are unable to mitigate.

Conclusions

Nonadherence to immunosupressants, an important risk factor for poor long-term transplant outcomes, may be modifiable- if only we were able to monitor adherence accurately and intervene successfully. The importance of modifying this behavior is critical and the concerns presented here should not be misconstrued to lessen the importance of this goal. To date, RCT’s show either no effect on transplant outcomes, or increased risk. Therefore, at this point, we cannot recommend any specific adherence monitoring or intervention strategy in clinical transplant settings. Adherence intervention research must make significant adjustments to avoid the “Streetlight Effect”: we believe that our only chance to “find the coin” (improve clinical outcomes) involves accepting a paradigm shift. We must “move the search back to the park” (recruit nonadherent patients into our intervention efforts), and “use a flashlight to guide the search” (use direct measures of adherence, and intervene in ways that would ensure engagement of seriously nonadherent patients rather than lead to their exclusion).