Measuring the multiple domains of medication nonadherence: findings from a Delphi survey of adherence experts

Medication adherence is comprised of multiple behaviors. Our findings will help researchers choose the measurement approach best suited to measuring each of these distinct behaviors.

Abstract

Consensus on a gold-standard measure of patient medication nonadherence has been elusive, in part because medication nonadherence involves multiple, distinct behaviors across three phases (initiation, implementation, and persistence). To assess these behaviors, multiple measurement approaches may be needed. The purpose of this study was to identify expert-recommended approaches to measuring nonadherence behaviors. Thirty medication nonadherence experts were e-mailed two consecutive surveys. In both, respondents rated their agreement with definitions of nonadherence behaviors and measurement approaches. In the second survey, respondents rated the suitability of each measurement approach for assessing each behavior and identified the optimal measurement approach for each behavior. Consensus was achieved for eight patient medication nonadherence behaviors: not filling initial prescription and not taking first dose (noninitiation); refilling prescription late, missing doses, taking extra doses, taking doses at wrong time, and improperly administering medication (incorrect implementation); and discontinuing medication early (nonpersistence). Consensus was achieved for seven measurement approaches: self-report, prescription fill data, pill count, drug levels, electronic drug monitoring (EDM), smart technology, and direct observation. Self-report questionnaires were most commonly rated “at least somewhat suitable” for measuring behaviors. EDM was rated as optimal for measuring missing doses, taking extra doses, and taking doses at the wrong time. Prescription fill data were rated as optimal for not filling initial prescription, refilling late, and discontinuing. Direct observation was rated as optimal for measuring improper administration. Suitable and optimal measurement approaches varied across nonadherence behaviors. Researchers should select the measurement approach best suited to assessing the behavior(s) targeted in their research.

Implications.

• Medication nonadherence is comprised of multiple behaviors that occur across three phases of medication-taking: noninitiation, incorrect implementation, and nonpersistence with prescribed medication.

• The suitability of various measurement approaches varies across these different nonadherence behaviors.

• Behavior-change scientists should select measurement approach(es) that are best suited to measuring the medication nonadherence behavior most relevant to their research.

INTRODUCTION

Patient medication nonadherence refers to the extent to which a person’s medication-taking behaviors correspond to agreed upon recommendations by a health care provider [1,2]. Despite growing recognition of medication nonadherence in the medical literature [3], the prevalence of nonadherence has remained persistently high [4–6]. Medication nonadherence is associated with worse health outcomes across a number of conditions [7–9]. For example, DiMatteo estimated that there was a 26% difference in health outcomes between high and low adherers [10]. Medication nonadherence also contributes to health care costs, accounting for nearly 300 billion dollars in the USA, alone [11,12].

Because of its substantial impact on health care costs and outcomes, patient medication nonadherence has become an important target for behavioral interventions. Systematic reviews have repeatedly concluded that existing nonadherence interventions have, at best, a modest impact on nonadherence and related health outcomes [11,13]. Furthermore, those few interventions that have substantially affected nonadherence are mostly complex and challenging to implement. Thus, there remains a pressing need to increase our knowledge of the mechanisms underlying medication nonadherence as well as to develop potent, scalable adherence interventions [14].

A key to advancing knowledge about medication nonadherence and to developing successful interventions lies in having reliable and accurate measures of nonadherence. Yet, a review of the existing literature reveals that there is a lack of agreement on how to best measure medication nonadherence. Self-report measures are viewed as being susceptible to reporting biases, and objective measurement approaches each have their limitations in terms of feasibility and accuracy [15]. This has led some experts to recommend obtaining multiple measures of nonadherence in behavioral research [16]. Yet, this recommendation has come with little guidance on how best to combine nonadherence measures [17]. The only consensus seems to be that there is no gold-standard measurement approach [18].

In recent years, there has been growing recognition that medication nonadherence represents not one, but multiple distinct patient behaviors [5,19]. The European Society of Patient Adherence, Compliance, and Persistence (ESPACOMP) has classified medication-taking into three broad phases: initiation, implementation, and persistence [20]. With this conceptual framework in place, one might hypothesize that rather than searching for a single best measure of medication nonadherence, different measurement approaches may be needed for measuring these different aspects of medication-taking behavior. For example, if one wanted to capture the extent to which patients were initiating a new medication, one might use a different method than one would use to assess their day-to-day implementation of the medication regimen [21]. To the best of our knowledge, consensus for this multidimensional approach to nonadherence measurement has not yet been assessed.

The goal of this study was to identify a set of expert recommendations that could inform researchers’ selection of approaches to measuring medication nonadherence. The premise was that the best approach would involve mapping measurement approaches to specific nonadherence behaviors, with different approaches being better suited to measuring different nonadherence behaviors. Consensus for recommended methods was sought by surveying experts in medication nonadherence using a Delphi approach.

METHODS

Overview of the study design

A reactive Delphi approach [22,23] was used to achieve consensus among experts in medication nonadherence on identifying distinct medication nonadherence behaviors and their definitions; identifying the range of measurement approaches available for measuring these distinct behaviors; and on determining the suitable and optimal measurement approaches for measuring each behavior. Consistent with a reactive Delphi technique, the initial list of survey items was generated based upon the study authors’ expertise and review of the literature. A purposive sample of adherence experts was surveyed to arrive at recommendations. Exploration of consensus occurred across two rounds of surveys. The institutional review board of Columbia University Irving Medical Center approved the study protocol.

Generation of the research question

The research question emerged from a needs assessment conducted by the Resource and Coordinating Center of the Science of Behavior Change (SOBC) program [14]. The SOBC program seeks to promote basic research on the initiation, personalization, and maintenance of healthy behaviors. Investigators within this program are applying a mechanism-based, experimental medicine approach to developing interventions that address multiple health behaviors, including medication nonadherence. The program highlights nonadherence to medications as a key target for behavior change research. Appropriate approaches to measuring medication nonadherence are essential to behavior-change research. This prompted members of the SOBC consortium to ask: which medication nonadherence measurement approach should I use in my research?

Generation of the initial framework

The initial framework for identifying nonadherence measurement approaches was developed by the study authors, who together represented the disciplines of internal medicine, health behavior, pharmaceutical health services research, and social psychology. The study authors also have experience using and evaluating the reliability and validity of a range of medication nonadherence measurement approaches (e.g., self-report, prescription refill records, and electronic drug monitoring) [6,24–34]. To develop the initial framework, the study authors used as a starting point the Ascertaining Barriers to Compliance (ABC) taxonomy provided by Vrijens and colleagues on behalf of ESPACOMP for categorizing nonadherence behaviors within a framework [20]. According to this taxonomy, nonadherence can be divided into three distinct phases: noninitiation, incorrect implementation, and nonpersistence with the medication. Based on their expertise in the field, the study authors then each individually generated subcategories of nonadherence behaviors within the phases of this framework. They then came together to achieve consensus on categories and to generate descriptions of each category. A similar process was used to generate preliminary categories and definitions of measurement approaches.

Sample and sampling procedures

Purposive sampling was used to identity medication nonadherence experts for the survey. The goal was to ensure diversity in the disciplines and the geographic representation of the respondents. Consistent with empirically derived recommendations for the Delphi panel size [35], the study authors aimed to obtain completed surveys from a minimum of 20 experts in each survey round. To account for possible refusal or attrition, 30 experts were invited to participate.

To be eligible, the Delphi respondents were required to be national or international experts in the science of medication nonadherence, as evidenced by an established track record in scientific publications in respected peer reviewed journals, published policy reports, and/or invited participation at conferences relevant to the science of medication nonadherence, with the first scholarly contribution published at least 5 years prior to being surveyed. Respondents were also required to have sufficient English proficiency to be capable of completing the survey in English without needing translation services. Respondents were also selected to ensure representation of the following disciplines: health services research, clinical research, statistics, medicine, nursing, pharmacy/pharmaceutical services, pharmacoepidemiology, behavioral medicine/health psychology, public health, and health policy.

Surveys were delivered to respondents via email with a link to a Qualtrics (Qualtrics International Inc.) questionnaire. The questionnaire was preceded by an information page that explained the risks and benefits of participating in the study. Completion of the survey indicated informed consent had been provided. Respondents were eligible to receive a US$50 honorarium if they completed each survey.

Survey 1: determining extent of agreement with nonadherence behaviors and measurement approaches

The purpose of the first survey was to determine the extent of agreement with the categories and descriptors of distinct nonadherence behaviors and measurement approaches that were generated by the study authors. Specifically, respondents were surveyed to determine whether they thought each behavior should be included or excluded from the preliminary framework, and if any additional behaviors should be added (Table 1, left column). They were then asked if they agreed with the study authors’ descriptions of the behaviors. Also as part of the first survey, respondents were asked whether they thought each measurement approach should be included or excluded in the framework (Table 2, left column). They were then asked whether they agreed or disagreed with the definition of each measurement approach and were provided with an opportunity to recommend additional measurement approaches for inclusion in the framework. Text fields were presented at the end of each section so that respondents could explain their responses. The first survey also contained open-ended items eliciting respondents’ three most preferred self-report measures and three most preferred electronic device monitors.

Table 1.

Preliminary and revised categories of nonadherence behaviors

Survey 1 (N = 24)	Survey 2 (N = 22)
Non-Initiation	Non-Initiation
• Not filling the initial medication (96%) • Not taking the first pill (88%)	• Not filling the initial medication • Not taking the first pill
Incorrect Implementation	Incorrect Implementation
• Missing doses (100%) • Taking extra doses (92%) • Taking doses at the wrong time (83%) • Refilling the medication late or not at all (92%) • Stockpiling medication defined as obtaining refills when large supplies were already on hand (58%) • Improperly administering medications, leading to incorrect dose (e.g., improper asthma inhaler technique, or failure to take levothyroxine with food) or increased risk for side effects (e.g., lying down after taking oral bisphosphonate) (92%)	• Missing doses • Taking extra doses • Taking doses at the wrong time • Refilling the medication late or not at all • Stockpiling medication (i.e., obtaining refills when large supplies were already on hand) • Improperly administering medications, leading to incorrect dose (e.g., improper asthma inhaler technique, or failure to take levothyroxine with food) or increased risk for side effects (e.g., lying down after taking oral bisphosphonate)
Non-Persistence	Non-Persistence
• Discontinuing the medication prior to the date recommended by the prescribing clinician (96%)	• Discontinuing the medication prior to the date recommended by the prescribing clinician

Survey respondents were asked whether each of the behaviors should be included or excluded from our framework. In survey 1, percent agreement with inclusion of each nonadherence behavior is provided in parentheses; consensus with classification was viewed as present if >70% of respondents agreed with inclusion. In survey 2, percent agreement with changes made to survey is provided in parentheses in right column).

Table 2.

Preliminary and revised categories and definitions of nonadherence measurement approaches

Preliminary Framework included in Survey 1	Revised Framework included in Survey 2
Self-report (100%) Patients report on their medication-taking behavior using a structured set of questions and response scales; may be administered orally by an interviewer or in self-administered format (paper or computer)	Self-report Patients report on their medication-taking behavior using a structured set of questions and response scales; may be administered orally by an interviewer or in self-administered format (paper or computer)
	Proxy report (94%) A patient’s caregiver or health care provider reports on a patient’s medication-taking behavior using a structured set of questions and response scales; may be administered orally by an interviewer or in self-administered format (paper or electronically)
Prescription fill data from pharmacy, insurance, or other administrative database (100%) Data are obtained from a clinic, healthcare system, payer, or pharmacy on prescriptions dispensed to patients, including medication identifiers (e.g., medication name, therapeutic class, National Drug Code), dates of refills, and quantity/days’ supply dispensed. The data are used to quantify medication coverage and/or gaps over a time period of interest (e.g., proportion of days covered, medication possession ratio)	Prescription fill data from pharmacy, insurance, or other administrative database Data are obtained from a clinic, healthcare system, payer, or pharmacy on prescriptions dispensed to patients, including medication identifiers (e.g., medication name, therapeutic class, National Drug Code), dates of refills, and quantity/days’ supply dispensed. The data are used to quantify medication coverage and/or gaps over a time period of interest (e.g., proportion of days covered, medication possession ratio)
Dose or pill count (92%) The number of doses remaining in a medication container (e.g., pill bottle, blister pack, or inhaler with dose counter) are counted, and a comparison is made of how many doses are supposed to be left versus how many are left. Dose counts can be conducted in different settings including office visits at which patients bring their medication containers, at home visits, and telephone visits at which patients gather all their medications	Dose or pill count The number of doses remaining in a medication container (e.g., pill bottle, blister pack, or inhaler with dose counter) are counted, and a comparison is made of how many doses are supposed to remain versus how many remain. Pill counts can either be announced or unannounced, and can take place in different settings, including office visits at which patients bring their medication containers, home visits, and telephone visits at which patients gather all their medications
Electronic drug monitoring (100%) Medications are put into a bottle or other device with a sensor that records the date and time when the device was opened. Data can be used to calculate the correct timing adherence (% doses taken within specific time intervals), correct dosing adherence (% days with correct number of pill bottle openings), and adherence (% prescribed doses taken over a time period)	Electronic drug monitoring Medications are put into a bottle or other device with a sensor that records the date and time when the device is opened. Data can be used to calculate the correct timing adherence (% doses taken within specific time intervals), correct dosing adherence (% days with correct number of pill bottle openings), and adherence (% prescribed doses taken over a time period)
Drug or drug metabolite level (96%) A blood, urine, or other biospecimen is taken, and the level of the drug of interest or its metabolite is measured in a laboratory	Drug or drug metabolite level A blood, urine, or other biospecimen is taken, and the level of the drug of interest or its metabolite is measured in a laboratory
Biomarkers (63%) A biospecimen is taken (e.g., blood), and the level of a biomarker of the drug’s effect is used to assess adherence (e.g., LDL level to assess adherence to statins). Alternatively, a physiologic measurement is taken (e.g., blood pressure), and the measure is used to assess adherence to a medication (e.g., antihypertensive)	Biomarkers    A biospecimen is taken (e.g., blood), and the level of a biomarker of the drug’s effect is used to assess adherence (e.g., LDL level to assess adherence to statins). Alternatively, a physiologic measurement is taken (e.g., blood pressure), and the measure is used to assess adherence to a medication (e.g., antihypertensive)
Ingestible sensors (74%) Patients take medication with an ingestible sensor that transmits a signal to an electronic device when the sensor is ingested	Smart technology (ingestible sensors) Patients ingest a pill or wear a device that passively or actively transmits a signal to an electronic device near or on the body (e.g., patch, breathing tube, wrist-worn device) to track whether a medication was taken
Direct observation (71%) Patients report to a location where a staff member watches them take a medication	Direct observation A staff member observes a patient taking a medication, either in person or remotely by video

Survey respondents were asked whether they agreed or disagreed with each of these approaches. In survey 1, percent agreement with inclusion of each nonadherence behavior is provided in parentheses; consensus with classification was viewed as present if >70% of respondents agreed with inclusion. In survey 2, percent agreement with changes made to survey are provided in parentheses in right column. Bold indicates words that were removed and italics indicates words that were added.

Refining the categories and definitions of nonadherence behaviors and measurement approaches

Ratings for extent of agreement with the inclusion of each nonadherence behavior and measurement approach were compiled for review by the study authors alongside associated comments. Agreement by a clear majority of survey respondents (i.e., ≥70%) was used as a benchmark for considering there to be consensus for each category and definition [36]. The authors then independently reviewed these data and determined whether each framework component and its associated description should be retained, modified, or discarded, and if any new elements of the framework should be added. The authors then came together to discuss aspects of the framework on which there was disagreement and to achieve consensus on a revised set of nonadherence behaviors and measurement approaches for the second survey.

Survey 2: assessing consensus with revised framework

In the second survey, the same group of 30 experts was asked whether they agreed with changes to categorizations and descriptions of nonadherence behaviors and measurement approaches in the revised framework. Changes from the initial framework were highlighted within the survey (Tables 1 and 2, right column). Survey respondents were also asked to provide comments in a text box to explain their responses. A similar consensus-building process was used by the authors to determine whether components of the proposed framework should be retained, modified, or discarded.

Survey 2: identifying the most suitable and optimal measurement approaches for nonadherence behaviors

In the second survey, respondents were also asked to rate the suitability of each measurement approach for measuring each nonadherence behavior. The instructions for this part of the survey prompted respondents to “consider factors such as the reliability, validity, cost, and feasibility of each approach in the context of health behavior research.” Response options were “not at all suitable,” “somewhat suitable,” and “very suitable.” The final task was to have respondents identify the optimal adherence measurement approach for each nonadherence behavior. Respondents could provide the response of “none of the above” if they did not view any of the measurement approaches as being optimal. Only one optimal approach could be selected for each nonadherence behavior. As in the first survey, a text field was provided at the end of each section so that respondents could explain their responses.

Descriptive statistics were used to describe the study population and the frequency with which measurement approaches were suitable or optimal for each nonadherence behavior. A threshold of agreement by >70% of survey respondents was used to indicate consensus. Open-ended responses that explained reasons for responses were content-analyzed. The qualitative data were triangulated with the quantitative data to inform the authors’ decisions about whether to include, exclude, or revise terms and definitions.

RESULTS

Survey respondent characteristics

Of 30 experts e-mailed the first survey, 24 responded (80% response rate). Respondents had a mean age of 54 years, nearly half were women, and the majority resided in the USA (Table 3). Respondents were highly experienced professionals, with a mean of 23 years since their terminal degree, and represented diverse disciplines and work settings.

Table 3.

Characteristics of experts in Delphi panel (N = 24)

Characteristics
Age in years, mean (SD)	54 (12) years
Women	11 (48%)
White	23 (96%)
Non-Hispanic	23 (96%)
Country of residence
United States	22 (92%)
Canada	1 (4%)
Belgium	1 (4%)
Disciplinea
Medicine	8 (33%)
Public Health	7 (29%)
Psychology	6 (25%)
Pharmacy	3 (13%)
Nursing	3 (13%)
Statistics	3 (13%)
Other	2 (8%)
Work settinga
Academia	18 (75%)
Government	4 (17%)
Industry	3 (13%)
Other	3 (13%)
Years since training, mean +/− SD (range)	23 +/− 11 years (9 to 40 years)

Data are presented as N (%) unless otherwise specified.

^aMore than one response possible.

Survey 1: extent of agreement with nonadherence behaviors

There was broad agreement with nearly all of the categories and descriptions of specific nonadherence behaviors that were presented in the first survey, with a few exceptions (Table 1, left column). The only nonadherence behavior for which there was <70% agreement was “stockpiling” medication, which was defined as obtaining refills when large supplies were already on hand (42% disagreed). A review of reasons for disagreement included the notion that stockpiling might not directly affect the manner in which pills are taken. Additionally, stockpiling could be construed as two different behaviors: missing/savings doses for later use versus purposefully obtaining more doses than needed. These comments led the study authors to remove stockpiling as a nonadherence behavior in the revised framework presented in the second survey.

With respect to the nonadherence behavior “refilling the medication late or not at all,” several respondents commented that it combined incorrect implementation (refilled late) with discontinuation (not at all). Because discontinuation was a separate behavior covered in the nonpersistence phase, the words “not at all” were removed from the description of the “refilled late” category in the second survey.

Survey respondents suggested additional nonadherence behaviors, but these were not added to the framework as the study authors viewed them as being better captured within the initial categories (e.g., “splitting doses” was not added as it was viewed as being subsumed within “incorrect dose”) or as indicating reasons for an already-captured nonadherence behavior, rather than the behavior itself [28,29] (e.g., “missing pills to save money” was recommended by one respondent but was not added as it was viewed as being captured by the category “missed doses”; “to save money” was viewed as a reason for nonadherence rather than as a distinct nonadherence behavior).

Survey 1: extent of agreement with measurement approaches

With respect to measurement approaches (Table 3, left column), there was <70% agreement for only one approach: “biomarker” (62% agreed). Respondents who disagreed with including biomarkers noted that this measure was only available for some medications and that it did not directly measure nonadherence behavior, as biomarkers can be influenced by biological factors unrelated to nonadherence. Based on these compelling reasons, biomarker was removed as a measurement approach in the revised framework shown in Survey 2 (Table 3, right column).

Although the majority of respondents (74%) agreed with the inclusion of the measurement approach “ingestible sensors,” multiple respondents commented that the category was insufficient for capturing the range of innovative technological nonadherence measurement solutions that were on the cusp of development (e.g., wrist worn sensors that track the motion of taking a pill, inhaler devices that track chemicals released in breath). Accordingly, the study authors modified the label of the method “ingestible sensor” to “smart technology” and expanded the definition to incorporate these emerging approaches.

Multiple survey respondents (21%) advocated for including “proxy report” as a distinct measurement approach, noting that it was particularly relevant in children or adults with cognitive impairment. After discussion, the study authors agreed that proxy report was sufficiently distinct from self-report to be added to the framework. Other respondents advocated for including “patient diary” as a distinct measurement approach. Because patient diary was viewed as a specific tool subsumed within the self-report measurement approach, the study authors did not add patient diary to the framework.

The remaining changes to measurement approaches from the first survey to the second survey corresponded to refinements of definitions recommended by survey respondents to improve clarity or acknowledge possible variations in implementation of the approach. Specifically, the definition of “direct observation” was broadened to incorporate the potential for observation to occur by video. The definition of “pill counts” was more clearly specified to account for the possibility of being conducted in an announced or unannounced manner. Finally, the definition of “electronic monitoring” was simplified by eliminating the description of different approaches to calculating extent of nonadherence through this methodology.

Survey 1: recommended self-report instruments

In response to the survey item eliciting survey respondents’ preferred instruments within the category of self-report, there was lack of consensus for any single measure, with the Morisky Medication Adherence Scales (MMAS, unspecified number of items, selected by four respondents; 8-item MMAS selected by two respondents [37]; 4-item MMAS selected by one respondent [38]), the Voils 3-item (selected by four respondents) [29], the Wilson 3-item (selected by four respondents) [39], and McHorney’s Adherence Estimator [40] (selected by three respondents) being ranked among respondents’ top three choices most frequently. However, one respondent also specifically recommended against using the MMAS due to concern that the scale combined a measure of the extent of nonadherence with reasons for nonadherence. The Wilson scale was highlighted as one that was available for use free of charge, and the Voils scale was highlighted as one that clearly separated extent of nonadherence from reasons for nonadherence.

Survey 1: recommended electronic monitoring devices

In response to the survey item eliciting preferred electronic adherence devices, again, there was lack of consensus for recommending a specific device. MEMS (Aardex Group) and Wisepill (Wisepill Technologies) were the most commonly recommended devices, with six survey respondents selecting MEMS and three selecting Wisepill. The primary reasons cited for recommending MEMS were that the devices were validated, well-accepted by the scientific community, and have a long track-record of use in behavioral research. The primary rationale for the Wisepill was its wireless capability that enabled remote data collection. Other commercially available electronic adherence devices, recommended by one survey respondent each, were the AdhereTech (AdhereTech) and eCAPs (Information Mediary Corp.) devices.

Survey 2: extent of agreement with revised framework

Of 30 experts emailed the second survey, 22 (73%) completed it. More than 85% of respondents agreed with the changes that were made to the categories and descriptions of nonadherence behaviors and measurement approaches (Tables 2 and 3, right hand columns). With respect to changing the measurement category “ingestible sensor” to “smart technology,” almost all agreed that this represented a logical change; one respondent cautioned that this term could become too broad, and that it should be carefully distinguished from electronic drug monitoring.

Survey 2: suitability of approaches for measuring nonadherence behaviors

With respect to suitability, there was consensus for multiple measurement approaches being at least somewhat suitable (rated as “somewhat suitable” or “very suitable” by >70% of respondents) for each of the nonadherence behaviors (Fig. 1). For example, for measuring “missed doses,” all measurement approaches were rated as at least somewhat suitable by >70% of survey respondents. In contrast, for “wrong doses,” only proxy report and direct observation were rated as at least somewhat suitable by >70% of respondents. At least 60% of respondents rated “self-report questionnaires” as at least somewhat suitable for measuring each of the nonadherence behaviors (Fig. 1). Self-report questionnaire was the only measurement approach for which a majority of respondents agreed it was at least somewhat suitable for measuring all of the nonadherence behaviors.

Fig. 1.

Percentage of respondents who rated each measurement approach “At Least Somewhat Suitable” for measuring each nonadherence behavior. Rx Fill = prescription refill data; Electronic = electronic drug monitoring; Smart Tech = smart technology such as digital pills or wearables; Observe = direct observation.

Survey 2: optimal approaches for measuring nonadherence behaviors

An optimal measurement approach was identified with ≥70% consensus for only two behaviors: prescription refill records for “not filling the initial prescription” (74%) and “refilling the prescription late” (77%) (Fig. 2). Respondents commented, however, that refill data were optimal in the context of a closed prescribing system in which one was aware of which medications were prescribed. For all but one remaining behavior (“not taking the first dose”), a single optimal measurement approach was identified by a majority of respondents, but fell short of the 70% threshold for consensus. “Electronic drug monitoring” was most commonly selected as the optimal approach to measuring “missed doses,” “taking extra doses,” or “taking doses at the wrong time.” “Direct observation” was most commonly selected as optimal for measuring “improper administration.” There was a clear absence of agreement for how best to measure “not taking the first dose,” with “self-report questionnaire,” “electronic drug monitoring,” and “smart technology” selected by 23% of respondents each. Despite being frequently identified as at least somewhat suitable for measuring all behaviors, “self-report questionnaire” was infrequently selected as the optimal approach.

Fig. 2.

Frequency each measurement method was rated as optimal for measuring each nonadherence behavior. There was consensus (>70% agreement) for and optimal measurement approaches for two nonadherence behaviors: using prescription refill data to measure not filling first prescriptionrefilling prescriptions late. Rx Fill = prescription refill data; Electronic = electronic drug monitoring; Smart Tech = smart technology such as digital pills or wearables; Observe = direct observation.

DISCUSSION

Health and behavior change scientists have long sought to incorporate the best approaches to measuring patient medication nonadherence into their studies. Although guidelines have recently been developed for reporting on medication nonadherence [41], to the best of our knowledge, there are no current guidelines that advise on which measurement approach one should select. Results of this Delphi poll support our premise that different approaches are more or less suitable or optimal for measuring each distinct nonadherence behavior, and that the selection of measurement approach should be matched to the distinct nonadherence behavior one wishes to measure. For example, if one wished to study characteristics associated with not filling initial prescriptions, then the optimal measure would likely be prescription fill data. In contrast, if one wished to assess the effect of a behavioral intervention designed to reduced missed doses of a medication, then electronic drug monitoring would likely be the optimal approach.

Another finding that emerged from our survey is that there was broad agreement with our framework for categorizing nonadherence behaviors. This lends support to the ABC taxonomy developed by ESPACOMP, which specifies that nonadherence should be understood as part of a spectrum of behavior defined by noninitiation, incorrect implementation, and nonpersistence [20]. Our findings extend this framework to specify the behaviors that fall within each of these phases.

Although these survey results inform the selection of nonadherence measurement approaches, the recommendations for optimal measurement approaches need to be customized to the specific research context. For example, although our survey suggests that prescription refill data would be optimal for determining the extent to which patients initiate the first fill of a medication, prescription fill data are not always available to researchers, so self-report might become the next best approach to measuring this nonadherence behavior. Behavioral scientists will need to balance feasibility with other issues for each setting.

The primary objective of this study was to guide researchers’ selection of one or more measurement approaches for assessing their nonadherence behavior(s) of interest, rather than their selection of a specific tool (e.g., the specific self-report scale, algorithm for calculating refill adherence, or electronic device to use when measuring a given nonadherence behavior). There are a range of available tools within each broad measurement approach, and the validity and reliability of these tools for assessing a given specific nonadherence behavior may vary across disease states and populations. Interestingly, when we asked respondents to identify their preferred tools for self-report questionnaires and electronic monitoring devices, there was a lack of consensus, indicating a need for more research in this area. Although it was beyond the scope of this study to make recommendations for specific tools to use for assessing different nonadherence behaviors, our work provides a systematic framework for reviewing and comparing existing measures and determining where new or better measures are needed to capture the full range of nonadherence behaviors. Future research should directly compare the reliability and validity of specific tools available within the measurement approach identified as optimal for assessing each of the eight specific nonadherence behaviors in the framework.

Some nonadherence experts, including those that drafted a landmark World Health Organization report on medication nonadherence, have recommended obtaining multiple measures of medication nonadherence within a single study [1]. Our results suggest a need to consider whether these multiple measures are intended to assess single or multiple specific nonadherence behaviors when deciding how to use them in analyses. When a set of nonadherence measures are intended to measure the same nonadherence behavior (and, thus, are likely to be highly correlated with one another), it may be appropriate to combine these measures into a single summary variable as a means of reducing measurement error. However, when multiple nonadherence behaviors are of interest in a study (e.g., assessing the impact of an intervention on filling initial prescriptions and on reducing missed doses after the initial prescription is filled), distinct measurement approaches (e.g., prescription refill records and electronic drug monitoring) may be needed. In such cases, it would be inappropriate to combine multiple measures into a single variable, given that they are conceptualized to assess two distinct behaviors.

Our findings also have implications for how best to use multiple measurement approaches to assess the validity of specific tools for measuring medication nonadherence. Electronic drug monitoring measures are commonly used as an imperfect reference standard for subjective nonadherence measures [5]. This may make sense in some contexts. For example, if one were designing a self-report questionnaire to assess the extent of missed doses, comparing the test properties with electronic monitoring data might be appropriate to determine the validity of the self-report questionnaire. Yet, our findings suggest that electronic monitoring should not be the reference standard for measuring other nonadherence behaviors, such as initiating a medication or administering a dose improperly. Thus, a more nuanced approach to identifying reference standards for subjective measures should be used.

It is notable that self-report questionnaires were commonly selected as at least somewhat appropriate for measuring nonadherence behaviors. Self-report questionnaires assessing the extent of nonadherence behaviors have the additional advantage of being readily linked to questions that assess reasons for nonadherence [29], including provider factors (e.g., poor communication. lack of provider trust) that can be key determinants of nonadherence [42–45]. Yet, many existing self-report measures focus on assessing the extent of missed doses or do not clearly specify which specific nonadherence domain they are intending to measure [46]. Future work examining the availability of valid self-report measures for assessing each of the eight specific nonadherence behaviors is needed, and new self-report measures may need to be developed to address gaps.

Limitations

There were several notable limitations to our findings. First, we had a moderate sample size of nonadherence experts. Nevertheless, our sample was reasonable for a Delphi poll, and the response rate of >70% across two surveys was satisfactory. Second, although respondents were asked to provide recommendations based on a general scenario, selections were likely to differ depending on the research context (e.g., budget for adherence measurement, chronic condition, and patient population being studied). Therefore, the application of recommendations for specific approaches to measure specific behaviors should be made judiciously. Third, although we achieved at least 70% agreement on each domain within our framework, we lacked strict agreement with several of the descriptions of adherence behaviors and measurement approaches. This indicates that additional work may be needed to more clearly refine our framework.

CONCLUSIONS

Expert consensus was achieved for a framework that classifies medication nonadherence behaviors and measurement approaches. Using this framework, we identified recommendations for which approaches were best suited to and optimal for measuring distinct nonadherence behaviors. This framework can be used to guide the selection of measurement approaches within adherence studies and as a basis for evaluating the availability of valid measures for assessing distinct nonadherence behaviors.

Funding: This study was funded by the National Institutes for Aging (U24-AG052175). Dr. Kronish received additional support from the National Heart, Lung, and Blood Institute (R01-HL123368 and R01-HL132347). Dr. Voils’ effort was supported by a Research Career Scientist Award from Veterans Affairs Health Services Research & Development Service (RCS 14-443).

Compliance with Ethical Standards

Conflicts of Interest: Dr. Kronish was responsible for data collection. All authors contributed to conceptualization of the manuscript, data analysis, data interpretation, manuscript writing, and revisions.

Ethical Approval: All procedures performed in studies involving human respondents were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The Institutional Review Board of Columbia University Irving Medical Center approved the study protocol. This article does not contain any studies with animals performed by any of the authors.

Informed Consent: Informed consent was obtained from all individual respondents included in the study.