Development and evaluation of the Measure of Drug Self-Management

Abstract

Background

Current adherence scales often fail to assess the full spectrum of behaviors associated with safe and appropriate drug use and may be unsuitable for patients with limited health literacy. We sought to develop and evaluate a comprehensive yet brief Measure of Drug Self-Management (MeDS) for use in research and clinical settings among diverse patient groups.

Methods

Expert opinion, literature reviews, and interviews with patients and providers were utilized to create and revise potential items. Item performance testing was then conducted among 193 adult English-speaking patients with hypertension and diabetes. Factor analysis was used to inform item selection. Reliability was assessed via calculations of internal consistency. To assess construct and predictive validity, MeDS scores were compared with scores from the 8-item Morisky Medication Adherence Scale and relevant clinical measures (HbA_1c, blood pressure, and low-density lipoprotein cholesterol).

Results

The MeDS demonstrated adequate internal consistency with a Cronbach’s α of 0.72. The MeDS was significantly correlated with the Morisky Medication Adherence Scale (r= −0.62; P<0.001). The MeDS was also associated with clinical measures, with statistically significant correlations found between MeDS scores and low-density lipoprotein cholesterol (r= −0.27, P≤0.001) and diastolic blood pressure (r= −0.18, P=0.01).

Conclusion

The MeDS seems to be a valid and reliable tool that can be used to assess medication self-management skills among diverse patients, including those with limited literacy skills. Future studies are needed to test the tool in actual use and explore clinical applications.

Introduction

Non-adherence to prescription drugs is highly prevalent and can jeopardize the effectiveness and safety of a patient’s treatment.^1–5 While estimates of non-adherence vary depending upon how adherence is measured and the characteristics of the drug in question, evidence suggests that approximately half of US patients do not take medication as prescribed.^1–5 Non-adherence is likely to contribute to more than 125,000 deaths annually in the USA and has long been recognized as a serious clinical concern.⁶ Ensuring that patients take medications safely, appropriately, and consistently has therefore become a public health issue, particularly for the growing number of patients managing chronic diseases that require multi-drug regimens.^7,8

While the importance of medication adherence is well documented, its measurement remains a challenge.⁹ Objective assessments, such as e-monitoring, pharmacy claims data, and pill counts, may be useful in research, but are not as well suited to clinical practice. While devices like medication event monitoring system (MEMS) caps have advanced the measurement of medication adherence, they are often too costly to use in practice, especially when patients take multiple drugs. Pharmacy data can be imprecise and incomplete as they only capture patients’ fill behaviors and can require a lengthy observation period. Pill counts can be time-intensive, cumbersome, and inaccurate; this methodology also requires that patients remember to bring their medications with them to clinic or research appointments.

Given these shortcomings, self-report remains the most common method of assessing adherence.^9,10 However, common types of self-report measures, such as visual analog scales or patient recall of pills taken, fail to assess the full spectrum of behaviors a patient must undertake to effectively manage medications, particularly within the context of multi-drug regimens.^11–14 Many scales are unsuitable for use in clinical settings and few have been designed for use among patients with limited literacy skills.^15,16 In this study, we sought to develop and evaluate a self-management scale that would address these shortcomings. Herein we describe the development and testing of the Measure of Drug Self-Management (MeDS) among patients with multiple chronic conditions and complex drug regimens.

Methods

Item generation

To generate potential MeDS items, we engaged in an extensive and iterative process that incorporated multiple diverse perspectives. First, a list of potential MeDS questions was generated by the study team through an in-depth review of the scientific literature, existing measures, and expert opinion. Both subjective and objective questions were included to better reflect patients’ attitudes, beliefs, and drug use. The Health Literacy-Informed Model of Medication Self-Management was used to guide item generation and selection.¹⁷ This conceptual model deconstructs the tasks associated with taking medications, focusing on the patient knowledge, skills, and behaviors needed to correctly self-administer medications in ambulatory care. It describes a series of six steps that patients must take to ensure safe, appropriate, and consistent drug use over time: fill, understand, organize, take, monitor, and sustain. A complete description of the model has been published previously.¹⁷ Investigators ensured that multiple items reflecting each of these steps were included for testing.

Potential items were also reviewed by physicians and information technology specialists during in-person meetings with investigators. The focus of this review was to determine the appropriateness and completeness of items, as well as the feasibility of delivering the tool in the future via an electronic health record (EHR) platform. Concurrent to this process, we also sought patient feedback on potential MeDS items. This was accomplished through three iterative waves of brief interviews and discussion groups with adult English-speaking patients with diabetes and hypertension (n=17). Discussion groups focused on reviewing potential MeDS items and generating ideas for improvement. Specifically, groups discussed potential items one by one and weighed in on their understanding of the item, providing suggestions for wording improvements when possible. Patients also provided opinions on the content of the item, and whether they believed it would be relevant to their medication self-management skills. Patient input was used to revise potential MeDS questions and response options for subsequent testing. Additional items were also generated based upon patient suggestions.

Finally, health literacy “best practices” were applied to potential items to promote understanding, particularly among patients with limited literacy skills.¹⁸ For example, plain-language terms and phrases were used and response options were simplified when possible. Drug-specific questions (eg, “Did you take your simvastatin yesterday?”) were also included in the item pool as it was theorized that such specificity would help patients answer questions more accurately, particularly those with low health literacy, who often have difficulty with vague or imprecise concepts.¹⁸ Drug-specific items were designed to be applicable to almost all daily use prescription medications.

Item performance testing

To test potential MeDs items, patients were recruited from one large, academic general medicine clinic from June 2014 to November 2014. They were eligible to participate if they: had chart diagnoses of diabetes and hypertension; were prescribed at least three medications according to their medical record; were aged 18 years or older; were English-speaking; were primarily responsible for filling, organizing, and taking their own medication; and had no severe cognitive, vision, hearing, or other impairment that would preclude informed consent.

To recruit patients, the study was first reviewed by a physician committee and its approval was obtained to generate lists of potentially eligible patients (by diabetes and hypertension diagnoses and age) with an upcoming appointment scheduled. A research assistant then reviewed patient logs and approached these pre-identified patients in person to ask whether they would be interested in participating in the study. If the patient was interested, the research assistant verified eligibility, engaged the patient in the informed consent process, and initiated a structured in-person interview. Patients were compensated $20 for their participation in item testing activities. The institutional review board of the University of North Carolina at Chapel Hill approved the study procedures.

Measures

The study battery included 67 potential MeDS items for testing. As 12 of these items were drug-specific and referred to a particular drug in the patient’s regimen, the research assistant reviewed the patient’s medication list in the EHR prior to initiating the study interview and selected up to four medications to be referenced in the tool. In addition to MeDS items, the battery included questions regarding patient sociodemographic and health characteristics and the 8-item Morisky Medication Adherence Scale, one of the most commonly used self-report adherence measures.¹⁹ The validated Rapid Estimate of Adult Literacy in Medicine (REALM), a word pronunciation test, was used to assess patient literacy skills.²⁰ At the conclusion of the interview, patients were asked questions regarding the perceived usefulness of the MeDS tool and potential avenues for its delivery and use in clinical settings. Finally, clinical measures (most recent HbA_1c, diastolic and systolic blood pressure, and low-density lipoprotein [LDL] cholesterol) were recorded by the research assistant from patients’ electronic medical records.

Analyses

Simple descriptive analyses were first performed on all potential MeDS items, sociodemographics, clinical variables, and other covariates. MeDS items were screened for extreme ceiling effects and redundancy by examining inter-item correlations (ie, assessing if correlations were 0.8 or greater). If necessary, items were reverse-coded such that higher scores indicated better medication self-management skills. Exploratory factor analysis was used to reduce the number of items from the initial item pool. Specifically, principal component analysis with orthogonal Varimax rotation was employed. Several criteria were used to determine number of factors: scree plots, a minimum number of items (ie, at least 3) with high factor loadings for each potential component, and general interpretability of the factors. Subsequently, items were considered for elimination based on multiple statistical and theoretical aspects. In particular, items were retained for further analyses if their absolute values of factor loadings were high (ie, greater than 0.45) based upon the criteria proposed by Hair et al.²¹ Additionally, items had to show strong support for the theoretical interpretation of the factor to which they belonged. The item elimination process also included differential item functioning analysis to evaluate if selected MeDS items performed consistently across literacy levels, race, and age groups. To allow for detection of uniform and non-uniform differential item functioning, logistic regression was used.

In the final item elimination step, items’ contributions to the total scale’s internal consistency, as measured by items’ contributions to the value of Cronbach’s α, were assessed. Construct validity was assessed by examining Spearman’s correlations with the 8-item Morisky Medication Adherence Scale.¹⁹ Predictive validity was evaluated by measuring the correlation with patients’ most recent HbA_1c, blood pressure, and LDL cholesterol measures. SAS statistical software version 9.3 (SAS Institute, Cary, NC, USA) was used for all analyses.

Results

A total of 318 patients were approached; 87 patients declined, 33 patients were ineligible, 198 were consented, and 193 completed the interview. Fourteen patients had missing or do not know responses to one or more MeDS questions and were consequently excluded from factor analyses, for a total of 179 patients. All patients were included in analyses of the construct and predictive validity of the tool as well as perceived usefulness of the measure (n=193).

Patient characteristics

Patients were diverse in terms of race, educational attainment, and income. The mean age of this sample was 61.1 (28–94) years. The majority were female (60.1%) and approximately half were African American (54.4%). Approximately half (48.7%) of participants had low or marginal literacy skills according to REALM. Table 1 describes the characteristics of the study sample (n=193). There were no significant differences in terms of patient characteristics between the sample included in factor analyses (n=179) and the total study sample (n=193).

Table 1.

Characteristics of study sample

Variable, n (%)	Participants (n=193)
Age, mean (range)	61.1 (28–94) years
Female	116 (60.1%)
Race/ethnicity
African American	104 (54.4)
White	83 (43.0)
Other	5 (2.6)
Educational attainment
Some high school or less	37 (19.2)
12th grade or GED	73 (37.8)
Some college	45 (23.3)
College graduate or more	37 (19.2)
Do not know/refused	1 (0.5)
Income
<$10,000	39 (20.2)
$10,000–$19,999	60 (31.1)
$20,000–$29,999	26 (13.5)
≥$30,000	58 (30.0)
Do not know/refused	10 (5.2)
Literacy skills
Limited	27 (14.0)
Marginal	67 (34.7)
Adequate	99 (51.3)

Note: All currencies are in US$.

Abbreviation: GED, General Educational Development diploma.

Item selection

Of the 67 tested MeDS items, 16 items did not correlate strongly (ie, measures of sampling adequacy less than 0.5) with other items in the matrix and therefore did not qualify for inclusion in further analyses. Several additional items were eliminated due to ceiling effects and low variability in the response profiles; four items were removed because they were repetitive with better performing items. After careful review of the remaining MeDS items, along with the frequencies of item responses, it was determined that all item responses could be coded dichotomously, and a matrix of tetrachoric correlations used in the factor analysis (n=31 items included in analyses). Examination of the scree plot of eigenvalues and factor loadings in this initial solution indicated a 3-factor solution. Following these analyses, additional items qualified for deletion because of low rotated factor loading (ie, less than 0.45), repetition, or poor conceptual fit. No differential item functioning was detected with the reduced pool of items. In the final item elimination phase, three items having a low correlation with the total scale were deleted. One item with a lower correlation with the total scale (#5) was retained for further analyses as it was conceptually important according to the theoretical model guiding our scale development process.

The original MeDS item pool was thus reduced to 12 final items, and the sum of the responses was calculated to create a final score. Examination of the scree plot of eigenvalues and factor loadings for this 12-item solution indicated that one factor was present, accounting for 30% of the variance. The mean total score of the scale was 9.4 (standard deviation 2.3), while the median was 10 with a minimum of 1 and maximum of 12. Table 2 describes the items included in the final scale.

Table 2.

Item analysis of final scale (n=179)

	Mean	Item-total correlation coefficient	Cronbach’s α if item is deleted
1. Did you forget to take your (insert drug name) at any time last week?	0.82	0.47	0.68
2. In the past month have you stopped taking (insert drug name) for any reason without telling your doctor?	0.90	0.31	0.71
3. I often forget to take my medicine.	0.58	0.39	0.70
4. I am organized about when and how I take my medicines.	0.77	0.45	0.69
5. I have a hard time paying for my medicines.	0.61	0.19	0.72
6. The print instructions on my prescription bottles are confusing.	0.91	0.29	0.71
7. Having to take medicines worries me.	0.80	0.35	0.70
8. I often have a hard time remembering if I have already taken my medicine.	0.81	0.38	0.70
9. I do not take my medicines when I am feeling sad or upset.	0.91	0.39	0.69
10. My medicines disrupt my life.	0.93	0.31	0.71
11. When my medicine causes minor side effects, I stop taking it.	0.68	0.38	0.70
12. The idea of taking medications for the rest of my life makes me very uncomfortable.	0.73	0.31	0.71

Reliability and validity

The total MeDS score demonstrated adequate internal consistency with a Cronbach’s α of 0.72. The MeDS tool was significantly correlated with the 8-item Morisky Medication Adherence Scale (r = −0.62, P<0.001). An assessment of the scale’s predictive validity showed that patients with higher HbA_1c, systolic blood pressure, diastolic blood pressure, and LDL cholesterol levels tended to demonstrate poorer medication self-management, with a statistically significant relationship found between the MeDS score and LDL cholesterol (r = −0.27, P<0.001) and diastolic blood pressure (r = −0.18, P=0.01; Table 3).

Table 3.

Internal consistency, construct validity, and predictive validity for the Measure of Drug Self-Management

Internal consistency		Construct validity	Predictive validity
		Correlation (P-value)	Correlation (P-value)
Cronbach’s α	Item-total correlation range	Morisky Medication Adherence Scale	HbA1C	Systolic BP	Diastolic BP	LDL levels
0.72	0.19–0.47	−0.62 (<0.001)	−0.07 (0.35)	−0.09 (0.19)	−0.18 (0.01)	−0.27 (<0.001)

Abbreviations: BP, blood pressure; LDL, low-density lipoprotein

Finalizing the tool

The final 12-item MeDS includes newly developed items as well as some which were derived from previously published scales. It includes two drug-specific questions that may be asked of any daily use medication in the patient’s regimen. All items are designed to be coded dichotomously, as either indicative of adequate drug self-management skills or not. This allows for the final tool to be a sum of scores, ranging from 0 (indicating poor medication self-management) to 12 (adequate medication self-management). The tool can also be used to categorize patients as having “adequate” drug self-management skills (score of 10 or greater) or “inadequate” drug self-management skills (score of less than 10) based upon receiver operating characteristic analyses.

The final tool, including the response options and coding, is displayed in Figure S1. Response options for three items on the scale (#7, 8, and 10) were changed from “strongly agree, agree, disagree, strongly disagree” to “agree or disagree” after testing. This revision was consistent with patient preferences, health literacy best practices, and the distributions for these questions. Additionally, one item (#3), originally worded as “How often do you forget to take your medicine?”, was reworded to “I often forget to take my medicine” in order to be consistent with the phrasing of other questions included in the scale.

Patients’ perceived usefulness of the MeDS and avenues for delivery

Most participants (88.1%; n=193) reported that they would be willing to answer a brief set of questions about their medication use before each clinic visit. A subset of these patients (72.6%) also reported that they would be able and willing to complete the MeDS tool online before coming for their visit. Of the 21 patients who were not willing to complete the final MeDS tool, five attributed this to time constraints, nine listed the inconvenience involved with surveys, and seven believed that the tool would have no benefit because they were adherent to their medicines. Overall, 88.6% of the participants interviewed believed that this tool could help them or other patients to take their medicines safely.

Discussion

Findings from this study indicate that the MeDS is a valid and reliable measure of patient self-management skills. The tool was developed using an extensive process that incorporated patient, physician, and expert opinions; health literacy best practices were also considered in the design of items and response options.¹⁸ Psychometric testing revealed adequate internal consistency and statistically significant correlations with the 8-item Morisky Medication Adherence Scale and meaningful clinical measures, including LDL cholesterol and diastolic blood pressure.

While numerous adherence measures exist, the MeDS is unique in its use of objective and subjective items addressing various patient behaviors and abilities that are essential for effective medication self-management (ie, filling a prescription, understanding directions for use, taking medications, organizing a regimen, monitoring side effects, and sustaining use over time).¹⁷ It is also notable as it was developed incorporating patient and provider perspectives and health literacy best practices; the resulting tool is written at a 6th grade reading level and is easy to use among diverse patient populations. Patient feedback on the tool suggests that patients would be willing to complete the assessment prior to a clinic visit and find value in such a tool. Thus, the MeDS may have both clinical and research applications.

This study has limitations that should be noted. First, it was conducted at one site among English-speaking, predominantly African American and White patients, which may limit its generalizability. As the MeDS tool relies upon patient self-report, it is possible that results are not reflective of actual medication use due to social desirability bias or patient misinterpretation of items; this, however, is true for all self-report measures. As the tool includes two drug-specific questions, it requires knowledge of the patient’s prescribed drug regimen. However, we chose to include drug-specific questions as we theorized that they would provide more accurate responses, particularly among patients with low health literacy, who often have trouble with vague or imprecise concepts.¹⁸ It is also possible that the drug-specific questions may not be suitable for all drugs (ie, medicines prescribed pro re nata [PRNs]).

While in this study the MeDS tool was completed with the help of a research assistant, the tool was ultimately designed to be linked to an EHR to allow for clinical application. We therefore sought input from information technology specialists to ensure that drug-specific questions could be automatically populated from EHR fields containing drug names, dosages, and corresponding instructions (also known as “sigs”). If successfully linked to an EHR, the MeDS tool could be used to routinely assess patient medication self-management skills and help identify those patients in need of further intervention. This is important, as physicians rarely utilize standardized means of assessing adherence; evidence also suggests that physicians’ clinical judgment of patient adherence is often inaccurate.²² A study conducted by Gilbert et al found that the sensitivity of the primary care physician’s clinical judgment for detecting non-adherence was only 10%; other studies have similarly shown that physicians and nurses overestimate patient literacy skills, which are likely to influence their medication management capabilities.^23–25 The MeDs could therefore serve as a valuable tool that would help standardize how adherence is assessed, helping to initially detect patients who may be struggling with medication use and could possibly benefit from further, more advanced intervention such as medication therapy management.

While medication non-adherence is a highly recognized public health and patient safety concern, current adherence measures often do not assess the full spectrum of knowledge and behaviors necessary to successfully take medication in outpatient settings. Findings from our evaluation of the MeDS indicate that the measure is predictive of meaningful clinical outcomes and that patients are supportive of its use in clinical practice. Further resources are needed to link the MeDS to EHR platforms and to test the tool in actual use.

Supplementary material

Figure S1

Measure of Drug Self-Management (MeDS).