Validation of a self-administered instrument to measure adherence to anticholinergic drugs in women with overactive bladder

Abstract

Aim

To validate a self-administered instrument, the Medication Adherence Self-Report Inventory (MASRI) for measuring adherence to anti-cholinergic medication for overactive bladder (OAB).

Methods

Prospective study in 131 women with OAB treated with fesoterodine. Adherence was measured at 8 and 12 weeks using an interviewer administered Brief Medication Questionnaire (BMQ) that assesses barriers to adherence (criterion standard), the MASRI, and pill count. Construct, concurrent and discriminant validity of the MASRI was assessed. We hypothesized that women who were non-adherent as measured by the MASRI would be more likely to have a belief barrier than women who were adherent to medication.

Results

Women diagnosed as non-adherent by the MASRI were more likely to report a belief barrier to taking medication as compared to adherent women at 8weeks (80 v 38%, p<0.001) and at 12 weeks (70% v. 40%, p=0.003). Significant correlations were noted between adherence rates measured by the MASRI and the BMQ at 8 weeks (r=0.87, p<0.001) and 12 weeks (r=0.90, p<0.001). Moderate correlation was noted between the adherence rate as measured by the MASRI and pill count at 8 weeks (r=0.49, p=0.02) but not at 12 weeks (r=0.05, p=0.87). The MASRI correctly identified 93% and 96% of non-adherent women at 8 and 12 weeks, respectively. Sensitivity, specificity and positive likelihood ratio of the MASRI for predicting non-adherence was 91%, 82%, and 5.1 at 8 weeks and 90%, 85% and 6.1 at 12 weeks.

Conclusions

The MASRI is a valid self-administered tool for measuring adherence to anti-cholinergic medication in women with OAB.

Introduction

Overactive bladder (OAB) affects over 15 million women in the United States and each year over a billion dollars worth of anticholinergic medications are prescribed to women with OAB (1, 2). The effectiveness of drug therapy depends on the patient’s adherence to the prescribed medication; and lack of adherence has been shown to lead to inappropriate dose escalation, change of medication, invasive testing, and increased cost of disease management (3).

Adherence data from traditional clinical trials of anticholinergic medications for OAB are not generalizable to clinical practice. Though adherence rates of 70–90% have been reported in randomized clinical trials, in clinical practice adherence is remarkably low reaching 9–20% at one year (4, 5). Despite this, adherence to anticholinergic medication is rarely measured in clinical practice. Electronic medication monitors and pill counts, widely used to measure adherence in randomized clinical trials, are expensive and/or cumbersome in the clinical setting (6). Several interviewer administered questionnaires have been developed and validated to measure adherence and identify barriers to adherence in chronic conditions such as hypertension, heart disease, and human immunodeficiency virus (HIV) (7, 8, 9). Interviewer administered questionnaires have high construct validity when compared to electronic medication monitors, pill count and pharmacy records; however, these questionnaires too can be difficult to operationalize in clinical practice. A valid self-administered instrument that can measure adherence to treatment for OAB in real-world clinical practice would allow providers to identify patients who are non-adherent to medications, as well as potential reasons for non-adherence. This would facilitate the opportunity to address barriers to adherence or offer alternative treatments and prevent waste of health care resources.

The Medication Adherence Self-Report Inventory (MASRI) is a concise self-administered instrument that is easily administered in the clinical setting and has been shown to accurately predict adherence to drug therapy in systemic lupus erythematosus (SLE) and HIV (10, 11). The validity of the instrument in OAB where symptoms are self-reported by patients is not known. The goal of the present study was to validate the MASRI against an interviewer-administered tool to measure adherence to anticholinergic medication in women undergoing treatment for OAB.

Methods

Patients

We performed a prospective study in women with OAB. Women complaining of OAB symptoms who presented to the Urogynecology and Urology clinics at the Hospital of the University of Pennsylvania were invited to participate in the study. Inclusion criteria were: age >18, urinary urgency for at least 3months with ≥8 voids and ≥1urgency episode/24hours on a bladder diary, and a rating of at least moderate bother on the single item Patient Perception of Bladder Condition (12). Exclusion criteria were: predominant stress incontinence, current or recent (<6months) use of anticholinergic medication for OAB, contraindication to anticholinergic medication, severe voiding difficulties, severe neurologic disease, recent (<6months) anti-incontinence or prolapse surgery or pregnancy, history of pelvic neuromodulation, other lower urinary tract disorders such as calculus, urethral diverticulum, and current or recurrent urinary tract infections. The protocol was approved by the Institutional Review Board at the University of Pennsylvania and all patients signed informed consent documents.

Study Protocol

The schedule of visits included a baseline visit, a telephone call at 2 weeks and two follow up visits at 8 and 12 weeks. All subjects provided demographic data and completed validated urinary symptom forms, the Urinary Distress Inventory (UDI) and the Incontinence Impact Questionnaire (IIQ) (13) at the baseline visit. Eligible women received 12 weeks of flexible-dose fesoterodine therapy (Toviaz; Pfizer, Inc, New York, NY) 4–8 mg. Participants were started initially on fesoterodine 4 mg. At their 2-week telephone call and their 8-week follow-up visit, they were offered the option of increasing their dose to fesoterodine 8 mg or receiving a prescription for a different anticholinergic. The purpose of the 2-week call was to assess if participants wanted to increase their dosage. All subjects received counseling on behavioral and lifestyle changes such as fluid intake modification and information on performing pelvic muscle exercises at the baseline visit. All participants also received written instructions at the baseline visit to bring the remaining supply of medication to their 8 and 12 weeks follow up visits. The schedule of visits was purposefully designed to be similar to usual clinical practice and frequent contact between study staff and participants was avoided.

Measurement of Adherence

Adherence to medication was measured at 8- and 12-week study visits using Brief Medication Questionnaire (BMQ)(8), the MASRI, and pill counts. At each follow up visit, patients first completed the MASRI. A research coordinator then interviewed the patient using the BMQ. After completing the BMQ, the research coordinator asked to see the patient’s medication bottle and performed a “pill count” if it was available.

Medication Adherence Self-report Inventory

The MASRI is a 12-item questionnaire of self-reported medication adherence that addresses two broad themes. The first section relates to the amount of medication taken (6 items, part A) and the second relates to the timing of the doses (6 items, part B). Since fesoterdine is taken once daily, part B of the MASRI was not administered to study participants. Part A consists of 4 items that asks patients to recall number of missed doses in previous day, two days before the visit, three days before the visit and the two weeks preceding the visit. There are four responses options (0, 1, 2, and “don’t know”). An additional item asks the patient to recall when the last time a dose was missed. The last item is a visual analog scale (VAS) that measures the adherence rate as a percentage ranging from 0 to 100%. Only the VAS item is used to get the numerical estimate of the adherence rate. The other items are designed to help the patients develop this estimate and do not contribute directly to the score. The MASRI is validated for measuring adherence to antiretroviral drugs in HIV patients when compared to medication event monitoring system (MEMS) (r= 0.63, p<0.001) and pill count (r=0.75, p<0.001) and has a sensitivity of 39–64% and specificity of 83–98% for detecting non-adherence (10). In subjects with SLE, the MASRI correlates moderately to strongly (r>0.55) with pharmacy refill data and has a sensitivity of 87% and specificity of 86% for identifying patients who are non-adherent (11). The MASRI has been used to measure adherence in a number of prospective studies (14, 15).

Brief Medication Questionnaire

The BMQ is an interviewer-administered tool for screening for the presence of adherence and barriers to adherence. The instrument consists of three scales, a Regimen screen, a Belief screen, and a Recall screen. The Regimen Screen consists of five items relating to how the patient took each medication in the past week. These items are used to classify the patient as being adherent or non-adherent. In addition, patient responses are used to calculate the rate of dose omission (proportion of prescribed dose omitted). A Belief Screen consists of two items that assesses the patient’s concerns or doubts about efficacy of the medication and concerns about side-effects and other bothersome features. These items are used to screen for the presence or absence of a belief barrier. A Recall Screen consists of two items that asks about potential difficulties remembering all doses and identifies the presence or absence of a recall barrier (8). In a population of patients prescribed angiotension-converting enzyme inhibitors, the Regimen Screen of the BMQ had good sensitivity (80%), specificity (100%), positive predictive value (100%), and overall accuracy (95%) for predicting nonadherence when compared to MEMS (8). Similarly, the Belief screen had good sensitivity (100%), specificity (80%), positive predictive value (62%), and overall accuracy (85%) for predicting non-adherence as measured by MEMS (8). The BMQ has been used to measure the presence of non-adherence and barriers to non-adherence in several chronic medical conditions such as diabetes, depression, and hypertension (16, 17, 18).

Pill Count

Adherence rates were calculated based on the number of pills initially given, the number of pills remaining and the interval between the baseline visit and the follow up visit.

Statistical Analysis

Descriptive characteristics of the sample were calculated from the baseline visit. Continuous variables were summarized using median and interquartile range and categorical variables were described using absolute and relative frequency. Women were defined as being adherent or non-adherent to the medication based on the Adherence Screen of the BMQ.

Construct validity, the degree to which an instrument measures the construct being investigated, was assessed by determining the relationship between non-adherence as measured by the MASRI and the presence of a belief barrier on the Belief Screen of the BMQ. We hypothesized that women who were non-adherent as measured by the MASRI would be more likely to have a belief barrier than women who were adherent to medication. We compared the proportions of women reporting a belief barrier in women who were adherent and non-adherent to the medication using the chi-square test. Concurrent validity, the relationship of an instrument to other similar instruments, was assessed by determining the relationship between the non-adherence rate as measure by the MASRI and the rate of dose omission as measured by the Adherence Screen of the BMQ as well as the pill count using the Spearman’s correlation coefficient. Spearman’s correlation coefficients were interpreted as follows: unrelated: r<0.2; weak: 0.2 ≤ r ≤ 0.4; moderate: 0.4 ≤ r ≤ 0.6; strong: r ≥ 0.6. Discriminant validity, the ability of an instrument to distinguish between distinct populations, was determined by the ability of the MASRI to distinguish between subjects who were adherent or non-adherent to medications as measured by the BMQ. We compared the proportion of women identified as non-adherent by the MASRI and the pill count to the BMQ using the chi-square test. Receiver operating characteristic (ROC) analysis was performed to identify optimum cut offs on the MASRI that would predict non-adherence, using the BMQ as the external standard. Area under the ROC curve (AUC) values closer to 1 are considered satisfactory and this was tested against a null hypothesis that the true AUC is 0.5 which indicates no discriminatory power. We also calculated the sensitivity, specificity, and likelihood ratios of the MASRI for predicting non-adherence and computed the 95% confidence intervals (CI) for each proportion. All analyses were performed in STATA version 12 (StataCorp LP, College Station, TX). Statistical tests were two-sided and p <0.05 was considered statistically significant.

Results

One hundred and thirty one women with a median age of 61 were enrolled. (Table 1). At 8 weeks, 22 (17%) women were lost to followup and 8 (6%) women had incomplete MASRI data. At 12 weeks, a total of 28 (21%) women were lost to followup and 4 (3%) women had incomplete MASRI data. Thus, adherence data was available for analysis in 101 (77%) and 99 (76%) women at 8 weeks and 12 weeks, respectively. Patients included in our analysis did not differ significantly from those who were lost to followup with respect to median age (61.4years vs. 57.4 years), UDI score (42.7 ± 22.6 vs 54.9 ± 23.0) and IIQ score (34.3 ± 27 vs 42.4 ± 30). Pill count data was available in only 21/101 (21%) patients at 8 weeks and 15/99 (15%) patients at 12 weeks.

Table 1.

Demographics and Baseline Characteristics

	N=131
Age	61(52, 70)
BMI	31(26, 37)
Race  White  Black  Other	76(58) 49(37) 6(5)
Education  Less than high school  HS/GED  Some college or more  Not reported	8(6) 47(36) 67(51) 9(7)
Prior anticholinergic use  Yes  No  Not reported	70(53) 56(43) 5(4)
Prior non-medical therapy for OAB  Yes  No  Not reported	59(45) 67(51) 5(4)
Number of other medications	6(4, 9)
Urinary Distress Inventory Score	45.8(25, 58.3)
Incontinence Impact Questionnaire Score	28.6(14.3, 57.1)

Data reported as median (interquartile range (IQR)) or frequency(%).

OAB = Overactive Bladder.

Based on the Regimen Screen of the BMQ, non-adherence to anticholinergics was common and occurred in 44/101 (44%) and 48/99 (48%) of women at 8 and 12 weeks, respectively. The rate of non-adherence was similar in women who were treatment naïve as compared to those previously exposed to anti-cholinergic medication (51% vs. 35% at 8 weeks, p=0.123). Based on the Belief screen of the BMQ, 55% women reported belief barriers at both 8 and 12weeks.

Construct Validity

At 8 weeks, 80% of the women diagnosed as non-adherent by the MASRI had a positive screen for a belief barrier on the BMQ Barrier Screen as compared to 38% of women who were adherent (p<0.001). Similar findings were noted at 12 weeks (Table 2).

Table 2.

Belief barriers in adherent and non-adherent patients

	Non-adherent (MASRI<80%)	Adherent (MASRI ≥80%)	p-value*
Negative beliefs at 8weeks (N = 101)	33/41(80.5%)	23/60(38.3%)	<0.001
Negative beliefs at 12weeks (N=99)	33/46(71.7%)	22/53(41.5%)	0.003

MASRI = Medication Adherence Self-Report Inventory.

^*chi-square test

Concurrent validity

MASRI adherence estimates were highly correlated with the rate of medication omission on the Regimen Screen of the BMQ at 8 weeks (r=0.87, p<0.001) and 12 weeks (r=0.90, p<0.001). In women with pill count data available, we noted moderate correlation between the adherence rates as measured by the MASRI and pill count at 8 weeks (r=0.49, p=0.02) but no correlation at 12 weeks (r=0.05, p=0.87).

Discriminant Validity

When compared to the BMQ, an adherence cut off rate of 80% on the MASRI correctly identified 41/44 (93%) women as non-adherent at 8 weeks and 47/48 (98%) at 12 weeks. When compared to the BMQ, the proportion of women correctly identified as non-adherent was significantly greater when using the MASRI compared to pill count at 8weeks (93% v. 9%, p< 0.001) and 12weeks (96% v 2%, p<0.001) owing mostly to lack of availability of pill count data.

ROC analysis

The adherence rates as measured by the MASRI and by the criterion standard, the Regimen Screen of the BMQ, were similar. AUC was 0.96 (95% CI (0.92, 0.99)) at 8 weeks (Figure 1) and 0.95 (95% CI (0.91, 0.99)) at 12 weeks (Figure 2). An optimal cutoff of MASRI adherence rate < 90% was 87% sensitive and 90% specific 8 weeks and 88% sensitive and 90% specific at 12 weeks for identifying women who were non-adherent to anticholinergics. (Table 3)

Figure 1.

Receiver operating characteristic (ROC) curve of the MASRI as compared to the BMQ at 8weeks. Area under the curve = 0.960 (95% CI 0.925–0.996).

MASRI= Medication Adherence Self-Report Inventory

BMQ= Brief Medication Questionnaire

Figure 2.

Receiver operating characteristic curve of the MASRI as compared to the BMQ at 12weeks. Area under the curve = 0.953(95% CI 0.914–0.992)

MASRI= Medication Adherence Self-Report Inventory

BMQ= Brief Medication Questionnaire

Table 3.

Diagnostic values of different cut points on the MASRI as compared to the BMQ at 8weeks

	MASRI Cutoff	Sensitivity (95% CI)	Specificity (95% CI)	Positive likelihood ratio(95% CI)	Negative likelihood ratio (95% CI)
8weeks	MASRI≥80%	0.91(0.80–0.97)	0.82(0.67–0.91)	5.02(2.67–9.44)	0.11(0.05–0.25)
	MASRI≥90%	0.88(0.76–0.95)	0.91(0.77–0.97)	9.65(3.77–24.69)	0.14(0.07–0.27)
	MASRI≥95%	0.75(0.62–0.85)	0.98(0.86–0.99)	33.19(4.75–230)	0.25(0.16–0.40)
12weeks	MASRI≥80%	0.90(0.78–0.96)	0.85(0.72–0.93)	6.18(3.10–12.34)	0.11(0.05–0.27)
	MASRI≥90%	0.88(0.75–0.95)	0.90(0.77–0.96)	8.47(3.67–19.54)	0.13(0.06–0.28)
	MASRI≥95%	0.78(0.64–0.88)	0.94(0.82–0.98)	12.55(4.16–37.89)	0.23(0.14–0.39)

MASRI= Medication Adherence Self-Report Inventory

BMQ= Brief Medication Questionnaire

Discussion

Though non-adherence to anticholinergic medication for OAB is common, a brief and easily administered instrument that can identify patients who are non-adherent in clinical practice is lacking. The most important finding of our study is that the MASRI is a valid tool to measure non-adherence to anticholinergic medication in the clinical setting. A cut off that corresponds to patients taking less than 80% of their medication has been commonly used in the literature to define non-adherence (19). Using the BMQ as the criterion standard, a cut off of 80% on the MASRI correctly classified 93% women as non-adherent at 8 weeks and 98% at 12 weeks. Women identified as non-adherent were significantly more likely to report negative beliefs about medications than women who were adherent. We also noted high correlation between adherence rates as measured by the MASRI and the rate of dose omission as calculated from the Adherence Screen of the BMQ. Taken together, these findings suggest that the MASRI is a valid instrument for identifying patients who are non-adherent to anticholinergic medications.

Our ROC curve analysis identified an optimal cut off of 90% on the MASRI for identifying non-adherence. A cut off of 80%, widely used in the literature to classify non-adherence, had higher sensitivity but lower specificity. The sensitivity and specificity of a test depend on the prevalence of the condition in the population and may not be generalizable outside the study. We therefore calculated the positive likelihood ratio, the increase in the odds of the abnormality when the test is positive. (20) The positive likelihood ratio of the 80% cut off on the MASRI was 5.02, which is lower than the ratio of the 90% cut off (9.65), but is still acceptable for predicting adherence to a medication for a condition such as OAB that is typically not life threatening in ambulatory women. Given the known high rate of non-adherence to anti-cholinergic medications for OAB in clinical practice, a cut off of 80% is likely reasonable for identifying women who are non-adherent.

The rate of non-adherence to anticholinergic medication in this study is higher than rates reported in most randomized clinical trials but similar to that reported in clinical practice (5, 21). Most randomized clinical trials are conducted in tightly controlled setting with frequent study visits and contact between study personnel and subjects. These measures can lead to higher rates of adherence than in clinical practice (22). The present study was designed to be pragmatic in nature and was conducted in conditions similar to clinical practice. This resulted in adherence rates similar to those previously reported through prescription refills and claims reimbursement data base (21). Our pragmatic design also demonstrated the difficulty of using pill count, the most commonly used in the clinical trials of OAB (3) to measure adherence, as only 20% subjects brought in their medication containers.

Unlike prior studies that have used electronic medication monitoring devices to validate adherence instruments (9, 10), we used an interviewer administered instrument as the criterion standard. Electronic medication monitoring devices are an imperfect ‘gold standard’ because patients can accidentally or intentionally activate the device without taking the medication. In a study of patients on antiretroviral therapy, Bova et al reported that a third of patients used a pill box instead of the prescribed electronic device, 41% took more than the prescribed dose at a time, 26% opened the cap but did not taking their dose of medications, and the device was thrown away or replaced by 10% of the subjects (23). Electronic monitoring devices can also increase adherence behavior (24) and reduce the generalizability of the findings to clinical practice. Given these limitations, we compared the MASRI to a well-validated interviewer administered instrument, the BMQ. The BMQ was developed using survey methodologies that minimize reporting errors such as asking about a behavior during a specific time period with shorter recall periods and using carefully worded questions that reduce memory errors and the level of threat or embarrassment experienced by patients who want to make a favorable impression (8). Though the BMQ requires dedicated time between the survey administrator and the patient, the BMQ has greater sensitivity and specificity compared to other existing interviewer administered adherence scales and allows assessment of barriers to adherence (25).

Our finding that the MASRI is a valid instrument to measure adherence in women with OAB is clinically useful. The MASRI can be easily and quickly administered as the initial part of the algorithm to identify patients with non-adherence. Since our study shows that adherence rates do not change from 8 to 12 weeks, consideration could be given to administering this instrument even before 8 weeks. Once these patients are identified, other tools such as the BMQ or the Belief in Medication questionnaire could be used to explore the underlying barriers to adherence, such as belief in the harm of medication or forgetfulness. These barriers can then be addressed before moving on to alternative treatments.

There are several strengths to our investigation. We validated the MASRI in a study that was purposefully designed to be similar to usual clinical practice and thus our findings have significant clinical applicability. We had a relatively large sample size and measured adherence at two time points (8 and 12 weeks) with similar findings. A potential limitation of our study is that we did not compare the MASRI to electronic monitoring devices. However, electronic monitoring devices have significant disadvantages as outlined above. The BMQ is a well-validated tool that reliably predicts the presence of non adherence. Future studies will be needed to measure the effect of adherence as measured by the MASRI on clinical outcomes in women with OAB.

In conclusion, the MASRI is a valid method of measuring adherence to anticholinergic medication in the clinical setting in women with OAB. The use of a self-administered patient questionnaire to measure adherence is attractive given its relative ease of use, decreased provider burden and reduced cost. The MASRI is a potentially useful tool for busy clinicians and can be used in future “pragmatic” trials examining anticholinergic use in women with OAB.