Abstract
Purpose
We examined how the choice of historic medication use criteria for identifying prevalent users may bias estimated adherence changes associated with a medication copayment increase.
Methods
From pharmacy claims data in a retrospective cohort study, we identified 6,383 prevalent users of oral diabetes medications from four VA Medical Centers. Patients were included in this prevalent cohort if they had one fill both 3 months prior and 4–12 months prior to the index date, defined as the month in which medication copayments increased. To determine whether these historic medication use criteria introduced bias in the estimated response to a $5 medication copayment increase, we compared adherence trends from cohorts defined from different medication use criteria and from different index dates of copayment change. In an attempt to validate the prior observation of an upward trend in adherence prior to the date of the policy change, we replicated time series analyses varying the index dates prior to and following the date of the policy change, hypothesizing that the trend line associated with the policy change would differ from the trend lines that were not.
Results
Medication adherence trends differed when different medication use criteria were applied. Contrary to our expectations, similar adherence trends were observed when the same medication use criteria were applied at index dates when no copayment changes occurred.
Conclusion
To avoid introducing bias due to study design in outcomes assessments of medication policy changes, historic medication use inclusion criteria must be chosen carefully when constructing cohorts of prevalent users. Furthermore, while pharmacy data have enormous potential for population research and monitoring, there may be inherent logical flaws that limit cohort identification solely through administrative pharmacy records.
Keywords: Medication adherence, pharmaceutical policy, cost sharing, research design, veterans, inclusion criteria
Studies have shown that increased medication cost sharing is associated with reductions in medication adherence among prevalent users (Goldman, Joyce, and Zheng 2007), yet the magnitude of adherence reductions has varied across studies. Prevalent users of a medication are patients with a history of filling a medication of interest within a defined lookback period (e.g., 12 months) prior to the copayment change (Ray 2003). Researchers interested in examining medication adherence of prevalent users typically utilize pharmacy administrative data both as a source from which to apply inclusion criteria as well as the substrate data from which adherence measurements are calculated. Prevalent use is typically specified within a defined duration of a lookback period, a number of intervals within the lookback period (e.g., two 6-month periods), and a minimum number of medication fills per interval. Such specifications may vary from inclusive (e.g., one medication fill in the prior 12 months) to restrictive (e.g., two fills within the prior 6 months).
The impact of different durations of the lookback periods have been examined in risk adjustment studies (Zhang, Iwashyna, and Christakis 1999; Preen et al. 2006) and in studies identifying incident, or new, medication users (Gardarsdottir, Heerdink, and Egberts 2006). However, the impact of applying different inclusion criteria for identifying prevalent medication users has not been examined previously and is a potential source of between-study difference in estimated adherence changes. This has been demonstrated in two prior evaluations of a $5 VA copayment increase on statin adherence. Both studies defined prevalent users as patients who filled one or more medications in the quarter prior to the copayment change (Doshi et al. 2009; Maciejewski et al. 2010a). One study also required 1+ fills within the 24 months prior to the copayment increase (Doshi et al. 2009) for inclusion, whereas the second study also required 1+ fills in the 4–12 months prior to the copayment increase. The first study with the longer (24-month) medication fill criteria found a significant reduction in statin adherence (Doshi et al. 2009). Another study (Maciejewski et al. 2010a) found no significant reduction, suggesting that the chosen lookback period and criteria may unduly influence findings or inferences. Studies of medication adherence have applied different lookback and historic fill criteria to define prevalent users (Table 1), which may have contributed to the variability in the estimated adherence response across studies.
Table 1.
Summary of Prior Literature on Medication Inclusion Criteria for Identifying Prevalent User Cohorts in Copay Change Evaluations
| Characteristics of Preperiod Inclusion Criteria of Medication Use | |||||
|---|---|---|---|---|---|
| Author (Year) | Preperiod Duration (Months) | Number of Preperiod Intervals | Overlapping Intervals? | Number of Fills per Interval | Each Preperiod Interval Length |
| Maciejewski et al. (2010a) | 12 | 2 | No | 1+, 1+ | 3, 9 |
| Doshi et al. (2009) | 24 | 2 | Yes | 1+, 1+ | 3, 24 |
| Yin et al. (2008) | 24 | 1 | No | 1+ | 24 |
| Blais et al. (#b3000) | 70 | 1 | No | 1+ | 70 |
| Roblin et al. (2005) | 6 | 1 | No | 4+ | 6 |
Alternative inclusion criteria are likely to retain differing proportions of poorly adherent, episodic, and highly adherent patients, as well as differing proportions of current and noncurrent medication users. In the absence of gold standard inclusion criteria for identifying prevalent medication users, we conducted two analyses to examine whether the choice of medication use inclusion criteria causes variation in estimated adherence changes associated with a copayment increase. In the first analysis, we applied different medication use inclusion criteria to a sampling frame for a fixed index date (February 2002) when the VA medication copayment increase was implemented. This first approach allowed us to examine whether different inclusion criteria generated similar or different adherence trends when the index date was held constant. This is analogous to four research teams independently evaluating the same policy change, each using a different set of inclusion criteria to identify prevalent users.
In the second analysis, we applied the same medication use inclusion criteria to the same sampling frame but at four different time periods. The time periods included the date of the actual copayment change (February 2002) when we expected an adherence response, and three alternative time periods when we did not expect an adherence response because no cost-sharing changes occurred. These alternative time periods were established 6 months before the copayment increase (August 2001), 6 months after (August 2002), and 12 months after the copayment increase (February 2003). We retained similar proportions of poorly adherent, episodic, and highly adherent patients using a common inclusion criterion. This allowed us to examine whether adherence trends associated with the copayment change differed from adherence trends observed in time periods when no medication copayment changes occurred. This second evaluation is analogous to testing the stability in adherence trends in the absence of copayment changes. We would have greater confidence that the adherence changes around the actual copayment increase were valid if no adherence changes were observed at these alternative three time points. On the other hand, the validity of our results would be called into question if similar adherence trends were observed at all four time periods because the similar trends would suggest that we induced an artifact by our particular choice of inclusion criteria.
These two analyses are informative for researchers and policy makers identifying prevalent medication user cohorts from secondary data to evaluate the effects of policy changes on medication use. To ensure that comparative effectiveness research (CER) for medication use based on observational studies generates unbiased results, it is important to understand the possible influence of inclusion criteria for identifying prevalent users on adherence results.
Methods
Data, Outcome, and Explanatory Variables
Details on the 2001–2003 data and sample of patients with diagnosed diabetes were published previously (Maciejewski et al. 2010a). In brief, the original study used a retrospective, prepost cohort design with a nonequivalent control group at four VA medical centers (VAMCs). Medication data for constructing the adherence outcome were obtained from the national Pharmacy Benefits Management database.
The medication adherence outcome is a dichotomous outcome indicating whether a patient had sufficient doses available of up to three oral hypoglycemic agent (OHA) classes—sulfonylureas, thiazolidinediones, or metformin. This outcome was based on a validated refill adherence algorithm called ReComp (Bryson et al. 2007), which estimates the proportion of days covered for a measurement interval using the date dispensed and the number of days supplied with each fill.
For each month, we first determined the proportion of days covered (PDC) for each OHA independently. This included accounting for oversupply from the prior month and applying it to the current month. The methods are described in detail in the study describing the algorithm (Bryson et al. 2007). Second, we then averaged the PDC for the regimen. For example, for a patient with a regimen of glyburide and metformin, if he/she had all 30 days of glyburide (100 percent) and no days of metformin (0 percent), he/she would be 50 percent adherent to the entire regimen. Third, we dichotomized these summary measures to denote adherent patients as having ≥80 percent of the days for that month covered.
Demographic characteristics (age, gender, race, and marital status) were obtained from VA inpatient and outpatient administrative data, while deaths for patient exclusion were obtained from the Benefit Identification and Record Locator System death record. Comorbidity burden was measured by the Diagnostic Cost Group Hierarchical Condition Categories (DCG) version 6.0 risk adjustment score because it has been shown to reliably predict veterans' total costs (Maciejewski et al. 2005; Maciejewski, Liu, and Fihn 2009) and risk of hospitalization or death (Fan et al. 2006). Diabetes severity was measured by the Diabetes Complication Severity Index (Young et al. 2008).
Identification of Alternative Prevalent User Cohorts and Alternative Time Periods
In the original study, we examined how adherence changed due to a $5 medication copayment increase among veterans diagnosed with diabetes or hypertension who were prevalent users of oral hypoglycemic agents or antihypertensive medications.
The initial sample for analysis 1 (different criteria, same timeframe) and analysis 2 (same criteria, different timeframes) consisted of 60,017 veterans with diabetes or hypertension who were prescribed a medication for either of these conditions in 2000 (Appendices 1 and 2). We excluded veterans who died prior to December 31, 2003 (n = 8,514) and those without diagnosed diabetes (n = 36,851) in January 1, 2000–December 31, 2003. This resulted in a sample of 14,652 veterans with diagnosed diabetes between January 1, 2000–December 31, 2003. From these 14,652 veterans, we replicated the prior study's inclusion criteria to identify the original unmatched cohort of 6,383 veterans who were prevalent users of oral hypoglycemic medications prior to VA copayment increase (February 4, 2002). These patients were identified as prevalent users if they had one or more fills in the quarter prior to the copayment increase (November 2001–January 2002) and had one or more fills in the 4–12 months prior to the copayment increase (second, third, or fourth quarter), which we refer to as the 3&9 criteria in Figure 1 and Appendix A.
Figure 1.
Illustration of Four Sets of Inclusion Criteria for Identifying Prevalent Medication Users in “Different Criteria, Same Time” Analysis
To assess the possibility that the choice of inclusion criteria of defining prevalent medication use might generate an artifact in the estimated adherence trend (Analysis 1), we constructed three alternative cohorts by applying three different inclusion criteria at the same index date (February 4, 2002) to the sample of 14,652 veterans (Figure 1, Appendix A). In the first alternative criteria (“2 in 12”), patients were identified as prevalent users if they had two or more fills in the year before the copayment increase (n = 7,297). In the second criteria (“6&6”), patients were identified as prevalent users if they had one or more fills in the two quarters (6 months) prior to the copayment increase and had one or more fills in the third and fourth quarters (6 months) before the copayment increase (n = 6,762). In the third alternative (“2 in 6”), patients were identified as prevalent users if they had two or more fills in the two quarters (6 months) prior to the copayment increase (n = 6,079). Since the index date was held constant and was linked to the medication copayment increase, we expected to observe adherence responses in all four cohorts but were unsure how the responses would differ across cohorts. To elucidate how the alternative inclusion criteria might generate different point estimates of the proportion of adherent patients, we also estimated adherence 12 months before and 12 months after the actual copayment change, the estimated change in adherence, and bootstrapped 95 percent confidence intervals (CI).
Next, we constructed three additional alternative cohorts of prevalent users using the same medication use criteria as the original cohort, but different time periods (Analysis 2), which we refer to as the “same criteria, different time” analysis (see Appendix B). The first alternative cohort (n = 6,248) was assigned an index date of 6 months before the actual copayment change (August 2001). The second alternative (n = 6,245) was assigned an index date of 6 months after the copayment change (August 2002). The third alternative (n = 6,129) was assigned an index date of 12 months after the copayment change (February 2003). We expected to see more modest (or no) adherence changes in these three alternative cohorts than in the original cohort, since the inclusion criteria were held constant but the index date in these three alternative cohorts was not linked to the medication copayment increase.
All analyses were conducted using generalized estimating equations. The unit of analysis was a person-month with each veteran having up to 36 repeated measures. The study was approved by the Human Subjects committees at the Durham and Seattle VA medical centers.
Results
Characteristics and Adherence Trends in Different Criteria, Same Time Analysis
Prevalent users were drawn from the same sample of 14,652 veterans, so a vast majority (91–100 percent) of the patients in the original cohort of 6,383 veterans were represented in the alternative cohorts in the “different criteria, same time” analysis and there were no significant differences in patient characteristics between cohorts (Table 2). Overall, oral diabetes medication users in the cohorts had an average age of 64, nearly all (98 percent) were male, 64 percent were married, 65 percent were white race, and only 15 percent were required to pay medication copayments. These patients had average DCG risk scores of 0.95–0.98 and had an average Diabetes Complication Severity Index score of 2.0–2.1.
Table 2.
Descriptive Statistics of Prevalent Users in “Different Criteria, Same Time” Analysis
| Number of Subjects | Cohort #1 | Cohort #2 | Cohort #3 | Cohort #4 |
|---|---|---|---|---|
| 7,297 | 6,383 | 6,762 | 6,079 | |
| Index date | February 1, 2002 | February 1, 2002 | February 1, 2002 | February 1, 2002 |
| Original or alternative cohort | Alternative 2 in 12 | Original 3&9 | Alternative 6&6 | Alternative 2 in 6 |
| Age (mean/SD) | 63.9 (10.6) | 64.1 (10.5) | 64.1 (10.5) | 64.1 (10.5) |
| <56 years (%) | 25.5 | 25.0 | 25.1 | 25.0 |
| 56–65 (%) | 25.1 | 25.2 | 25.2 | 25.0 |
| 66–75 (%) | 34.4 | 34.8 | 34.7 | 34.7 |
| 76–85 (%) | 14.6 | 14.7 | 14.7 | 14.9 |
| >85 (%) | 0.3 | <1 | 0.3 | 0.3 |
| Female (%) | 2.1 | 2.2 | 2.2 | 2.1 |
| Married (%) | 60.2 | 60.9 | 60.6 | 60.7 |
| White (%) | 64.5 | 64.9 | 64.5 | 65.0 |
| Non-white (%) | 16.3 | 15.3 | 15.8 | 15.6 |
| Unknown race (%) | 19.2 | 19.8 | 19.6 | 19.4 |
| Copay exempt from low income (%) | 48.0 | 48.0 | 48.0 | 48.5 |
| Copay exempt from disability (%) | 37.0 | 36.6 | 36.7 | 36.4 |
| Must pay copay (%) | 15.0 | 15.4 | 15.3 | 15.1 |
| DCG score (mean/SD) | 1.0 (1.4) | 1.0 (1.4) | 1.0 (1.3) | 1.0 (1.3) |
| Diabetes complication severity index | 2.1 (1.7) | 2.1 (1.7) | 2.1 (1.7) | 2.1 (1.7) |
| Hypertension (%) | 68.5 | 68.9 | 68.4 | 68.8 |
| Overlap:% of original cohort present | 100 | – | 96.4 | 91.1 |
Note: All comparisons are statistically insignificant.
Consistent with prior copayment evaluations, adherence for the original cohort increased in the months leading up to the $5 copayment increase in February 2002 and declined thereafter (“3&9” curve, Figure 2). Notably, the proportion of adherent patients increased markedly in the months just prior to the medication copayment increase (from 70 percent in October 2001 to 82 percent in January 2002) and then dropped by a similar, sizable amount in the months just after the copayment increase (from 82 to 71 percent in June 2002).
Figure 2.
Adherence Trends in the Original and Alternative Cohorts in “Different Criteria, Same Time” Analysis
The adherence trend from the original (3&9) cohort was most closely mirrored by the “2 in 6” cohort of patients who were defined as prevalent users if they were required to have two or more fills in the 6 months prior to the copayment increase, possibly because these criteria were more likely to include recent medication fillers in the analytic sample. However, the adherence trends in the other two (“2 in 12” and “6&6”) cohorts were quite different from the adherence trend of the original cohort. Adherence increased in the months leading up to January 2002 for the cohort of patients who were required to have two fills in the 12 months prior to the copayment increase, but much more modestly than the other three cohorts. Specifically, the proportion of adherent patients for the “2 in 12” cohort peaked at 72 percent in January 2002, 10 percent points lower than adherence in the original cohort and 9 percent lower than adherence in the “2 in 6” cohort. The adherence trend for the “6&6” cohort was higher than the trend for the “2 in 12” cohort, but lower than the original cohort and “2 in 6” cohort. Adherence in the “6&6” cohort peaked at 76 percent in January 2002, 6 percent lower than the original cohort.
The estimated change in the proportion of adherent patients 12 months before and 12 months after the copayment increase ranged from 0.3 to 6.1 percent across the four sets of inclusion criteria (Table 3). The estimated change in the proportion of adherent patients was most similar for the “6&6” cohort (0.3 percent, 95 percent CI: −0.7 percent, 1.4 percent) and the “2 in 12” cohort (0.9 percent, 95 percent CI: −0.2 percent, 2.0 percent), which both indicated no adherence change. Significant increases in the proportion of adherent patients were estimated using the original (3&9) cohort (4.6 percent, 95 percent CI: 3.3 percent, 6.0 percent) and the “2 in 6” cohort (6.1 percent, 95 percent CI: 4.7 percent, 7.4 percent). An important source of concordance between the “6&6” and “2 in 12” cohorts and between the original (3&9) and “2 in 6” cohorts were the proportion of patients in each cohort that had no fills in the 3 months prior to the copayment increase (October–December 2001). In the “6&6” cohort, 8.4 percent of patients had no fills in October–December 2001 compared to 11.1 percent of patients in the “2 in 12” cohorts (results not shown). In contrast, all of the patients in the original (3&9) cohort filled medication in October–December 2001 and only 3.0 percent of patients in the “2 in 6” cohort had no fills.
Table 3.
Estimated Change in Medication Adherence 12 Months before and 12 Months after the Actual Copayment Increase from Four Different Inclusion Criteria in “Different Criteria, Same Time” Analysis
| Estimated Adherence 12 Months before Copay Change | Estimated Adherence 12 Months after Copay Change | Estimated Change in Adherence | Bootstrapped 95% Confidence Interval | |
|---|---|---|---|---|
| Original approach (3&9) | 61.9% | 66.5% | 4.6% | 3.3%, 6.0% |
| Alternative #1 (2 in 12) | 59.2% | 60.1% | 0.9% | −0.2%, 2.0% |
| Alternative #2 (6&6) | 64.1% | 64.4% | 0.3% | −0.7%, 1.4% |
| Alternative #3 (2 in 6) | 60.3% | 66.4% | 6.1% | 4.7%, 7.4% |
Note: Adherence was defined as the proportion of adherent patients.
Characteristics and Adherence Trends in Same Criteria, Different Time Analysis
As in the “different criteria, same time” analytic cohorts, a vast majority (80–84 percent) of the patients in the original cohort were represented in the alternative cohorts from the “same criteria, different time” analysis. As in the other analytic cohorts, there were no significant differences in patient characteristics between cohorts (Table 4).
Table 4.
Descriptive Statistics of Prevalent Users in “Same Criteria, Different Time” Analysis
| Number of Subjects | Cohort #1 | Cohort #2 | Cohort #3 | Cohort #4 |
|---|---|---|---|---|
| 6,248 | 6,383 | 6,245 | 6,129 | |
| Index date | August 1, 2001 | February 1, 2002 | August 1, 2002 | February 1, 2003 |
| Original or alternative cohort | Alternative 1 | True-event | Alternative 2 | Alternative 3 |
| Age (mean/SD) | 64.1 (10.5) | 64.1 (10.5) | 64.1 (10.5) | 63.9 (10.5) |
| <56 years (%) | 24.9 | 25.0 | 25 | 25.5 |
| 56–65 (%) | 25.1 | 25.2 | 25.1 | 25.4 |
| 66–75 (%) | 34.9 | 34.8 | 35.1 | 34.6 |
| 76–85 (%) | 14.8 | 14.7 | 14.6 | 14.2 |
| >85 (%) | <1 | <1 | <1 | <1 |
| Female (%) | 1.9 | 2.2 | 2.3 | 2.0 |
| Married (%) | 60.8 | 60.9 | 60.9 | 60.7 |
| White (%) | 64.5 | 64.9 | 64.9 | 64.3 |
| Nonwhite (%) | 15.7 | 15.3 | 15.4 | 16.0 |
| Unknown race (%) | 19.9 | 19.8 | 19.7 | 19.7 |
| Copay exempt from low income (%) | 48.2 | 48.0 | 47.8 | 47.4 |
| Copay exempt from disability (%) | 36.4 | 36.6 | 37.0 | 37.2 |
| Must pay copay (%) | 15.4 | 15.4 | 15.2 | 15.4 |
| DCG score (mean/SD) | 0.97 (1.33) | 0.96 (1.35) | 0.95 (1.32) | 0.95 (1.31) |
| Diabetes complication severity index | 2.10 (1.73) | 2.07 (1.71) | 2.05 (1.71) | 2.03 (1.69) |
| Hypertension (%) | 68.4 | 68.9 | 68.4 | 68.6 |
| Overlap: % of original cohort present | 100 | 88.3 | 84.1 | 80.7 |
Note: All comparisons are statistically insignificant.
Contrary to our expectations that refill adherence would be stable in the three alternative cohorts that used the same inclusion criteria but in time periods without a copayment increase, we observed a significant increase in adherence during the months leading up to the index date (alternative curves in Figure 2). We also observed a subsequent adherence decline after the index date that mirrored the decline observed in the original cohort. These “adherence spikes” for these three alternative cohorts were of similar magnitude as the adherence trend in the original cohort, suggesting that the 3&9 criteria induced an artifactual spike in the original analysis.
Discussion
Evaluation of changes in adherence to chronic medications following a policy change (e.g., copayment increase, formulary change) is commonly conducted using prevalent user cohorts identified from administrative claims data. There is great variability in the historic medication use criteria used to identify prevalent users in prior research, including the duration of the lookback period, number of intervals in the lookback period, and a minimum number of medication fills per interval (Table 1). Each criterion choice impacts the proportions of poorly adherent, episodic, and highly adherent patients that are included. Many studies defined prevalent users on the basis of two prior medication fills to include patients who were routinely refilling chronic medications as directed by their providers prior to the policy change and to exclude historically less adherent patients. However, no prior studies have examined whether medication use criteria for identifying prevalent users introduce bias when estimating adherence changes following a medication policy change.
In this study and the original study on which this cohort is based (Maciejewski et al. 2010a), medication adherence increased in the months prior to the $5 copayment increase and subsequently declined. As in other studies of medication copayment increases (Roblin et al. 2005; Yin et al. 2008), we attributed the initial adherence increase in the preperiod to stockpiling by patients in anticipation of the copayment increase from $2 to $7 per 30-day fill. We attributed the decline in adherence after the copayment increase to cost-related nonadherence to oral hypoglycemic agents.
Based on the “different criteria, same time” analysis, results were not robust enough to choice of medication use inclusion criteria. Adherence trends and peak adherence at the time of the copayment increase (Figure 1), as well as estimated change in the proportion of adherent patients (Table 3), varied widely across the four sets of inclusion criteria. The adherence trend from the original cohort was mirrored when patients were included in the sample if they had two or more refills in the 6 months prior to the copayment increase, but adherence was much lower if patients were required to have only two or more refills in the 12 months prior to the copayment increase. In addition, adherence increased prior to the copayment increase and declined thereafter in all inclusion criteria sets.
Compared with the original (3&9) adherence trend, the difference in trends may in part reflect the proportion of patients without any medication fill in the 3 months prior to the copayment increase as a result of different inclusion criterion. The “2 in 12” cohort resulted in 11.1 percent of patients without any medication fill in 3 months prior to the copayment change, followed by the “6&6” cohort (8.4 percent), “2 in 6” cohort (3.0 percent), and “3&9” cohort (0 percent). This finding suggests that our original estimated adherence response to a copayment increase is not robust to the inclusion criteria for medication use. Estimated adherence changes 12 months before and 12 months after the actual copayment change were 4.6 percent in the original (3&9) cohort and 6.1 percent in the “2 in 6” cohort, but not statistically different from zero in the “6&6” and “2 in 12” cohorts. These results suggest that the choice of these three inclusion criteria (duration of the lookback period, number of intervals in the lookback period, and a minimum number of medication fills per interval) will strongly influence the proportions of poorly adherent, episodic, and highly adherent patients as well as proportions of current medication users retained in the final sample. Two sets of criteria (3&9 and “2 in 6”) that differed in all three inclusion criteria (Figure 1) had similar estimated change in adherence (Table 3) and identical postperiod adherence trends (Figure 2).
Our expectation that the “adherence spike” represented stockpiling leading up to the copayment increase followed by cost-related nonadherence is also belied by similar adherence spikes in the “same criteria, different time” analysis (Figure 3) when the same inclusion criteria were applied to alternative index dates when no copayment changes occurred. Copayments had already increased 6 months before and 12 months before the index dates of the second two alternative cohorts, so it is unlikely that patients would be stockpiling medications in the months around these index dates. By moving the timing of the inclusion criteria, we demonstrate that the “adherence spike” in the original cohort was likely attributable to an artifact induced by the specific historic medication use inclusion criteria. Furthermore, a similar adherence decline was observed across the four cohorts after the index date, possibly due to regression to the mean because of inclusion of recent medication users prior to the index date.
Figure 3.
Adherence Trends in the Original and Alternative Cohorts in “Same Criteria, Different Time” Analysis
These two sets of results suggest that greater attention is needed to identify a gold standard of inclusion criteria for identifying prevalent users to reduce the likelihood of an artifactual response to a copayment, intervention, or other policy change. It is critically important to replicate these findings in other health systems to determine whether this phenomenon is unique to veteran patients or is a more general consequence of this specific choice of historic medication use criteria. If this result is replicated in other health systems, then it suggests that adherence trends of prevalent users identified from administrative claims may increase prior to an index date and decline thereafter regardless of the lookback duration, the number of intervals in the lookback, and the minimum number of fills per interval. The criteria that defined a prevalent user if he/she had two or more fills in the 12 months prior to the copayment increase appears to be much more conservative because the adherence response was more muted, possibly because these criteria decreased the proportion of the sample that was recent fillers. Assessing the impact of different inclusion criteria may identify best practices for constructing prevalent cohorts in administrative data, which are critical in medication policy evaluation given the greater statistical power of prevalent users compared with incident user cohorts (Ray 2003).
The results of this study suggest that researchers using these or similar medication use inclusion criteria should consider three issues. First, a priori hypotheses regarding preperiod adherence changes are important to avoid confusing adherence changes that are an artifact resulting from the inclusion criteria from changes due to anticipatory behavior (e.g., stockpiling or switching medications). Having a priori hypotheses would inform how adherence just prior to the policy change may differ from the longer term preperiod trend, which could be evaluated by conducting “same criteria, different time” tests (Figure 3). That is, if a copayment increases significantly in a group of cost-conscious patients, one might expect significant medication stockpiling and a sizable “spike” in adherence in anticipation of the copayment increase. If the policy change is likely to have little impact on stockpiling, switching, or discontinuation, then one might expect that adherence in the period just prior to the policy change to be consistent with longer term preperiod adherence trends.
Second, researchers may want to think carefully about defining when the preperiod ends, when the postperiod begins, and whether a “transition period” is needed to wash out any artifact induced by the medication use inclusion criteria (Schneeweiss et al. 2002). The inclusion of a transition period may be particularly important for evaluations of short-term adherence because any adherence spike may bias short-term postperiod adherence, but it is unlikely to bias long-term postperiod adherence. In the original evaluation of the VA copayment increase (Maciejewski et al. 2010a), we did not include a transition period between the preperiod and the postperiod. As a result, the preperiod adherence estimate included the beginning half of this adherence spike and the decline in adherence after the index date. The inclusion of a transition period may have resulted in different short-term adherence estimates but similar longer term adherence.
Third, there may be value in considering novel ways of identifying prevalent user cohorts in administrative data with very long panels because patients are observed for many years or even decades (e.g., Medicare, VA). For example, it may be useful to identify a cohort of patients with a newly diagnosed condition or attestation of prescriber intention to initiate a medication for a newly diagnosed patient at some time in the (near) future. With many years of administrative data, a researcher could then identify the date of medication initiation and subsequent discontinuation or switches to alternative therapies. The subset of patients that do not discontinue the initial medication or switch to an alternative medication could then represent the cohort of prevalent users whose response to a policy (e.g., copayment) change could then be examined. This novel approach would likely require very large sample sizes for many years (even decades) because the proportion of patients that discontinue or switch medications would probably represent the majority of patients who were initially diagnosed. The exact proportion identified as prevalent users would vary by condition and drug class.
The study is subject to several limitations. The sample was drawn from four large geographically dispersed VAMCs, so these results may not generalize exactly to medication adherence trends of commercial, Medicaid or Medicare beneficiaries or patients with different conditions. However, every study of medication policy changes by prevalent medication users faces a similar choice about medication use inclusion criteria, so it is possible that a similar regression to the mean bias would be observed in these populations. Future research and perhaps new analyses of those prior studies are needed to validate these results in other settings and populations, as well as to confirm the assertions made in prior literature.
Rigorous claims-based evaluations of medication policy impacts on adherence require careful selection of several other factors: (1) an incident or prevalent cohort; (2) a control group; (3) the specific measure used to assess adherence (Hess et al. 2006); (4) duration of preperiod adherence assessment and expected preperiod medication trends; (5) duration of postpolicy follow-up; and (6) extent of covariate adjustment (Trygstad, Hansen, and Wegner 2006) and covariate balance in the presence of a control group. Different medication adherence measures have been used in cost-sharing evaluations, which partly explain variation in estimated adherence changes between studies (Hess et al. 2006). Adherence changes may also vary between studies due to evaluation of different populations (Maciejewski et al. 2010b) or different subgroups within a given population (Wang et al. 2011). Adherence estimates may also vary due to differences in quasi-experimental study design elements used to control for threats to internal validity, including use of a control group, as well as the equivalence of treatment and control groups and duration of preperiod adherence assessment (Shadish, Cook, and Campbell 2002).
Medication adherence measures derived from administrative data are being increasingly used as performance measures of providers and health systems by the Centers for Medicare and Medicaid Services, the National Committee for Quality Assurance and the Pharmacy Quality Alliance (PQA) (National Quality Forum 2011; PQA 2011), but how to define prevalent medication users has received less attention than seems merited. Medication use criteria has been considered in identifying incident users (Gardarsdottir, Heerdink, and Egberts 2006), but more work is needed to evaluate alternative medication use criteria for identifying prevalent users across a range of conditions and drug classes. For observational studies to be a fundamental building block of CER, it is imperative to develop validated standards for identification of prevalent user cohorts from widely available secondary datasets. As the pressure grows to generate consensus on methodology for CER based on observational study designs (Concato et al. 2010; Danaei, Tavakkoli, and Hernan 2012), inclusion criteria for identifying prevalent medication user cohorts merit greater attention to ensure that future medication policy evaluations generate unbiased, or at least less biased, results.
These results suggest that researchers may introduce an artifact in adherence trends depending on which definition they choose for identifying prevalent users. Additional research is needed to validate these results by comparing adherence trends with a consistent index date but varying the medication use criteria (duration of medication use, number of intervals, and number of fills per interval), and by comparing adherence trends in an analysis applying consistent medication use inclusion criteria but varying index dates. This combined approach would provide analysts a nuanced sense of the impact of medication use inclusion criteria on medication adherence estimates. These results suggest that more work remains to be done.
Acknowledgments
Joint Acknowledgment/Disclosure Statement: This work was supported by the Office of Research and Development, Health Services Research and Development Service, Department of Veterans Affairs, project number IIR 03-200. Dr. Maciejewski was also supported by a Research Career Scientist award from the Department of Veterans Affairs (RCS 10-391). The authors would like to acknowledge tremendously helpful comments and editorial suggestions from the editors, two reviewers, Hollis Weidenbacher and Jaclyn Lemon. Dr. Maciejewski has received consultation funds from Takeda Pharmaceuticals, Novartis, the Surgical Review Corporation, and the Research Data and Assistance Center (ResDAC) at the University of Minnesota, and owns stock in Amgen via his wife's employment at Amgen. The views expressed are those of the authors and do not reflect the views of the Department of Veterans Affairs and their affiliated institutions.
Disclosure: None.
Disclaimer: None.
SUPPORTING INFORMATION
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Appendix SA2: Sample Size of Prevalent Users from Different Criteria but Same Index Date
Appendix SA3: Sample Size of Prevalent Users from Same Criteria but Different Index Dates
References
- Blais L, Couture J, Rahme E, LeLorier J. “Impact of a Cost Sharing Drug Insurance Plan on Drug Utilization among Individuals Receiving Social Assistance”. Health Policy. 2003;64(2):163–72. doi: 10.1016/s0168-8510(02)00158-6. [DOI] [PubMed] [Google Scholar]
- Bryson CL, Au DH, Young B, McDonell MB, Fihn SD. “A Refill Adherence Algorithm for Multiple Short Intervals to Estimate Refill Compliance”. Medical Care. 2007;45:497–504. doi: 10.1097/MLR.0b013e3180329368. [DOI] [PubMed] [Google Scholar]
- Concato J, Lawler EV, Lew RA, Gaziano JM, Aslan M, Huang GD. “Observational Methods in Comparative Effectiveness Research”. American Journal of Medicine. 2010;123(12 suppl 1):e16–23. doi: 10.1016/j.amjmed.2010.10.004. [DOI] [PubMed] [Google Scholar]
- Danaei G, Tavakkoli M, Hernan MA. “Bias in Observational Studies of Prevalent Users: Lessons for Comparative Effectiveness Research from a Meta-Analysis of Statins”. American Journal of Epidemiology. 2012;175(4):250–62. doi: 10.1093/aje/kwr301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Doshi JA, Zhu J, Lee BY, Kimmel SE, Volpp KG. “Impact of a Prescription Copayment Increase on Lipid-Lowering Medication Adherence in Veterans”. Circulation. 2009;119(3):390–7. doi: 10.1161/CIRCULATIONAHA.108.783944. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fan VS, Maciejewski ML, Liu CF, McDonell M, Fihn SD. “Comparison of Risk Adjustment Measures Based on Self-Report, Administrative Data and Pharmacy Records to Predict Mortality and Hospitalization”. Health Services and Outcomes Research Methodology. 2006;6(1–2):21–36. [Google Scholar]
- Gardarsdottir H, Heerdink ER, Egberts AC. “Potential Bias in Pharmacoepidemiological Studies Due to the Length of the Drug Free Period: A Study on Antidepressant Drug Use in Adults in the Netherlands”. Pharmacoepidemiol Drug Safety. 2006;15(5):338–43. doi: 10.1002/pds.1223. [DOI] [PubMed] [Google Scholar]
- Goldman DP, Joyce GF, Zheng Y. “Prescription Drug Cost Sharing: Associations with Medication and Medical Utilization and Spending and Health”. Journal of the American Medical Association. 2007;298(1):61–9. doi: 10.1001/jama.298.1.61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hess LM, Raebel MA, Conner DA, Malone DC. “Measurement of Adherence in Pharmacy Administrative Databases: A Proposal for Standard Definitions and Preferred Measures”. Annals of Pharmacotherapy. 2006;40(7–8):1280–8. doi: 10.1345/aph.1H018. [DOI] [PubMed] [Google Scholar]
- Maciejewski ML, Liu CF, Fihn SD. “Performance of Comorbidity, Risk Adjustment, and Functional Status Measures in Expenditure Prediction for Patients with Diabetes”. Diabetes Care. 2009;32(1):75–80. doi: 10.2337/dc08-1099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Maciejewski ML, Liu CF, Derleth A, McDonell M, Anderson S, Fihn SD. “The Performance of Administrative and Self-Reported Measures for Risk Adjustment of Veterans Affairs Expenditures”. Health Services Research. 2005;40:887–904. doi: 10.1111/j.1475-6773.2005.00390.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Maciejewski ML, Bryson CL, Perkins M, Blough DK, Cunningham FE, Fortney JC, Krein SL, Stroupe KT, Sharp ND, Liu CF. “Increasing Copayments and Adherence to Diabetes, Hypertension and Hyperlipidemic Medications”. American Journal of Managed Care. 2010a;16(1):e20–32. [PubMed] [Google Scholar]
- Maciejewski ML, Farley JF, Parker J, Wansink D. “Copayment Reductions Generate Greater Medication Adherence in Targeted Patients”. Health Affairs (Millwood) 2010b;29(11) doi: 10.1377/hlthaff.2010.0571. [DOI] [PubMed] [Google Scholar]
- National Quality Forum. Washington, DC: National Quality Forum; 2011. “Compact Action Brief: A Roadmap for Increasing Value in Health Care. Improving Patient Medication Adherence: A $100+ Billion Opportunity.”. [Google Scholar]
- PQA. Washington, DC: Pharmacy Quality Alliance; 2011. “Executive Update on Medication Quality Measures in Medicare Part D Star Ratings.”. [Google Scholar]
- Preen DB, Holman CD, Spilsbury K, Semmens JB, Brameld KJ. “Length of Comorbidity Lookback Period Affected Regression Model Performance of Administrative Health Data”. Journal of Clinical Epidemiology. 2006;59(9):940–6. doi: 10.1016/j.jclinepi.2005.12.013. [DOI] [PubMed] [Google Scholar]
- Ray WA. “Evaluating Medication Effects outside of Clinical Trials: New-User Designs”. American Journal of Epidemiology. 2003;158(9):915–20. doi: 10.1093/aje/kwg231. [DOI] [PubMed] [Google Scholar]
- Roblin DW, Platt R, Goodman MJ, Hsu J, Nelson WW, Smith DH, Andrade SE, Soumerai SB. “Effect of Increased Cost-Sharing on Oral Hypoglycemic Use in Five Managed Care Organizations: How Much Is Too Much?”. Medical Care. 2005;43(10):951–9. doi: 10.1097/01.mlr.0000178216.23514.b7. [DOI] [PubMed] [Google Scholar]
- Schneeweiss S, Soumerai SB, Glynn RJ, Maclure M, Dormuth C, Walker AM. “Impact of Reference-Based Pricing for Angiotensin-Converting Enzyme Inhibitors on Drug Utilization”. Canadian Medical Association Journal. 2002;166(6):737–45. [PMC free article] [PubMed] [Google Scholar]
- Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin; 2002. [Google Scholar]
- Trygstad TK, Hansen RA, Wegner SE. “Evaluation of Product Switching after a State Medicaid Program Began Covering Loratadine OTC 1 Year after Market Availability”. Journal of Managed Care Pharmacy. 2006;12(2):108–20. doi: 10.18553/jmcp.2006.12.2.108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang V, Liu CF, Bryson CL, Sharp ND, Maciejewski ML. “Does Medication Adherence Following a Copayment Increase Differ by Disease Burden?”. Health Services Research. 2011;46(6 pt 1):1963–85. doi: 10.1111/j.1475-6773.2011.01286.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yin W, Basu A, Zhang JX, Rabbani A, Meltzer DO, Alexander GC. “The Effect of the Medicare Part D Prescription Benefit on Drug Utilization and Expenditures”. Annals of Internal Medicine. 2008;148(3):169–77. doi: 10.7326/0003-4819-148-3-200802050-00200. [DOI] [PubMed] [Google Scholar]
- Young BA, Lin E, Simon M, Von Korff G, Ciechanowski P, Ludman EJ, Everson-Stewart S, Kinder L, Oliver M, Boyko EJ, Katon WJ. “Diabetes Complications Severity Index and Risk of Mortality, Hospitalization, and Healthcare Utilization”. American Journal of Managed Care. 2008;14(1):15–23. [PMC free article] [PubMed] [Google Scholar]
- Zhang JX, Iwashyna TJ, Christakis NA. “The Performance of Different Lookback Periods and Sources of Information for Charlson Comorbidity Adjustment in Medicare Claims”. Medical Care. 1999;37(11):1128–39. doi: 10.1097/00005650-199911000-00005. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.