Evaluating the Impact of Sample Medication on Subsequent Patient Adherence

Abstract

BACKGROUND:

Medication nonadherence is problematic throughout health care practice. Patient nonadherence is a result of several factors, such as financial issues, confusion about the medication, or concerns about possible side effects. Efforts to improve adherence have been implemented, but new strategies are needed to ensure that patients fill their medication prescriptions and adhere to their prescribed use.

OBJECTIVE:

To investigate whether providing patients with a free 30-day supply of medication at the point of care via a dispensing kiosk—a secure, computerized cabinet placed in the prescriber’s office—that provides sample medication and educational materials had a measurable impact on adherence and health care cost.

METHODS:

The study sample consisted of patients drawn from the electronic health records of a large health care provider who were prescribed medications to treat diabetes, hypertension, and dyslipidemia. The comparison groups included a treatment group of patients who each received a 30-day generic sample of medication and a control group of patients who did not receive a sample. The study outcome was primary medication non-adherence (PMN), defined as whether a patient filled a prescription within 90, 180, or 365 days of prescribing. Only patients receiving a prescription for the first time were considered; patients on a medication before receipt of the sample were dropped. Postprescription medication adherence (PPMA), measured as proportion of days covered (PDC) and proportion of days covered ≥ 80% (PDC80), was also examined. Propensity score methods and multivariate regression models were used to examine the outcomes and group differences. Costs to the patient before and after the prescription were also analyzed. Key informant interviews were conducted with physicians, and qualitative analyses were performed.

RESULTS:

Patients who received a 30-day generic medication sample had a higher probability of filling a first prescription within 90 days (72.2% for treatment patients vs. 37.6% for controls, P < 0.001); 180 days (79.1% vs. 43.3%, respectively, P < 0.001); and 365 days (85.5% vs. 48.6%, P < 0.001). The medication sample had a positive effect on PDC for 90 days, with treatment patients having 72.8% adherent days versus 35.1% adherent days for controls (average treatment effect [ATE] = 37.5%, P < 0.001). At 180 days, PDC adherence was 57.1% for treatment patients versus 35.4% for controls (ATE = 21.5%, P < 0.001), and 43.6% versus 33.9%, respectively (ATE = 9.5%, P < 0.001) for the 365-day period. PDC80 was significantly better among treatment patients at 90 days (53.5% vs. 31.2%, respectively, ATE = 22.4%, P < 0.001) and 180 days (38.4% vs. 29.1%, ATE = 9.2%, P < 0.001), but not at 365 days (23.7% vs. 23.7%, ATE = -0.02, not significant). Costs were reduced by $395 for the treatment group. Interviews with clinicians indicated a positive view of the program.

CONCLUSIONS:

Providing a free sample medication improved the probability of patients filling their initial prescriptions and adhering to those medications. This program can affect health care costs, as evidenced by lower costs for the treatment group.

What is already known about this subject

• Despite substantial evidence for effectiveness of medications prescribed for chronic diseases such as diabetes and cardiovascular disease, large numbers of patients consistently fail to initiate a first fill of their prescriptions and remain adherent to those prescriptions.

• Failure to initiate a first fill of a prescription (PMN) occurs in as many as 22.5% of new prescriptions.

• Strategies that have been implemented to improve PMN include provider/pharmacist interventions, reducing medication costs, and promoting access to medications by providing medication samples at the initial prescription of a new medication.

What this study adds

• A persistent, increasing impact was seen in the probability of filling a first prescription for patients who received a free 30-day sample generic medication.

• Patients receiving free sample generic medications had higher adherence rates to their medications that was sustainable over time and lower total health care costs over the course of a year, compared with those patients who were only supplied a prescription.

• Clinicians who had access to the free 30-day sample generic medication program had an overall positive view of the program, with high marks given to the program’s usefulness to patients and clinicians.

Medication nonadherence is a costly problem that results in an estimated $100 billion to $300 billion in unnecessary health care costs per year.¹ Medication nonadherence is defined in terms of medication initiation, implementation, and persistence.² While rates of medication implementation and persistence are unacceptably low,³ an additional concern is that 22.5% of patients with electronic prescriptions do not initiate the medications they are prescribed.⁴ Reasons for nonadherence include financial considerations (including copay and travel time), fear of negative side effects, denial of the need for medication, and concern or confusion about the options for filling a prescription.⁵ It is difficult to design and implement strategies for improving medication adherence because prescription information is not shared between prescribers and pharmacists. Pharmacists are not aware of prescribers writing patients new prescriptions, and prescribers do not know when pharmacists fill patient prescriptions. Strategies have been implemented to improve primary medication nonadherence, such as provider/pharmacist interventions and reduction of medication costs (via generics in some cases), with mixed results.⁶

This study investigated whether providing a free 30-day supply of generic medication to what primary care physicians deemed as “high-risk” patients had a measurable and meaningful impact on medication adherence. The term “high risk” is based on an analysis of qualitative data collected via key informant interviews with prescribers who reported that they were likely to provide sample medications to patients who they evaluated as being at risk for poorer adherence (e.g., who were elderly or who were taking a larger number of medication) and/or requiring an immediate need for medication access (e.g., those with respiratory infections).

The medication sample was sponsored by UPMC Health Plan and dispensed from a kiosk (provided by MedVantx) that was located in the prescribing clinician’s office. Medication adherence was measured using 2 indices: the initial fill of medication, and the proportion of days covered (PDC) by the medication following receipt of the prescription. Quantitative analyses of these 2 outcomes were examined using propensity score methods. Total health care costs in the year before the medication sample or initial prescription, compared with the year following the prescription, were examined to determine the change in total health care costs to the patient. Qualitative analyses were conducted via key informant interviews with clinicians who used the medication sample program. The results of these qualitative analyses provided important justification for the study design and context for interpreting the quantitative results.

Methods

Study Design

This retrospective cohort study, which also used propensity score methods, compared medication adherence in patients who received free medication samples (treatment group) with patients who did not receive free samples (control group). The patient sample was drawn from a large regional integrated health care provider claims database of patients with a diagnosis of diabetes (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] code 250.xx), hypertension (401.xx), or hyperlipidemia (272.xx) in 2010 or 2011. Patients also needed to have a new prescription for at least 1 of 5 medication classes: calcium channel blockers (CCB), oral antidiabetic medications, beta blockers (BB), statins, and renin-angiotensin-aldosterone system medications (RAAS). Additionally, a dementia diagnosis at any time between 2010 and 2012 was considered an exclusion criterion (ICD-9-CM code 290.xx). This study received approval from the University of Pittsburgh Institutional Review Board (#PRO13020034).

Study Population

To compile the necessary data, MedVantx provided the University of Pittsburgh Medical Center (UPMC) with medication data for patients who received sample medications in 2011, which included the following identifying information: date of birth, first name, last name, and member ID. Since the study population was only composed of UPMC patients, all patients shared the same benefit structure. Demographic information, electronic prescription data, and diagnoses (2010-2012) were appended by the UPMC data management organization. Two data files were created with 1,732 patients: 974 patients who received sample medication (treatment group) and 758 patients who did not receive sample medication (control group). The data files were provided to an honest broker (HB) for deidentification before analyses by the research team. The HB de-identified and limited the data to those patients who met inclusion criteria (aged 19-65 years and diagnosis of diabetes, hypertension, or hyperlipidemia in 2010 or 2011) and excluded those with a diagnosis of dementia at any time. Inclusion diagnoses were identified using ICD-9-CM codes 272.xx (hyperlipidemia), 250.xx (diabetes), and 401.xx to 403.xx (hypertension). After this screening process and additional diagnostics to support the propensity score matching, 127 treatment group patients and 175 control patients were dropped from the sample. The resultant sample consisted of 847 treatment group patients and 583 control patients (Figure 1).

FIGURE 1.

Sample Selection Flowchart

Intervention

The intervention was the receipt of a free 30-day supply of medication from a primary care provider. Patients were not told that they were part of a study, and neither patients nor the providers knew of this study at the time. This sample medication program was sponsored by UPMC Health Plan and was carried out by using kiosks within primary care settings. Since there was no discernible pattern to the placement of the kiosks, the overall sample was not comprised of a specific study population and should be considered a nonrepresentative sample of UPMC patients. Moreover, physicians often supplied samples to patients they felt most at risk for poor adherence. The automated MedStart Connect cabinet offers automated sample medication reordering and printed patient education materials and captures sample medication usage data, which can be provided to payers and incorporated into members’ medication profiles. The cabinet can also interface with electronic health record and practice management systems.

Study Outcomes

Two outcomes were examined in this study: primary medication nonadherence (PMN) and postprescription medication adherence (PPMA). PMN was defined as 0 if a patient did not fill a new prescription within the 90-, 180-, or 365-day time periods following the prescription, or as 1 if a patient filled a new prescription at least once during those time periods. PPMA, expressed as PDC, was calculated as the total number of days supplied during an interval divided by the total number of days during that interval. PDC of 80% or greater (PDC80) was defined as 0 if a patient’s PDC was < 80%, or as 1 if a patient’s PDC was ≥ 80%, indicative of high adherence. PDC and PDC80 were calculated for 90, 180, and 365 days following the date of the initial prescription. It is important to note a difference between the standard PDC calculation used in this study and PDCs reported for PPMA in large patient populations using claims databases. Typically, population-level PDCs are calculated only for patients with at least 2 fills for a study medication in the observation period. However, this study has a less common study design (using a prescribing event as the starting point for outcomes and using a comparison sample) and an intervention that by definition influences the adherence outcome measure (a sample medication adds to days of supply). In order to best reflect the days supply available to patients, we chose to accumulate days supply from the time of the new prescription. Thus, the PPMA PDC calculation included patients with zero or otherwise lower supplies. Adherence was examined for 5 medication classes targeting diabetes, hypertension, and dyslipidemia, specifically CCBs, oral antidiabetics, BBs, statins, and RAAS medications. These medication classes were selected because they are commonly associated with chronic disease management.

Additional outcomes included the effect of the medication sample on total patient costs and qualitative outcomes, such as provider satisfaction, and experience using the medication sample program and kiosk. Health care costs data were obtained from a summary of all allowable charges in the health care and pharmacy claims data during the 365 days before and after the delivery of either the sample or the initial prescription. This outcome did not account for the cost of the kiosk or the cost of the medication samples. Key informant interviews were conducted with clinicians, using a structured facilitation instrument that gauged their opinions of the medication sample program, how they used the program, and suggestions for improvement.

Statistical Analysis

To improve the scientific rigor of this study, quantitative and qualitative methods were employed. Propensity score methods were used to evaluate PMN and PPMA and to reduce the possibility of selection bias associated with observable differences in patient characteristics. Multivariate regression models using control variables were also used to analyze PMN and PPMA. The propensity score framework created weights for the treatment group that reflected how well they served as comparisons to people similar to them in the control group. This framework controlled for observable characteristics (diagnoses, past adherence, and demographics), as well as polynomials and interaction terms for these variables. Propensity scores were determined by using a probit model with a binary indicator for treatment as the dependent variable and the set of observable differences as covariates.

The cost analysis outcome variable was the total cost of care to the payer. The covariates included were gender; age; the 5 drug class variables (CCBs, oral antidiabetics, BBs, statins, and RAASs); a past medication adherence variable (for all drug classes combined); past diagnoses of 3 core diagnosis variables (diabetes, hypertension, and dyslipidemia); and interactions of age/gender and diagnoses. Also included was a quadratic term for age. The appropriateness of these specifications was gauged by comparing c-statistics and propensity score overlap.

The resulting propensity scores were used to evaluate balance between the treatment group and the control group by comparing the means of the covariates within groups before and after weighting. Standardized differences (or Cohen’s D) assessed balance in the analysis. To account for remaining unbalanced differences, a doubly robust procedure was used and included inverse weighting and multivariate control variables. Robust standard errors were utilized, while repeated measures were not.⁷ This propensity score routine was used for all outcomes. Estimates were based on the DR command using STATA, version 13 (StataCorp, College Station, TX).

To qualitatively assess the utility of the sample medication program between high and low users of the program, individual key informant interviews were conducted telephonically with clinicians. Clinicians were read an informational script detailing the 10- to 15-minute length of the audiorecorded interview and anonymity of responses before their voluntary consent. The interviews used a standard instrument (Appendix A, available in online article) composed of 10 questions regarding the clinicians’ experiences with the sample medication program. The interviews were subsequently transcribed and analyzed using a modified grounded theory approach to find common and salient themes.⁸

Results

C-statistics measured the extent to which the probit model accurately assigned patients to the correct group based on the propensity score. The c-statistic for the probit model in this study was 0.659 (Appendix B, available in the online article). The comparison of propensity scores shows reasonable overlap or similarity of the patients in terms of their likelihood to have received medication samples. In other words, a large portion of the potential control group, based on their observable characteristics, are similar to patients who did receive a sample and can thus serve as valid comparisons.

The unweighted and weighted means of the baseline characteristics in the model and standardized differences represented as Cohen’s D are presented in Table 1. These data demonstrate relatively small differences when means were unweighted and had slight differences when the propensity score weighting was applied. In order for a covariate to be balanced across groups, it should have a standardized difference of no more than 10%.⁹ In this case, there were no standardized differences, unweighted or weighted, that were greater than 10%. Overall balance improved with weighting. Nonetheless, several variables, such as hypertension diagnosis, still stood out as somewhat different across groups. For this reason, the doubly robust framework was chosen, which incorporated this improvement in balance from the propensity score weighting and also used these standout variables as additional multivariate controls.

TABLE 1.

Unweighted and Weighted Covariate Means of Baseline Characteristics by Study Group: Means (SDs) and Cohen’s D

Variable	Unweighted (N = 1,430)			Weighted (N = 1,430)
	Control (n = 583)	Treat ment (n = 847)	Cohen’s D	Control (n = 583)	Treatment (n = 847)	Cohen’s D
Age	54.070	52.470	1.673	53.206	52.041	1.226
	(9.916)	(9.317)		(10.165)	(9.022)
Agea	3,021.768	2,839.785	1.933	2,925.685	2,790.229	1.452
	(995.434)	(901.906)		(1,011.644)	(874.233)
Female	0.475	0.577	-2.054	0.520	0.610	-1.765
	(0.500)	(0.494)		(0.531)	(0.495)
Any diabetes 2010	0.081	0.071	0.378	0.068	0.073	-0.190
	(0.272)	(0.257)		(0.266)	(0.262)
Any RAAS 2010	0.226	0.187	0.970	0.207	0.174	0.820
	(0.419)	(0.390)		(0.435)	(0.378)
Any statin 2010	0.184	0.155	0.779	0.165	0.149	0.440
	(0.387)	(0.362)		(0.386)	(0.349)
Any CCB 2010	0.084	0.070	0.529	0.078	0.065	0.526
	(0.278)	(0.255)		(0.266)	(0.233)
Any BB 2010	0.129	0.130	-0.030	0.126	0.132	-0.169
	(0.335)	(0.336)		(0.362)	(0.349)
Past PDC: all drugs	38.075	37.284	0.087	36.241	37.943	-0.179
	(93.579)	(89.777)		(95.906)	(94.382)
Any diabetes diagnosis 2010	0.125	0.033	3.647	0.057	0.019	2.497
	(0.331)	(0.179)		(0.193)	(0.116)
Any hyperlipidemia diagnosis 2010	0.254	0.109	3.950	0.156	0.081	2.539
	(0.436)	(0.311)		(0.338)	(0.262)
Any hypertension diagnosis 2010	0.288	0.107	4.827	0.173	0.070	3.671
	(0.453)	(0.310)		(0.338)	(0.233)
Age × any diabetes diagnosis 2010	6.878	1.792	3.612	3.055	1.044	2.567
	(18.538)	(9.907)		(9.706)	(6.228)
Age × hyperlipidemia diagnosis 2010	14.238	5.949	4.008	8.576	4.407	2.639
	(24.825)	(17.260)		(18.326)	(13.795)
Age × any hypertension diagnosis 2010	15.561	5.616	4.876	9.085	3.629	3.783
	(25.000)	(16.500)		(17.892)	(11.438)
Female × any diabetes diagnosis 2010	0.063	0.015	2.633	0.029	0.007	2.466
	(0.244)	(0.123)		(0.121)	(0.058)
Female × hyperlipidemia diagnosis 2010	0.101	0.063	1.417	0.078	0.053	1.081
	(0.302)	(0.242)		(0.266)	(0.204)
Female × any hypertension diagnosis 2010	0.149	0.073	2.502	0.108	0.054	2.224
	(0.357)	(0.261)		(0.290)	(0.204)

^a Age squared, a variable used in the statistical modeling.

BB = beta blocker; CCB = calcium channel blocker; PDC = proportion of days covered; RAAS = renin-angiotensin aldosterone system; SD = standard deviation.

Results from the doubly robust propensity score model for PMN are presented in Table 2. In addition to these results, the average treatment effect was also calculated from a simple regression model using the same covariates as the doubly robust model. These treatment effect estimates on PMN show that treatment group patients had a higher probability of filling a first prescription through a community or mail order pharmacy. This was the case for 90 days, 180 days, and 365 days (all time periods exhibited high statistical significance, with P < 0.001). The greatest treatment effect was observed at 365 days; however, the effect sizes were similar at all 3 time periods. These similar effect sizes were in contrast to patient adherence levels. Adherence levels increased over time for both groups as some patients eventually filled their first prescriptions.

TABLE 2.

Estimates of Primary Medication Nonadherence Following an Index Prescription over 90, 180, and 365 Days: Results from a Doubly Robust Propensity Score Model (N = 1,430)^a

Number of Days Since Index Prescription	Regression Coefficient (%)	Z (P Value)	Adjusted Nonadherence Among Control Group Patients (%)	Adjusted Nonadherence Among Treatment Group Patients (%)	Average Treatment Effect (%)	Doubly Robust Z (P Value)
90	-35.0	-12.6 (< 0.001)	62.4	27.8	-34.4	-12.7 (< 0.001)
180	-35.8	-13.4 (< 0.001)	56.7	20.9	-35.3	-13.3 (< 0.001)
365	-36.7	-14.3 (< 0.001)	51.4	14.5	-36.9	-14.3 (< 0.001)

^a Covariates: age; gender; any medications 2010 (diabetes, RAAS, statins, calcium channel blockers, beta blockers); past PDC (all medications); any diabetes diagnosis 2010; any hyperlipidemia diagnosis 2010; any hypertension diagnosis 2010; and interactions (age × 3 diagnoses variables, gender × 3 diagnoses variables).

PDC = proportion of days covered; RAAS = renin-angiotensin-aldosterone system medications.

Results for PPMA are presented in Table 3. These data demonstrate that providing a sample had a positive effect on PDC for 90 days, 180 days, and 365 days. All of these results were statistically significant (P < 0.001). The largest treatment effect was in the first 90 days, with a 0.375 higher PDC. The PDC80 results for 90 and 180 days were also statistically significant (P < 0.001), with 0.224 and 0.092 more patients being highly adherent, respectively. The only outcome without a treatment effect was PDC80 for 365 days, which was nearly the same between the comparison group and the treatment group.

TABLE 3.

The Effect of Medication Samples on Postprescription Medication Adherence: Results from a Doubly Robust Propensity Score Model (N = 1,430)^a

Number of Days Since Initial Script	Regression Coefficient (%)	Z (P Value)	Adjusted PDC Among Control Patients (%)	Adjusted PDC Among Treatment Patients (%)	Average Treatment Effect (%)	Doubly Robust Z (P Value)
90	37.3	18.0 (< 0.001)	35.1	72.8	37.5	17.3 (< 0.001)
180	21.3	10.0 (< 0.001)	35.4	57.1	21.5	9.8 (< 0.001)
365	9.4	4.6 (< 0.001)	33.9	43.6	9.5	4.5 (< 0.001)
Number of Days Since Initial Script	Regression Coefficient (%)	Z (P Value)	Adjusted Proportion of PDC80 Patients Among Control Patients (%)	Adjusted Proportion of PDC80 Patients Among Treatment Patients (%)	Average Treatment Effect (%)	Doubly Robust Z (P Value)
90	22.1	8.1 (< 0.001)	31.2	53.5	22.4	8.4 (< 0.001)
180	8.9	3.4 (0.010)	29.1	38.4	9.2	3.5 (< 0.001)
365	-0.3	-0.14 (0.885)	23.7	23.7	-0.02	-0.01 (0.994)

^a Covariates: age; gender; any medications 2010 (diabetes, RAAS, statins, calcium channel blockers, beta blockers); past PDC (all medications); any diabetes diagnosis 2010; any hyperlipidemia diagnosis 2010; any hypertension diagnosis 2010; and interactions (age × 3 diagnoses variables, gender × 3 diagnoses variables).

PDC = proportion of days covered; PDC80 = proportion of days covered 80% or greater; RAAS = renin-angiotensin aldosterone system.

Cost analysis results indicate a difference in costs between the matched groups from the year before the sample delivery or initial prescription to the following year. The treatment group was found to have a savings of $395 per patient per year, with the average costs for the treatment group of $3,419 versus $3,815 for the matched controls. This amount reflects only health care cost savings for the patient; it does not take into account the cost of the sample or the cost of the kiosk. While the difference was not statistically significant, the results are nonetheless suggestive of an overall savings in total costs over the course of the year for those who received a sample medication.

The sample medication program was integrated within medical offices for this study. Clinicians involved with the program were physicians, physician assistants, and nurse practitioners. Key informant interviews were conducted individually with 10 clinicians who used the medication sample program. Those selected for the key informant interviews were 5 clinicians who used the program often (“high” users, defined as being in the 80th percentile or greater for total number of samples given to patients) and 5 clinicians who did not use the program as often (“low” users, defined as being in the 20th percentile or lower for total number of samples given). Common and salient themes from each interview question response were derived and summarized (Figure 2).

FIGURE 2.

Summary of Common Themes from Key Informant Interviews

The majority (90%) of clinicians interviewed expressed a positive view of the program. High users of the program remarked on the ease of implementing the program into the office and its convenience. Low users expressed satisfaction with having the actual medication bottle present to supplement explanations of new medications with the patient. Both high and low users were aware of the primary intention of the program—to provide medication samples during a patient’s office visit—and found the program important to promoting discussion between clinician and patient as a strategy for improving adherence. One high user pointed out that patients need to make sure they understand the importance of taking their medications, potential side effects, and the importance of notifying their physicians of side effects. A low user stated that the program allowed the patients to become partners in treating and managing their health problems. Other adherence strategies expressed by clinicians included promoting patient follow-up by asking patients to bring in their medicine bottles, so the clinician could review their electronic health records or insurance information.

Clinicians also indicated that they found the program particularly useful with helping a patient obtain a new medication or switch dosage of a medication and that this could not only potentially increase confidence with the medication in the long term but also reduce patient costs. Costs were an important consideration as financially burdened or elderly patients could be targeted by clinicians to receive medication samples. Results from the qualitative analysis revealed that this was the case more frequently with low users of the program, suggesting that low users were more selective, while high users offered to all patients. Low users did point out that awareness of all medication samples offered by the program via a catalog or user training could influence their likelihood of using the program more frequently.

Discussion

The results of the propensity score weighting analyses revealed a robust impact of receiving a medication sample, with an approximate 35% improvement in the probability of filling the first medication. In addition, the statistical analyses showed a significantly higher adherence rate for those patients receiving a medication sample over all 3 time periods: 90 days, 180 days, and 365 days. At 90 days, the difference in PDC was 0.375, which translates to approximately 34 more pills. It should be noted that, in the case of 90 days, 30 of the 34 pills were the free sample. Therefore, the 4 additional pills represent the secondary impact of receiving a sample. At 180 days, the secondary impact was approximately 9 pills, which implied that the receipt of a sample had a persistent and even increasing effect up to 6 months later. The effect diminished to approximately 4.5 pills by 365 days.

With regards to PDC80, patients who received a sample had a higher PDC80 at 90 days and 180 days than patients who did not receive a sample. These differences were substantial; at 180 days, 9.2 percentage points higher PDC80 was achieved, which reflects an additional 132 people with high adherence. However, by 365 days, this effect had receded, and PDC80 rates were almost exactly the same. Therefore, PDC80 at 1 year was the only measure where the treatment did not have an overall positive effect but was simply equivalent rather than an overall negative effect.

The results of the health care costs analyses suggested a possible savings of nearly $400 in total costs over the course of the year for those who received a sample medication, not including the cost of the sample. It cannot be ruled out that the lower health care cost of the patients receiving a sample was because of random variation. While propensity score matching reduces this potential bias, it may not eliminate the bias in a sample of this size. This is a common challenge for studies of health care costs, whose distribution can be skewed and contain large uncommon cost events. A study involving larger patient samples and a broader set of medication classes might result in significant reductions in health care costs.

The key informant interviews conducted with clinicians who were high and low users of the program showed no systematic differences between the 2 groups, based on factors such as prescribers’ overall use of the program and the types of patients with whom they used the program. However, high users of the program provided more comments related to its usefulness, which could be reflective of frequent use of the program. Providing training and communication to clinicians may be beneficial for low users and may help these users to recognize the program’s benefits, thereby increasing use of the program.

Limitations

There are some limitations to this study that need to be considered. First, PDC remained significantly higher at 365 days, while PDC80 was no longer higher. This could be explained by attrition of patients in the real world. By the end of 365 days, a nontrivial, but unknowable, portion of the patients had “left the study”—they moved, changed health plans or pharmacies, or were deceased. It was not possible to know whether patients had stopped filling their medications or whether they had simply left the study. This uncertainty was because there was no study for them to officially “leave”; patient data were observed after the fact, using administrative claims data. Such a limitation suggests that the 365-day results were not completely indicative of a decrease in the effect of the samples but may simply reflect a loss of subjects that would cause high PDCs at 180 days to drop below the 80% threshold 6 months later.

Second, it should be noted that the design of this study does not obviate the possibility of selection bias, since the treatment group patients might have been systematically different than the control group patients. The patients might have different characteristics that could affect the validity of the study comparisons. Regarding the overall sample, the total of almost 1,800 patients might have been lower than expected. From that total sample, it should also be acknowledged that the treatment group was larger than the control group, albeit not large enough where it would affect the results. Although the sample size used was large enough to answer the proposed questions, one might assume that a larger sample of patients would be pulled from a big health plan with 3 large chronic disease states. However, it must be considered that the total sample comprised a relatively narrow population, since the study population only consisted of patients who had not filled a prescription before receiving the medication sample.

Moreover, the recipients of samples (at discretion of their prescribing providers) may reflect a more at-risk patient population. Additionally, care must be taken when comparing these results with other studies of adherence. PPMA typically refers to population-level performance outcomes that are a subset to patients who are already at least partially adhering to study medications because of the 2+ fill inclusion criterion. This inevitably creates a lower bound on PDC calculations, (e.g., two 30-day supply fills mean PDCs of at least 16% [60/365]). In contrast, these results include all patients regardless of having any fills, which necessarily lowers the PDC results.

A third limitation arises regarding how patients filled their prescriptions, whether it be through a community pharmacy or a mail order pharmacy. In the case of mail order pharmacies, if a patient gets a 90-day supply, the 90-day measurement used in this study will show a more positive result as compared with someone who had to fill multiple times within a 90-day period. Therefore, there could be less behavioral improvement for those who received a 90-day mail order supply. Also, it is not possible to detect whether a prescription is mail order or not, so someone could delay shipment and delivery of a prescription after a set period of time (e.g., after 1 month). This delay would also affect behavioral improvement. These limitations do not introduce sources of bias, since the control and treatment groups could be composed of these types of patients.

Finally, key informant interviews were not conducted with clinicians who did not use the program and did not have access to it in their offices. Comparing patients of clinicians who use the program to its fullest extent with patients of clinicians who have access to the program but do not use the program could provide a useful juxtaposition. Also, in this study, only 10 clinicians were interviewed. It would have been beneficial to gather more interviews to better understand the prescribers who use the program. These interviews could be used to develop methods to increase provider awareness and confidence in using the program, which could lead to an improvement in the intervention and help improve medication adherence outcomes even further.

Conclusions

All forms of medication nonadherence are attributed to between $100 and $300 billion of avoidable health care costs in the United States annually, representing 3%-10% of total U.S. health care costs^1,10; thus, improving adherence could have a huge impact on health care costs. The results of this study indicate that medication sample programs, such as the one described, were associated with significantly higher probabilities of patients filling their prescriptions (initiation) and subsequent better adherence to their medication regimens (persistence). Along with the propensity score framework, which helped to control for the differences between the treatment and control groups, the overall differences in adherence observed between the study groups can be considered robust. Finally, common themes from interviews with clinicians using the program revealed no systematic differences between high and low users. This finding provides justification of the quantitative results by removing this possible source of bias.

APPENDIX A. Key Informant Interview Instrument

MedVantx Study Key Informant Interview

The University of Pittsburgh Program Evaluation and Research Unit would like to conduct an interview with you regarding your experience with the MedVantx program. MedVantx, which provides medication samples via a kiosk at your practice site, has provided the University of Pittsburgh with contact information for clinicians who have used the program. We are seeking input to learn about your experience with MedVantx. We will record this interview, but your responses will be recorded anonymously. Therefore, your identity will not be disclosed to MedVantx. Your participation in this interview, which will take approximately 15 minutes of your time, is voluntary and you retain the right to withdraw at any time. There are no foreseeable risks associated, nor are there any direct benefits to you.

Do I have your permission to proceed with the interview?

YES _____ NO _____

1. How long have you been practicing medicine?

2. What types of strategies do you utilize to ensure medication adherence with your patients?

3. How long have you been involved in the MedVantx program?

4. Please describe your experience with the MedVantx program in general.

5. Please describe the usefulness of the MedVantx program. Do you find it useful? Why or why not?

6. How often did you utilize the MedVantx program for your patients in 2011?

   Choices: 0-25, 25-100, 100-300, 300-500, 500 or more

7. Please describe how your practice integrated the MedVantx program.

   1. What were the reasons for your practice’s utilization of the MedVantx program?
      OR
   2. What were the reasons for your practice not fully utilizing the MedVantx program?

8. What types of patients did you use the MedVantx program for (e.g., financially challenged, patients on multiple medications, patients with limited mobility, etc.)? What types of patients did you not use the program for?

9. How do you feel the MedVantx program could be made scalable across all community medical practices?

10. How and why has your use of the MedVantx program changed from 2011 to the present?

APPENDIX B. Distribution of Propensity Scores

The c-statistic for the probit model in this study was 0.659. Based on observable characteristics, the majority of patients were likely to be in the treatment group (most of the mass is to the right of the graph above a 0.4 probability of being in the treatment group). This suggests that there is no extreme difference in the study groups.