Development and Validation of an Index to Predict Personal Prescription Drug Importation by Adults in the United States

Abstract

Objective

Personal prescription drug importation (PPDI) is prevalent in the United States (U.S.) because of the high cost of U.S. medicines and lower cost of foreign equivalents. The practice carries the risk of exposure to counterfeit, adulterated, and substandard medicines. No known tools are available for predicting person-level PPDI risk. The objective of this study was to develop and validate a predictive PPDI index for policymakers, researchers, and clinicians.

Methods

Using 2011 and 2012 National Health Interview Survey (NHIS) data as the development and validation cohorts respectively, we identified predictors, built multivariable logistic regression models, and validated the index by comparing predicted risk of PPDI in the development cohort to the observed risk in the validation cohort. We assessed calibration using the Hosmer-Lemeshow goodness-of-fit test and discrimination with C-statistics. The outcome measure was survey-reported PPDI (1=yes; 0=no).

Key Findings

In the development cohort, prevalence of PPDI in respondents with 0–2, 3, 4, 5–6, or ≥7 risk factors were 0.32%, 0.57%, 1.09%, 2.95%, and 13.67% (C-statistic=0.78), and in the validation cohort, were 0.32%, 0.54%, 0.95%, 2.89%, and 10.80% (C-statistic=0.76). The Hosmer-Lemeshow test indicated absence of a gross lack of fit (P=0.58) in the validation cohort. On the basis of index performance in the validation cohort, if an intervention to reduce importation were applied to all patients with scores of ≥7, it would be applied to 31.1% of patients who engage in PPDI and 0.6% of the overall population.

Conclusion

This predictive index accurately stratifies U.S. adults into groups at differential risk of PPDI and may provide value to those who are responsible for health policy and regulation of pharmaceutical importation.

Introduction

As a result of the high cost of domestic prescription drugs and lower cost of products from foreign countries, personal prescription drug importation (PPDI) remains a critical health policy issue in the United States (U.S.) (1–4). Some have proposed greater use of domestically sold generic drugs as a more viable solution to the challenge of rising prescription drug costs (5–7). Nevertheless, despite increased generic drug utilization over time, U.S. consumers continue to import foreign prescription drugs for personal use from Canada and other countries by ordering them from Internet sources or carrying them across borders (8–10). This practice is potentially hazardous because foreign medications are more likely to be counterfeit, adulterated, or otherwise substandard (7, 11–13).

Policymakers need current data that characterize consumers who import and tools to predict who is most likely to engage in PPDI. Information on factors that prompt importation aside from cost, such as difficulty accessing health care or low health literacy, (14–17) will guide risk mitigation strategies and allow regulators to improve importation policies that help underserved populations.

A PPDI index could efficiently stratify individuals into groups with different risks of importing medicines. There are several reasons why an index would be useful. First, a simple index is more intuitive to use than an array of predictive characteristics (18). Second, policymakers and researchers could readily employ an index for targeting subpopulations in which to implement studies clarifying the causal mechanisms behind the decision to import. Third, policymakers could use an index to focus resource-efficient interventions designed to minimize the potential risks of importation and improve access to affordable prescription medicines. Fourth, researchers could use a validated index to improve inferences about the effects of interventions to reduce PPDI (19, 20). Unfortunately, risk factors for PPDI have not been well-documented and no index is available.

Our objectives were to identify predictors of PPDI among adults living in the U.S in 2011 and 2012 and to develop and validate a practical predictive index that differentiates individual risk of importing prescription drugs for personal use. We considered a priori variables from conceptual domains that could be easily obtained from basic demographic data and a short verbal or written questionnaire. To assess the generalizability of our index for use by policymakers and public health professionals, we developed and validated it using data from two large, nationally-representative U.S. samples.

Methods

Subjects

The study data came from the 2011 and 2012 National Health Interview Survey (NHIS), an in-person survey conducted annually by the National Center for Health Statistics (NCHS) of the Centers for Disease Control and Prevention (CDC) (21). The survey is intended to “monitor the health of the U.S. population through the collection and analysis of data on a broad range of health topics” and is designed to represent the civilian noninstitutionalized population residing in the United States (21). Populations excluded are individuals in long-term care institutions; patients in hospitals for the chronically ill, physically disabled and intellectually disabled persons; inmates of correctional facilities; active-duty Armed Forces personnel; and U.S. nationals living in foreign countries (21). Identification of the sample involves multistage cluster sampling of households from across the country. A field representative administers a questionnaire consisting of four main components: Household Composition, Family, Sample Adult, and Sample Child (21). According to the NCHS, “in very rare instances where the sample adult was not able to respond for him or herself, a proxy was used”(21). Black, Hispanic, and Asian populations, particularly adults aged 65 years and older, are oversampled to provide more precise estimates for these growing populations (21).

Our development sample is comprised of adults ≥18 years old who participated in the 2011 Sample Adult questionnaire (n=33,014) (21). The 2011 conditional response rate, which is the number of completed Sample Adult questionnaires divided by the total number of eligible sample adults (n=40,496), was 81.6%. The unconditional (i.e., final) response rate for the 2011 Sample Adult questionnaire was calculated by the NCHS as the product of the conditional response rate and the overall family response rate of 81.3%, yielding a final Sample Adult response rate of 66.3%.

Our validation sample is comprised of adults ≥18 years old who participated in the 2012 Sample Adult questionnaire (n=34,515). The 2012 conditional response rate was 79.7% (34,515/43,323). The unconditional response rate for the 2012 Sample Adult questionnaire, calculated using the family response rate of 76.8%, was 61.2%. Additional information on the NHIS is available at http://www.cdc.gov/NCHS/NHIS.htm.

Measures

Outcome

Our primary outcome was importation of one or more prescription drugs for personal use from a foreign country. The operational expression of this outcome was a dichotomous variable derived from the survey question: “During the past 12 months, are any of the following true for you? … You bought prescription drugs from another country to save money”(21). Respondents who replied, “Yes,” were classified as importers. All individuals in the sample adult survey component were asked the question, not just those who were taking prescription drugs.

Predictors

We considered variables that we selected a priori based on substantive knowledge and available literature (7, 22–24). We classified predictors into four conceptual domains: sociodemographic characteristics, diagnoses and health status, access to medical care, and health-related Internet usage. Sociodemographic characteristics included age, sex, race, Hispanic ethnicity, highest educational attainment, current employment status, household income-to-poverty ratio, citizenship status, region of residence, and travel to developing countries.

We derived variables measuring access to medical care from responses to questions on health insurance coverage, total family out-of-pocket (OOP) medical expenses, and trouble finding a doctor. Health insurance status is a four-level variable: no coverage, public only, private only, and both public and private insurance. We created this variable using the responses to a series of questions about insurance type. Total annual family OOP medical expenses for medical and dental care, excluding health insurance premiums and over-the-counter medications, were categorized as zero U.S. dollars, <$500, $500–1999, and ≥$2000.

Self-reported general health status and comorbid conditions were utilized to measure health status. Self-reported health status was ascertained with the question: “Would you say that your health in general was excellent, very good, good, fair, or poor?” Proxy responses were accepted for adults unable or unwilling to answer the question. Comorbid conditions were measured by asking respondents if a doctor had ever told them that they had a series of conditions. The following conditions were of interest: myocardial infarction, heart failure, peripheral vascular disease, cerebrovascular disease, dementia, chronic pulmonary disease, connective tissue disease, ulcer disease, liver disease, diabetes, cancer, and renal disease. To assess Internet usage over the past 12 months, we developed dichotomous variables for looking up health information on the Internet, using a computer to fill a prescription, and using online chat groups to learn about health topics.

Statistical Analysis

Index Development

Initially, we used χ² tests to inspect the relationship between each suspected predictor and dichotomous outcome PPDI in the development cohort. We then performed multivariable logistic regression modeling in several stages. First, we constructed a model that included predictors associated with PPDI in the bivariable analyses (P < 0.05). Second, we created individual models for each domain of predictors: sociodemographic characteristics, access to medical care indicators, diagnoses and health status indicators, and health-related Internet usage indicators.

Within each domain, the variables were entered into three different models for the outcome PPDI: a standard model using survey weights, a stepwise backward model without survey weights, and a selection logistic model without survey weights. Although we checked correlation of the a priori selected variables using correlation matrices prior to regression, backward and stepwise selection helped ensure that the predictors were not highly correlated (24). We selected a P-value criterion of <0.05 for the stepwise regression models because many variables were statistically associated with importation, but weak predictors. Survey weights are incompatible with backward elimination and stepwise regression models and therefore were not applied for these models.

Using results from the models specific to each domain to guide variable selection, we constructed a final model using variables from all domains. Given the limitations of stepwise regression, such as artificially small p-values and standard errors of parameter estimates due to multiple comparisons, we considered the statistical significance of variables from the domain-specific models alongside our own hypotheses when constructing the final model (25). We used a final backward elimination stepwise logistic regression model with variables from all domains to verify variable selection and applied Akaike’s Information Criterion (AIC) and Schwarz’s Bayesian Information Criterion (BIC) to quantify the strength of the prediction (26–28). Both the AIC and BIC penalize the log-likelihood of the model by a factor proportional to the number of predictors in the model, but the BIC imposes a greater penalty. The effect is that these criteria set higher thresholds for inclusion of new variables into the models with the aim of balancing improvements in predictive ability through inclusion of more predictors with the increased model complexity of such additions.

The variables in the final model were then included in our index. We tested pre-defined interaction terms between U.S. citizenship status and Hispanic ethnicity, Hispanic ethnicity and health insurance status, and U.S. citizenship and health insurance status (24).

We sought to have parsimonious prediction models for two reasons. First, a parsimonious model is more easily interpreted by policymakers and public health professionals and more likely to be used (29). Second, inclusive models can be over-fitted to the peculiarities of the data and begin to erode precision of the estimates (24). To examine the degree to which variable reduction decreased the discrimination of our predictive index, we calculated the area under the receiver operating characteristic (ROC) curve, also known as the C-statistic, at each modeling stage (24). The C-statistic is the probability that for each possible pair of respondents in which one engaged in PPDI and the other did not, the index assigned a higher risk of engaging in importation behavior to the subject who actually did engage in PPDI. We used a scoring system similar to that used by D’Agostino and colleagues in the Framingham Heart Study (30). For this system, we assigned one point to each significant level of the multi-category predictors and then summed the points into a score for each respondent. An individual can accrue a maximum of one point for each predictor since individuals cannot be present in more than one category of a predictor.

Index Validation

To validate the index, we applied the model created in the 2011 NHIS development cohort to the validation cohort from the 2012 NHIS. We assessed the calibration of the index—the agreement between the fitted and observed risks—by comparing the predicted risk of PPDI from the development cohort to the observed risk in the validation cohort. Calibration of the final model was also checked using the Hosmer-Lemeshow goodness-of-fit test with 10 groups, which compares the predicted and observed events within deciles of the predicted risks (24). We assessed the discrimination of the index—how effectively the model can distinguish between PPDI and non-PPDI occurrence—by calculating the C-statistics for the development and validation cohorts.

Sensitivity Analyses

We examined several additional models to determine whether the discrimination of our final model would have differed with the selection of other variables. First, since there was some collinearity between predictors prior to modeling, we created models that excluded the strongest predictor in each domain and allowed alternative predictors to enter the model during stepwise regressions in place of these variables. We compared the discrimination of this alternative model to the final model. Second, we repeated our analyses classifying respondents with missing PPDI outcome data as engaged in PPDI. In addition to the one-point scoring system that we selected, we also examined two other scoring systems: 1) assigning points in proportion to the beta coefficients, and 2) calculating risk scores directly from the multivariable model.

Comparative Risk

To examine how risk of PPDI differs from one level of the index to the next, we utilized simple logistic regression modeling of the index with a backward difference coding system (31). Backward difference coding allows for comparisons between adjacent levels of the index and provides the association for each level minus the prior level of the index (24, 31). The mean of one level of the categorical variable is compared to the mean of the dependent variable for the prior adjacent level. Backward difference coding is thus particularly useful for an ordinal outcome like the PPDI index (31).

All analyses were conducted with Stata version 12.1 (StataCorp LP, College Station, TX, USA) using procedures to account for the complex sampling design and weighting of the NHIS. This study used de-identified publicly available data and therefore did not require approval of an institutional review board.

Results

Characteristics of Respondents

The mean age of respondents in the development cohort was 46 years (Table 1). Nearly half (48.3%) were male and 24.1% reported Hispanic ethnicity. Common comorbid conditions were hypertension (32.4%), asthma (12.4%), arthritis (24.8%), diabetes (9.8%), and cancer (8.7%). The mean age of respondents in the validation cohort was 46 years. Nearly half (48%) were male and 22.8% reported Hispanic ethnicity. As in the development cohort, common comorbid conditions were hypertension (32.8%), asthma (12.7%), arthritis (23.9%), diabetes (10.2%), and cancer (9%). The overall risk of PPDI was 2.2% in the development cohort and 2.0% in the validation cohort.

Table 1.

Sociodemographic, access to medical care, health status, and Internet usage characteristics: National Health Interview Survey, 2011 (Development Cohort) and 2012 (Validation Cohort)

Characteristic	Development Cohort (n=33,014) %	Validation Cohort (n=34,525) %
Sociodemographic
Age, mean (SD)	46 (17.6)	46 (17.6)
18–44	49	48
45–64	34.6	35.1
≥65	16.7	16.9
Male Sex	48.3	48
Region of Residence
South	36	35.3
Northeast	16	17.1
Midwest	20.4	19.6
West	27.6	28
Race
White	66.1	67.9
Black	15.2	14.9
Other a	18.7	17.2
Hispanic Ethnicity	24.1	22.8
Highest Education
Completed college b	28.5	28.8
Some college	15.4	15.7
High school completed	22	22.2
Less than high school	34.1	33.3
Current Employment Status
Employed	57.4	58.1
Unemployed	6.6	6
Not in labor force	36	35.9
Household Income-to-Poverty Ratio
Not poor (≥2)	51.9	50.5
Near poor (1 to <2)	16.8	18.2
Poor (<1)	16.3	16.5
Did not report income	15	14.7
U.S. Citizen	90.5	91.2
Travel to Developing Countries c	32.6	32.0
Medical Care Access
Health Insurance Status
Private only	46.6	45.2
Public only	26.7	27.4
Public and private	6	6.2
None	20.7	21.2
Family OOP Medical Expenses in the Past Year (U.S. Dollars)
<$500	36.8	34.6
$500 to <$2,000	29.7	30.2
$2,000+	19.7	22.4
None	13.9	12.9
Trouble Finding a Doctor	3.4
Health Status
Self-reported Health Status
Excellent/very good	64	64.4
Good	25.2	24.7
Fair/poor	10.8	10.9
Comorbid Conditions
Myocardial infarction	3.7	3.7
Heart condition	8	7.5
Heart disease	5.4	5.3
Angina pectoris	2.4	2.2
Stroke	3.2	3.2
Hypertension	32.4	32.8
Asthma	12.4	12.7
Emphysema	2.2	1.9
Arthritis	24.8	23.9
Ulcer	7.3	7
Liver disease	1.6	1.5
Diabetes	9.8	10.2
Cancer	8.7	9
Renal disease	2.2	2
Health-related Internet Usage
Looked up health information on the Internet	43.3	40.2
Filled a prescription on the Internet	6.4	6.2
Used online chat groups to learn about health topics	3.6	2.9

^aAmerican Indian/Alaska Native, Asian, and Native Hawaiian or other Pacific Islander.

^bCompleted college, graduate degree, or professional degree.

^cTravel outside the U.S. since 1995 other than Canada, Europe, Japan, Australia, and New Zealand.

Abbreviations: SD=standard deviation; Ref=reference; U.S.=United States; OOP=out-of-pocket.

Predictors

Most predictors were statistically associated with PPDI (P<0.05; Table 2) except for sex and some comorbidities, which were excluded from the preliminary models (Table 3) and the final model (Table 4). Variables in the final model included seven sociodemographic, three access to health care, one health status, and two health-related Internet usage measures. We could not validly estimate the interaction terms due to collinearity and an inadequate number of individuals within strata of the variables.

Table 2.

Predictors of personal prescription drug importation (PPDI): National Health Interview Survey, 2011 (Development Cohort)^a

Domain	Characteristic	Category	PPDI Risk (%)	P value
Sociodemographic	Age	18–44 (n=14,952)	2.3	<0.01
		45–64 (n=11,008)	2.5
		≥65 (n=6,841)	1.7
	Sex	Male (n=14,710)	2.1	0.27
		Female (n=18,092)	2.3
	Region of Residence	South (n=11,797)	2.4	<0.01
		Northeast (n=5,267)	1.3
		Midwest (n=7,300)	1.2
		West (n=8,438)	3.5
	Race	White (n=22,883)	2.1	<0.01
		Black (n=5,034)	0.7
		Otherb (n=4,885)	4.5
	Hispanic Ethnicity	No (n=26,968)	1.4	<0.01
		Yes (n=5,834)	6.2
	Highest Education	Completed collegec (n=12,355)	2.2	<0.01
		Some college (n=6,445)	1.7
		High school (n=8,422)	1.7
		Less than high school (n=5,407)	3.7
	Employment Status	Employed (n=18,836)	2.2	<0.01
		Unemployed (n=2,149)	3.3
		Not in labor force (n=11,789)	2
	Household Poverty	Not poor (n=17,795)	1.7	<0.01
		Near poor (n=5,828)	3.2
		Poor (n=5,598)	3.4
		Did not report income (n=3,581)	1.5
	U.S. Citizen	No (n=3,205)	8.1	<0.01
		Yes (n=29,539)	1.6
	Travel d	No (n=21,932)	1.3	<0.01
		Yes (n=10,628)	4.1
Medical Care Access	Health Insurance	Private only (n=14,794)	1.4	<0.01
		Public only (n=7,693)	1.6
		Public and private (n=3,123)	1.6
		None (n=7,085)	4.9
	Family OOP Medical	<$500 (n=11,857)	1.9	<0.01
		$500 to <$2,000 (n=9,543)	2.6
		$2,000+ (n=6,330)	2.8
		None (n=4,459)	1.5
	Trouble Finding MD	No (n=31,672)	2.0	<0.01
		Yes (n=1,106)	8
Health Status and Comorbidities	Self-reported Health	Excellent/very good (n=18,720)	1.8	<0.01
		Good (n=9,001)	2.7
		Fair/poor (n=5,064)	3.0
	Heart Attack	No (n=31,548)	2.3	0.16
		Yes (n=1,219)	1.6
	Heart Condition	No (n=30,167)	2.2	0.88
		Yes (n=2,608)	2.2
	Heart Disease	No (n=30,994)	2.3	0.07
		Yes (n=1,754)	1.6
	Angina Pectoris	No (n=31,946)	2.2	0.59
		Yes (n=798)	2.5
	Stroke	No (n=31,730)	2.2	0.54
		Yes (n=1,039)	2.5
	Hypertension	No (n=22,167)	2.4	0.03
		Yes (n=10,592)	2
	Asthma	No (n=28,702)	2.2	0.23
		Yes (n=4,069)	2.5
	Emphysema	No (n=32,042)	2.2	0.35
		Yes (n=731)	2.7
	Arthritis	No (n=24,640)	2.3	0.65
		Yes (n=8,121)	2.2
	Ulcer	No (n=30,368)	2.2	0.02
		Yes (n=2,392)	2.9
	Liver Disease	No (n=32,266)	2.2	0.27
		Yes (n=511)	2.9
	Diabetes	No (n=29,560)	2.2	0.02
		Yes (n=3,216)	2.8
	Cancer	No (n=29,930)	2.3	0.06
		Yes (n=2,840)	1.7
	Renal Disease	No (n=32,048)	2.2	0.01
		Yes (n=731)	3.6
Internet Usage	Looked up info	No (n=18,520)	2.1	0.05
		Yes (n=14,140)	2.4
	Filled a prescription	No (n=30,583)	2.1	<0.01
		Yes (n=2,094)	4.3
	Used online chat	No (n=31,488)	2.1	<0.01
		Yes (n=1,184)	5.7

^aData weighted using Sample Adult survey weights to account for the sampling scheme and compute valid standard errors, confidence intervals (CIs), and P values by accounting for the survey design, multistage cluster sampling, stratification, and differential probabilities of sampling members of the target population.

^bAmerican Indian/Alaska Native, Asian, and Native Hawaiian or other Pacific Islander.

^cCompleted college, graduate degree, or professional degree.

^dTravel outside the U.S. since 1995 other than Canada, Europe, Japan, Australia, and New Zealand.

Abbreviations: U.S.=United States; OOP=out-of-pocket.

Table 3.

Preliminary Logistic Regression Models Including All Significant Bivariable Results and All Variables Selected in Each Domain: National Health Interview Survey, 2011 (Development Cohort)^a

Characteristic	Model Including All Variables Significant in Bivariable Analysesb		Model Including All Variables Selected in Domain Modelsb
	POR	95% CI	POR	95% CI
Sociodemographic
Age (years)
18–44	Ref		Ref
45–64	1.7	1.3–2.2	1.7	1.3–2.1
≥65	2.2	1.4–3.2	2.0	1.3–2.9
Region of Residence
South	Ref		Ref
Northeast	0.6	0.4–0.8	0.6	0.4–0.8
Midwest	0.8	0.6–1.1	0.8	0.6–1.1
West	1.2	1.0–1.5	1.3	1.0–1.6
Race
White	Ref		---
Black	0.5	0.3–0.8	---
Other c	0.9	0.7–1.2	---
Hispanic Ethnicity
No	Ref		Ref
Yes	2.3	1.8–3.0	2.4	1.9–3.1
Highest Education
Completed colleged	Ref		Ref
Some college	0.7	0.5–0.9	0.6	0.5–0.8
High school completed	0.7	0.5–0.9	0.6	0.5–0.8
Less than high school	0.9	0.7–1.1	0.8	0.6–1.0
Employment Status
Employed	Ref		---
Unemployed	1.0	0.7–1.4	---
Not in labor force	1.3	0.6–1.0	---
Household Poverty
Not poor (≥2)	Ref		Ref
Near poor (1 to <2)	1.9	1.4–2.4	1.8	1.4–2.3
Poor (<1)	1.7	1.2–2.4	1.6	1.2–2.2
Did not report income	0.9	0.6–1.3	0.8	0.6–1.2
U.S. Citizen
Yes	Ref		Ref
No	2.5	1.9–3.4	2.4	1.8–3.2
Travel to Developing Countriese
No	Ref		Ref
Yes	2.5	2.0–3.1	2.6	2.1–3.2
Medical Care Access
Health Insurance Status
Private only	Ref		Ref
Public only	1.0	0.7–1.4	0.9	0.7–1.3
Public and private	1.3	0.8–2.1	1.3	0.8–2.0
None	2.5	1.9–3.2	2.4	1.9–3.1
Family OOP Medical
<$500	Ref		Ref
$500 to <$2,000	1.3	1.1–1.6	1.3	1.1–1.7
$2,000+	1.5	1.1–1.9	1.5	1.2–2.0
None	0.7	0.5–1.0	0.7	0.5–1.0
Trouble Finding MD
No	Ref		Ref
Yes	2.3	1.7–3.1	2.4	1.8–3.2
Health Status and Comorbidities
Self-reported Health Status
Excellent/very good	Ref		Ref
Good	1.4	1.1–1.8	1.4	1.1–1.7
Fair/poor	1.9	1.4–2.5	1.9	1.4–2.5
Hypertension
No	Ref		Ref
Yes	0.9	0.7–1.2	0.9	0.7–1.1
Ulcer
No	Ref
Yes	0.94	0.7–1.3	---
Diabetes
No	Ref
Yes	1.2	0.8–1.7	---
Renal Disease
No	Ref
Yes	1.7	1.0–3.0	---
Health-related Internet Usage
Looked up info
No	Ref
Yes	1.3	1.1–1.7	---
Filled a prescription
No	Ref		Ref
Yes	2.5	1.8–3.3	2.7	2.0–3.6
Used online chat
No	Ref		Ref
Yes	2.2	1.5–3.2	2.4	1.6–3.5

^aData weighted using Sample Adult survey weights to account for the sampling scheme and compute valid standard errors and confidence intervals (CIs) by accounting for the survey design, multistage cluster sampling, stratification, and differential probabilities of sampling members of the target population.

^bEach predictor adjusted for all other predictors listed in the column; dashes indicate that predictor was excluded from the model after stepwise regressions.

^cAmerican Indian/Alaska Native, Asian, and Native Hawaiian or other Pacific Islander.

^dCompleted college, graduate degree, or professional degree.

^eTravel outside the U.S. since 1995 other than Canada, Europe, Japan, Australia, and New Zealand.

Abbreviations: POR=prevalence odds ratio; CI=confidence interval; Ref=reference; U.S.=United States; OOP=out-of-pocket.

Table 4.

Final Logistic Regression Model of Personal Prescription Drug Importation: National Health Interview Survey, 2011 (Development Cohort)^a

Predictor	POR	95% CI
Sociodemographic
Age (years)
45–64	1.7	1.3–2.1
≥65	2.0	1.4–2.9
Region of Residence
South	1.7	1.3–2.2
West	2.1	1.6–3.0
Hispanic Ethnicity
Yes	2.4	1.9–3.1
Highest Education
Completed college b	1.3	1.0–1.7
Household Poverty
Near poor (1 to <2)	1.8	1.4–2.3
Poor (<1)	1.6	1.2–2.2
U.S. Citizen
No	2.4	1.8–3.2
Travel to Developing Countries c
Yes	2.5	2.0–3.1
Medical Care Access
Health Insurance Status
None	2.4	1.9–3.1
Family OOP Medical
<$500	1.5	1.0–2.1
$500 to <$2,000	1.9	1.3–2.9
$2,000+	2.2	1.5–3.3
Trouble Finding MD
Yes	2.4	1.8–3.2
Health Status and Comorbidities
Self-reported Health Status
Good	1.4	1.1–1.8
Fair/poor	1.8	1.4–2.4
Health-related Internet Usage
Filled a prescription
Yes	2.7	2.1–3.7
Used online chat
Yes	2.5	1.7–3.7

^aData weighted using Sample Adult survey weights to account for the sampling scheme and compute valid standard errors and confidence intervals (CIs) by accounting for the survey design, multistage cluster sampling, stratification, and differential probabilities of sampling members of the target population.

^bCompleted college, graduate degree, or professional degree.

^cTravel outside the U.S. since 1995 other than Canada, Europe, Japan, Australia, and New Zealand.

Abbreviations: POR=prevalence odds ratio; CI=confidence interval; U.S.=United States; OOP=out-of-pocket.

Index Performance

Index scores ranged from 0 to 11 in the development (mean [SD], 2.48 [1.67]) and validation (mean [SD], 3.20 [1.66]) cohorts. In respondents with 0–2 predictors, the most common predictor was total family OOP medical expenses $500–1,999. In the development cohort, the risk of PPDI ranged from 0.3% in respondents with 0–2 predictors to 13.7% in respondents with 7 or more predictors (Table 5, top). The index demonstrated acceptable discrimination with a C-statistic of 0.78. In the validation cohort, the risk of PPDI ranged from 0.3% in respondents with 0–2 predictors to 10.8% in respondents with 7 or more predictors. There was only a slight loss of discrimination in the validation cohort (C-statistic=0.76), and the index was well-calibrated, confirmed by the Hosmer-Lemeshow test statistic indicating absence of a gross lack of fit (P = 0.58).

Table 5.

Performance of Personal Prescription Drug Importation Index: National Health Interview Survey, 2011 (Development Cohort) and 2012 (Validation Cohort)^a

Index Score b	Risk of Personal Prescription Drug Importation (PPDI)
	Development Cohort, n/N (%)	Validation Cohort, n/N (%)
0–2	14/4,384 (0.32)	14/4,421 (0.32)
3	42/7,307 (0.57)	40/7,359 (0.54)
4	95/8,749 (1.09)	90/9,479 (0.95)
5–6	306/10,356 (2.95)	313/10,886 (2.89)
≥7	274/2,006 (13.67)	206/1,907 (10.80)
C-statistic	0.78	0.76
Comparative Risk of PPDI (Each Group v. Prior Group)c	POR (95% CI)	POR (95% CI)
3 v. 0–2	2.0 (0.9–4.1)	1.9 (1.0–3.7)
4 v. 3	1.9 (1.2–2.9)	1.4 (0.9–2.3)
5–6 v. 4	2.7 (2.1–3.5)	3.2 (2.4–4.2)
≥7 v. 5–6	4.9 (3.9–6.3)	4.2 (3.3–5.3)

^aData weighted using Sample Adult survey weights to account for the sampling scheme and compute valid standard errors and confidence intervals (CIs) by accounting for the survey design, multistage cluster sampling, stratification, and differential probabilities of sampling members of the target population.

^bIndex is the sum of the number of predictors in Table 4 that were present for each person in the study population.

^cEstimates obtained using backward difference contrast coding (the mean of PPDI for one index subgroup is compared to the mean of PPDI for the prior adjacent index subgroup).

Abbreviations: POR=prevalence odds ratio; CI=confidence interval; n=number of individuals who imported; N=number of individuals at risk of importing.

Comparative Risk

As expected, we observed that the risk of PPDI increased as one transitioned to the next higher subgroup with more predictors (Table 5, bottom). Notably, the relationship between importation score and risk of importation did not appear to be linear.

Policymaker Use of Index

To describe the policy implications of our index, we considered a hypothetical scenario in which the office of a state-level policymaker used our index to select individuals for an intervention to reduce PPDI. Since most policymakers are constrained by a budget, the intervention may be too expensive to deliver to all constituents (e.g., Medicaid insurance). On the basis of the index performance in the validation cohort, if the intervention were applied to all patients with scores of 7 or more, the intervention would be applied to 31.1% of patients who engage in PPDI and 0.6% of the overall population. These proportions would be 78.3% and 1.5%, respectively, for a cutoff of 5–6 or more points and 91.9% and 1.8%, respectively, for a cutoff of 4 or more points.

Performance of Alternative Models

The model that included all variables significant in the bivariable analysis had a C-statistic of 0.83 in the development and 0.82 in the validation cohorts. The model constructed after we considered results from the domain-specific stepwise regressions had a C-statistic of 0.83 in the development and 0.81 in the validation cohorts. Application of the AIC and BIC criteria produced divergent results with the AIC indicating that more inclusive models fit the data better and the BIC indicating more parsimonious models fit better. After we applied the BIC, a final stepwise backward selection procedure, and the point scoring system, the results suggest that the simplification of our index had a small impact on model accuracy. The C-statistic of the final model in the validation cohort (C-statistic = 0.76) was similar to the validation C-statistics of the more inclusive models.

Sensitivity Analyses

After eliminating the predictor from each domain that was most strongly associated with PPDI (i.e., travel to developing countries, health insurance status, self-reported health status, and filled a prescription on the Internet), the revised index had a C-statistic of 0.70 in the development and 0.69 in the validation cohorts. Repeating our analyses after we reclassified respondents who had missing PPDI data as having engaged in PPDI resulted in the selection of the same final model, but reduced predictive accuracy (C-statistic = 0.67). Alternative scoring systems provided small improvements in discriminative ability that were outweighed by complexity and impracticality. Final validation models using either alternative scoring system had a C-statistic of 0.78.

Discussion

We developed and validated an index to predict the likelihood of personal prescription drug importation by adults residing in the U.S. Our index has good calibration, demonstrated by the similar importation prevalence in the development and validation cohorts, and good discrimination, as demonstrated by a C-statistic of 0.78 in the development and 0.76 in the validation cohorts. The index is parsimonious without significant losses in discriminative ability: the C-statistic using all 20 candidate variables significant in bivariate analyses was 0.82, whereas the C-statistic using the 13 variable model was 0.76. These characteristics suggest that our index could potentially inform health policy development and guide future research on personal pharmaceutical importation.

Our index demonstrates that variables from multiple conceptual domains predict PPDI better than any single variable. This suggests that the consumer decision to import is a complex interplay of many factors and not only a function of income or personal capital, but also important demographics like Hispanic ethnicity. The final parsimonious index no longer included comorbid conditions, but the absence of other comorbidities does not imply that they have no etiologic role in the decision to engage in importation of foreign drug products. Instead, self-reported health status likely serves as a measure of disease severity and a strong proxy of many comorbid conditions that we considered (32). We believe that the extent to which a disease has caused self-perceived declines in health is a more important predictor of PPDI than which diseases an individual has (33).

The ability to predict the risk of PPDI allows physicians to select patients to educate about the risks of importation and safer sources of affordable pharmaceuticals like prescription assistance programs. More importantly, our index is useful to policymakers and public health professionals who wish to identify individuals for whom interventions to reduce PPDI are most appropriate. For example, as shown by Baicker et al., one possible intervention to improve access to and affordability of health care is Medicaid health insurance (34). Under at least two assumptions—1) that access and affordability are two key causal factors of PPDI; and 2) that effects of other factors like beliefs or health literacy are irrelevant—Medicaid health insurance could be a reasonable intervention. As Medicaid insurance programs expand under the Patient Protection and Affordable Care Act, policymakers could potentially use our index to help identify target populations for providing additional insurance coverage.

Our analysis is subject to several limitations. First, it was limited to subjects who were willing to participate in the NHIS. If the characteristics of non-participants are differentially related to PPDI, estimates of the relationship between predictors and outcome may be biased in both magnitude and direction. Without information about the characteristics of non-participants, we cannot assess the impact of this limitation. Second, due to the imprecision of the outcome survey question, we are unable to stratify individuals by mode of importation. It is possible that those who travel across U.S. borders or to other countries for medications are different in important ways from those who import medications by purchasing them online. If individuals do differ by mode of purchase, we may have underestimated the importance of some predictors for less prevalent PPDI subtypes. Lastly, while we considered an extensive range of predictors, we did not have data on some potentially important ones like health-related beliefs, health literacy, and perceptions of U.S. pharmaceutical product quality.

Conclusion

In conclusion, our index is a potentially useful method of predicting the risk of personal prescription drug importation by adults in the United States. Our index retained its predictive accuracy in an independent sample by demonstrating good calibration and good discrimination. These characteristics suggest our index may provide value to those who are responsible for health policy and regulation of pharmaceutical importation as well as other researchers.