Instrumental Variable Methods to Assess Quality of Care

Abstract

Background

Hypertension is poorly controlled. Team-based care and changes in the process of care have been proposed to address these quality problems. However, assessing care processes is difficult because they are often confounded even in randomized behavioral studies by unmeasured confounders based on discretion of healthcare providers.

Objective

To evaluate the effects of process measures including number of counseling sessions about lifestyle modification and number of antihypertensive medications on blood pressure change and payer-perspective treatment costs.

Methods

Data were obtained from two prospective, cluster randomized controlled clinical trials (Trial A and B) implementing physician-pharmacist collaborative interventions compared with usual care over six months in community-based medical offices in the Midwest. Multivariate linear regression models with both instrumental variable methods and as-treated methods were utilized. Instruments were indicators for trial and study arms. Models of blood pressure change and costs included both process measures, demographic variables, and clinical variables.

Results

The analysis included 496 subjects. As-treated methods showed no significant associations between process and outcomes. The instruments used in the study were insufficient to simultaneously identify distinct process effects. However, the post-hoc instrumental variable models including one process measure at a time while controlling for the other process demonstrated significant associations between the processes and outcomes with estimates considerably larger than as-treated estimates.

Conclusions

Instrumental variable methods with combined randomized behavioral studies may be useful to evaluate the effects of different care processes. However, substantial distinct process variation across studies is needed to fully capitalize on this approach. Instrumental variable methods focusing on individual processes provided larger and stronger outcome relationships than those found using as-treated methods which are subject to confounding.

INTRODUCTION

The Institute of Medicine has suggested that health care is often not delivered optimally with either overutilization or underutilization of certain services which can lead to medical errors.¹ On average, patients with hypertension received the recommended quality of care only 65% of the time based on a list of 27 quality measures regarding physical examination, history, laboratory tests, counseling or education, appropriate medication, and encounter or intervention.² Less than optimal quality of care such as failure to intensify therapy when clinically indicated may explain why only 39% of visits for hypertension were at recommended blood pressure (BP) goals according to JNC 7 guidelines (Seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure).³

Several proposed changes to ameliorate quality-of-care problems include reorganizing practices to meet the needs of patients through multidisciplinary teamwork^1,4,5 and assessing both outcomes and processes of care.^6,7 Process of care measures for patients with hypertension including screening, diagnosis, treatment, and follow-up have been proposed.² Treatment and follow-up processes such as counseling and utilization of antihypertensive medications may be more strongly related to outcomes than other processes such as diagnosis that primarily determine the cause of hypertension.⁸ The few studies attempting to demonstrate a link between processes and outcomes exhibited certain limitations such as insufficient variation in process measures⁹, lack of control for the effects of other processes¹⁰, and potential unmeasured confounders biasing estimated relationships between process and outcome.^8,9,11

In randomized studies of process improvement interventions, the average effect of the total process improvement package is validly estimated through intention-to-treat analyses. However, it is often desirable to know the contribution of each individual process in the package to the outcomes achieved. As long as the providers who are delivering the intervention have discretion in the types or level of processes to deliver, there is an opportunity to evaluate how this variability relates to outcomes.

The confounding problem intrinsic to as-treated analyses of such “randomized process studies with discretion” can be alleviated using instrumental variable (IV) estimators.^12,13 IV estimators use randomization as the “instrument” to exploit only the process change related to randomization when assessing the effects of process on outcomes. However, when employing this IV approach using a single randomized study, it is only possible to assess the effects of a single process measure. This study tried to ascertain the distinct effects of patient counseling and drug utilization processes on outcomes for patients with hypertension by employing two techniques: mega-trial analysis¹⁴ and IV methods. Individual patient data from two prospective, cluster randomized controlled clinical trials implementing physician-pharmacist collaborative interventions for treating hypertension were combined and the data were analyzed as if they were from a single trial (mega-trial analysis). These interventions were designed with distinct characteristics in treating patients, which were theorized to lead to differences in the amount of patient counseling and drugs prescribed to patients and thus became an instrument for IV methods. The study objective was to evaluate the effects of the number of counseling sessions about lifestyle modification and the number of antihypertensive medications over six months on BP change and treatment costs by comparing the estimates from as-treated and IV methods.

METHODS

Data and study period

The data were obtained from two prospective, cluster randomized controlled clinical trials, namely trial A in 2009¹¹ and trial B in 2008¹⁵ by Carter and colleagues. Further description of the studies has been published previously.^11,15 Both trials examined the ability of physician-pharmacist collaborative interventions to improve BP control compared with usual care. The two trials were similar with respect to patient selection and baseline characteristics (Table A1 in Appendix A), methods to implement the interventions, and outcome measurement. The homogeneity test evaluating the consistency of the collaborative intervention effects across trials for meta-analysis showed that the variability in the intervention effects between the two studies was likely to be due to chance alone (Appendix B).

Trial A implemented a 6-month collaborative intervention whereas trial B implemented a 9-month intervention and measured BP at six months. For consistency this study evaluated 6 months of data from both trials. The number of subjects was slightly different from the totals across the original trials because the subjects included in this study were required to complete 6 months in their respective study to provide healthcare utilization data to estimate treatment costs.

The use of 6-month data of trial B implementing the 9-month intervention is valid because the process of care mostly occurred in the first few months and BP outcomes at six months and those at nine months were similar. In trial B, the vast majority of pharmacist recommendations to change medications occurred in the first two months of the intervention (77%) and only 12% occurred between the 6-9 month visits.¹⁶ From those pharmacist recommendations, 97% of them were accepted by physicians. So, the number of recommendations nearly equal the number of drug changes. Additionally, approximately 58% of drug therapy changes occurred in the first month in trial B while 55% of that occurred in the first month in trial A.¹⁷ Moreover, the average systolic and diastolic blood pressure (SBP and DBP, respectively) in mm Hg in the intervention group of trial B at six months were 126.6(standard deviation (SD) =11.9)/76.1(10.3) similar to that at nine months which were 124.2(9.7)/74.7(9.6).¹⁵ Therefore, the 6-month individual patient data from the two trials were combined.

Both trials prospectively and systematically collected the data about the care processes of the interventions, BP outcomes, and healthcare utilization during the study period. The interventions involved clinical pharmacists who were faculty members in medical offices. They collaborated with primary care physicians through face-to-face, phone, and written communication. Each clinical pharmacist had a PharmD-degree, and nearly all were residency trained. Their primary focus was addressing suboptimal medication regimens, recommending therapies consistent with JNC 7 guidelines¹⁸, and educating physicians with background information if necessary. The number of counseling sessions dealing with lifestyle modification for (1) weight reduction, (2) dietary approaches to stop hypertension (DASH), (3) sodium restriction, (4) increasing physical activity, (5) decreasing alcohol consumption and (6) others such as smoking cessation that were provided by either physicians or pharmacists during the interventions was counted for each patient. Moreover, types and doses of antihypertensive medications prescribed and the changes in the regimens during the study period were collected for each patient. Baseline characteristics and BP outcomes were collected by the research nurses. Treatment costs were obtained from a companion cost-evaluation study utilizing data from the same two trials.¹⁹

Subjects and settings

Subjects from the trials were patients aged 21 years or older with a diagnosis of essential hypertension. The recruited subjects represented prevalent cases where BP remained uncontrolled at baseline. The trials assigned 11 community-based medical offices in the Midwest to be in either the intervention group or the usual care group. Five community-based medical offices were assigned to be the intervention group and six in the control group.

Outcomes

Dependent variables included SBP change, DBP change, and treatment costs at six months. BP change was the difference between BP at six months and at baseline (mm Hg; a minus sign refers to reduction). Measurement of BP followed the standard guidelines.²⁰ BP was measured by trained nurses three times on the same day using a previously used protocol²¹; then the second and the third values were averaged to be the study BP. Both studies used 24-hour BP monitoring to ensure the reliability of the nurse-measured BP data.

Treatment costs were estimated from the payer’s perspective from each patient’s utilization related to primary care physician time, specialist time, pharmacist time, overhead, laboratory tests, and antihypertensive medications, multiplied by the respective prices per unit.¹⁹ The amount of primary care physician, specialist, and pharmacist time allocated to each patient was estimated from number of direct patient care and collaboration activities a patient received multiplied by average time (minutes) per activity. Estimates of minutes per activity were based on averages from survey responses. The National Ambulatory Medical Care Survey in 2003 provided average physician visit time. A second survey of pharmacists involved in the two Carter studies (the response rate was 87.5%) provided estimates of the average times to perform the lifestyle modification activities. The estimates were consistent across trials. From both trials, average times spent for each activity included 10.4 minutes for a weight reduction session, 7.7 minutes for a session describing the DASH plan, 5.8 minutes for sodium reduction discussions, 6.5 minutes for discussion to increase physical activity, 4.2 minutes to discuss decrease in alcohol consumption, and 7.3 minutes to encourage smoking cessation. Provider wage rates were obtained from published reports for primary care physicians ($79.64), specialists ($77.64), and pharmacists ($50.14) in 2008 value.²² Each laboratory test was assigned its costs from the Medicare laboratory fee schedule.²³ Medication costs considered changes in the regimens including starting a new medication and changing a dose during the study period. The market cost per day of the medication was estimated from a generic version, if available, with a 30-day supply. Total treatment costs were eventually adjusted to the US dollar values in 2013 using overall medical care price indexes obtained from the Bureaus of Labor Statistics.²⁴

Process-of-care measures

The number of counseling sessions received by each patient was measured by summing the number of all lifestyle modification sessions provided by both physicians and pharmacists to each patient over the study period of six months. Lifestyle modification counseling included weight reduction, DASH, sodium restriction, increasing physical activity, decreasing alcohol consumption, and others such as smoking cessation. The number and types of counseling sessions provided to each patient in the studies were left to provider discretion. It was assumed that the counseling was provided at equally acceptable quality to all subjects because all of the intervention pharmacists possessed a PharmD degree and nearly all of them had residency training and received similar training for the intervention. Moreover, only faculty physicians provided care to subjects in the trials. Therefore, given equal quality of counseling, number of counseling sessions reflects the impact of the quantity of counseling.

The measure of use of antihypertensive agents for each patient was the total number of specified-dose antihypertensive medications prescribed during the study period. This measure counted every specific dose of an antihypertensive agent prescribed. If a specific dose was discontinued and a new dose of the same agent was started, the count was two. However, a reorder or restart of the same dose of the same agent was not counted. Also, if it is assumed that patients purchased the medications and took them as prescribed, this measure represents the impact of all antihypertensive agents a patient experienced to lower his/her BP during the study period.

Control variables

Control variables included age, gender, race/ethnicity, marital status, smoking status, alcohol intake, number of antihypertensive medications at baseline, number of co-existing conditions, SBP at baseline, and DBP at baseline. The co-existing conditions included diabetes mellitus, peripheral artery disease, left ventricular hypertrophy, coronary bypass surgery, stroke, chronic kidney disease, heart failure, angina, and myocardial infarction. The control variables were the predictors of BP control and healthcare utilization suggested by the previous literature.^25-27

Our model did not specify a measure of adherence to antihypertensive medications because the data were missing for 17% of subjects (Appendix C). Subject adherence to medications was measured by self-reported responses to the Morisky scale; adherence was defined as answering no to 3 or more of 5 questions.¹¹ There was no statistically significant difference in number of subjects who were adherent between the intervention and the usual care groups for each trial (89% vs. 91% in trial A (p-value = 0.51) and 96% vs. 93% in trial B (p-value = 0.43), respectively). Also, no statistical difference in adherence was found across the trials (p-value = 0.13).

Analysis

Discretion in the number of counseling sessions provided to each patient and the number of medications prescribed to each patient was allowed in both trials A and B. The trials differed in the minimum number of required pharmacist contacts in the intervention groups. Trial A specified two pharmacist visits and one telephone call over six months, while trial B required four pharmacist visits over six months. Beyond the required protocols, additional phone calls or visits were allowed at the discretion of pharmacists if BP was not controlled. Neither trial required a minimum number of physician visits. Physician visits were scheduled at discretion of physicians in both the intervention group and the usual care group of studies.

The discretion available to providers in the in the trials to initiate counseling sessions and prescribe medications may cause bias in estimating the effects of these processes on outcomes when using as-treated methods. For example, additional care processes may have been provided to subjects with more severe clinical circumstances that were unmeasured in our data. If higher measured severity has direct negative effects on BP reduction and positive effects on healthcare costs, then directly estimated relationships between processes and BP reductions will be biased low and the relationships between the processes and healthcare costs will be biased high.

To address bias caused by unmeasured confounders driven by discretion, IV methods were utilized. IV methods provide an alternative approach to addressing problems with unmeasured confounders by using “instruments” to isolate the variation in the processes of care that is not associated with unmeasured confounders.^13,28 Instruments are measured variables that must be correlated with process of care (instrument relevance property), but are uncorrelated with unmeasured factors affecting outcomes and have no direct effect on outcomes (instrument exogeneity property). For typical studies, randomization at a patient level is a natural instrument because patients are randomized into intervention and control arms which will affect the processes they receive and randomization is not correlated with unmeasured factors or directly related with outcomes.²⁹

To identify distinct process effects, the number of instruments in an IV study must be greater than or equal to the number of processes being analyzed and the instruments must have independent effects on each process.^28,30 The second condition is needed because IV estimation only uses the variation in the process measures that is associated with the instruments. If the instruments affect each process measure in the same manner, there will be insufficient variation in each process measure identified by the instruments to estimate the independent effect of each process on the outcome.

An example of an IV estimator is a two-stage least square estimator.³¹ In the first stage, separate choice equations are estimated for each process of care as a function of specified instruments and control variables. Then, in the second stage, the outcome is regressed on control variables and each predicted process of care level produced by the first stage models.^28,32 The estimated effects of processes on outcomes in the second-stage regression are appropriately generalized to the subset of patients whose processes of care are affected by the instrument(s).³³

To operationalize IV methods in this study had two instruments available: the cluster randomization at the clinic level within each study; and the distinct design differences between trials with respect to number of provider visits. Both instruments were theorized to influence contacts that subjects had with providers which in turn affected both the amount of counseling and antihypertensive medications each subject received. These instruments divided the patients from the study into three groups: the intervention subjects from trial A; the intervention subjects from trial B; and the usual care subjects from both trials. The exogeneity requirement for both instruments should be satisfied because the two trials recruited very similar subjects with hypertension based on the similar inclusion and exclusion criteria, it was expected that the cluster randomization and the study designs would not be correlated with unmeasured factors affecting study outcomes. However, this may not always be the case because cluster randomization at a clinic level may not fully balance patient characteristics between groups.

Descriptive statistics of the covariates, process measures, and outcomes between three groups divided by the instruments of cluster randomization and the study designs were calculated to help assess the extent that the property of exogeneity held here.

The IV models were estimated using two-stage least squares (2SLS). The fully-specified 2SLS model for each outcome model included a first-stage counseling equation and a first-stage antihypertensive medication utilization equation. Each first-stage regression equation was explained by the control variables and two indicator variables of the instruments (the first indicator variable = 1 if the subject was in the intervention group in trial A, 0 otherwise; and the second indicator variable = 1 if the subject was in the intervention group in trial B, 0 otherwise; and the usual care groups in both trials were the reference group). In the second-stage of 2SLS, the predicted number of counseling events, the predicted number of specified-dose antihypertensive medications, and the same set of control variables were used to estimate the process effects on the outcome.

The F-tests for the first-stage regression models were used to assess whether the instruments had significant effects on the process measures. However, in a two process model such as this, estimation also requires that the predicted process measures from each first-stage regression equation contained sufficient independent information to estimate the distinct effects of each process. Lack of independent variation is called “under-identification” and is akin to multicollinearity in standard multiple regression models. The Kleibergen-Paap test was used to assess whether the outcome equation of the second-stage regression was sufficiently identified. A statistically significant Kleibergen-Paap test signifies sufficient identification.³⁴

If under-identification was found in the fully-specified models, post-hoc IV models including one process measure at a time while directly controlling for the actual value of other process measure will be utilized. This post-hoc IV method is akin to ridge regression approaches that mediate the effects of multicollinearity in multiple regression models by adding random variation to the independent variables to break up relationships among them.^35,36 This approach produces biased coefficient estimates but often substantially reduces estimated standard errors thereby providing more precise upper and lower bounds for the true parameter values.

As a comparison, ordinary-least-squares (OLS) linear regression models were utilized to estimate the effects of the processes on outcomes using an as-treated approach. The outcome model was explained by number of counseling sessions about lifestyle modification, number of specified-dose antihypertensive medications, and the control variables.

For consistency, linear specification was used for both as-treated and IV models. In addition, robust standard errors were estimated throughout because the distribution of the error terms across observations was unknown. The unit of the analysis was the individual subject.

SAS version 9.3 was used in managing data and performing descriptive statistics, comparisons, and diagnostic tests. Stata version 11.2 was used for the regression analysis (syntax: regress and ivreg2 with the robust option). A significance level of 5% was utilized for all analyses.

RESULTS

Descriptive statistics

Across both studies, 496 subjects were included. The sample patients had an average age of 60.15 years (SD = 13.32) and 60% were female. The majority of the subjects (88%) were Caucasians. On average, subjects took 1.50 (SD = 1.03) antihypertensive medications at baseline. Approximately 63% of the sample had no co-existing condition at baseline and the majority of the remaining had one condition (30%). Smokers represented 19% of the sample and 14% drank alcohol daily. The mean SBP and DBP at baseline were 152.16 (SD = 12.30) and 84.76 (SD = 11.90) mm Hg, respectively. To explain subjects excluded from the pool of subjects from trials A and B due to the requirement of complete 6-month data, there were 85 excluded subjects and 51% were female. The average age was 53 years and 60% of them were white/Caucasian. Although the excluded subjects were relatively younger than the included subjects, the average BP outcomes of the included subjects (N = 496) were in the range of the BP outcomes from the subjects in their original trials.^11,15,19

Table 1 contains average outcome, process of care, and baseline subject characteristic measures among the three subject groups defined by the instruments. On average, patients in trial A had 2.65 counseling sessions and 3.93 specified-dose antihypertensive medications whereas patients in trial B had 3.67 counseling sessions and 4.49 specified-dose antihypertensive medications. These counseling sessions were provided mostly by pharmacists (73% of the counseling sessions in the intervention groups were performed by pharmacists). Further details about time of counseling sessions by types of providers can be found in a separate study.¹⁹ Average processes measures were highest in the intervention group from trial B in which the protocol specified the highest minimum number of pharmacist visits as compared to the intervention group from trial A and the combined usual care group. The intervention group from trial B had the greatest unadjusted SBP reduction (25.82 mm Hg compared with 21.24 mm Hg from the intervention in trial A and 10.44 mm Hg from the usual care groups in both trials). The difference in SBP of 5 mm Hg is considered clinically significant because it approximately reduces incidence of coronary heart diseases events and stroke by 10 to 20%.³⁷ However, the intervention group from trial A had the highest DBP reduction (9.51 mm Hg) and the highest treatment costs ($792.44). These findings suggest that the process changes resulting from the randomization and the distinct characteristics between the two trials influenced BP changes and treatment costs.

Table 1.

Comparisons of variable values among the intervention group from trial A, the intervention from trial B and the usual care groups from both trials

Variable	Intervention group from trial A		Intervention group from trial B		Usual care groups from both trials
	N	Average (SD)	N	Average (SD)	N	Average (SD)
Outcome
Systolic blood pressure change (At 6 months – At baseline; mm Hg)	158	−21.24 (19.31)	94	−25.82 (14.07)	243	−10.44 (19.86)
Diastolic blood pressure change (At 6 months – At baseline; mm Hg)	158	−9.51 (11.12)	94	−8.94 (8.72)	243	−4.42 (11.46)
Total treatment costs (2013 US dollar value)	158	792.44 (405.74)	94	772.28 (291.61)	244	510.57 (347.95)
Process measure
Number of counseling sessions about lifestyle modification by physicians and pharmacists	158	2.65 (3.38)	94	3.67 (4.24)	244	0.71 (1.98)
Number of specified-dose antihypertensive medications prescribed during the study period	158	3.93 (2.23)	94	4.49 (2.26)	244	3.09 (1.82)
Control variables (Baseline characteristic)
Age (years)	158	58.60 (13.99)	94	59.81 (13.23)	244	61.29 (12.85)
Number of baseline antihypertensive medications	158	1.19 (1.07)	94	1.45 (0.96)	244	1.73 (0.99)
Number of co-morbiditiesa	158	0.34 (0.65)	94	0.40 (0.75)	244	0.62 (0.87)
Systolic blood pressure (mm Hg)	158	154.15 (12.75)	94	152.39 (9.86)	244	150.78 (12.72)
Diastolic blood pressure (mm Hg)	158	87.18 (11.57)	94	85.00 (11.84)	244	83.10 (11.90)
	N	Percentageb (%)		Percentageb (%)		Percentageb (%)
Female	158	64.56	94	57.45	244	57.38
Black	158	5.70	94	0.00	244	10.66
Other race	158	3.80	94	11.70	244	3.69
White or Caucasian	158	90.51	94	88.30	244	85.66
Married or living as married (vs. living alone)	158	67.72	94	59.57	244	54.92
Current smokers	158	21.52	94	7.45	244	22.54
Ex-smokers	158	31.01	94	32.98	244	32.79
Never smoked	158	47.47	94	59.57	244	44.67
No alcohol intake or less than 1 drink per day (vs. ≥ 1 drink per day)	158	90.51	94	78.72	232	86.21

^aCo-morbidities included diabetes mellitus, peripheral artery disease, left ventricular hypertrophy, coronary bypass surgery, stroke, chronic kidney disease, heart failure, angina, and myocardial infarction.

^bPercentages do not add up to one for some variables due to rounding.

Moreover, from Table 1, eight baseline measured covariates were quite similar across the three groups while six characteristics had slight to moderate differences across groups. The variables with differences were percentages of African-American subjects, subjects of other races, subjects who were married or lived as married, current smokers, subjects who never smoked, and subjects consuming alcohol. These variables were directly controlled for in our analysis, but they could be symptomatic of other unmeasured differences in potential confounders across practices.

As-treated methods

Table 2 shows as-treated estimates of number of counseling sessions and number of specified-dose antihypertensive medications on study outcomes. Neither process measure had a statistically significant impact on SBP or DBP. In contrast, both process measures had statistically significant positive relationships with total costs. An additional counseling session about lifestyle modification would increase in total costs by $33.02 (SE = $4.69, 95% CI = ($23.80, $42.24), p-value < 0.001) and an additional specified-dose antihypertensive medication was associated with an increase in total costs by $90.57 (SE = $8.74, 95% CI = ($73.41, $107.74), p-value < 0.001). Full parameter estimates are available in Appendix D.

Table 2.

Comparisons of process-of-care estimates between IV models, as-treated models, and post-hoc IV models^a for each outcome

Outcome/Process measure	Methods
	As-treated methods			IV methods			Post-hoc IV methodsa
	Coefficient (SE)	P-value	95% CI	Coefficient (SE)	P-value	95% CI	Coefficient (SE)	P-value	95% CI
SBP change
No. Counseling session	−0.45 (0.24)	0.06	(−0.92, −0.02)	−10.82 (14.79)	0.46	(−39.80, 18.16)	−5.30 (1.13)	<0.001	(−7.52, −3.08)
No. Medications	−0.05 (0.46)	0.92	(−0.94, 0.85)	10.78 (23.19)	0.64	(−34.66, 56.23)	−7.19 (1.57)	<0.001	(−10.27, −4.12)
DBP change
No. Counseling session	−0.10 (0.14)	0.49	(−0.38, 0.18)	−0.43 (3.04)	0.89	(−6.38, 5.53)	−1.65 (0.52)	0.002	(−2.68, −0.63)
No. Medications	−0.28 (0.23)	0.23	(−0.74, 0.18)	−1.67 (5.00)	0.74	(−11.48, 8.14)	−2.68 (0.81)	0.001	(−4.26, −1.10)
Total costs
No. Counseling session	33.02 (4.69)	< 0.001	(23.80, 42.24)	−383.15 (677.64)	0.57	(−1711.31, 945.01)	89.08 (14.74)	< 0.001	(60.18, 117.98)
No. Medications	90.57 (8.74)	< 0.001	(73.41, 107.74)	832.18 (1051.69)	0.43	(−1229.11, 2893.46)	191.81 (25.08)	< 0.001	(142.66, 240.96)

^aPost-hoc instrumental variable methods included one process measure as an endogenous regressor and the other process measure as a control variable.

IV methods

The first-stage F test statistics showed that the combined study instruments had significant effects on number of counseling sessions (F-statistic of 37.02, p-value < 0.001) and number of specified-dose antihypertensive medications (F-statistic of 47.02, p-value < 0.001).

Next, under-identification tests were conducted to assess whether the predicted process values were sufficiently independent to enable estimation of distinct process effects on each outcome. Unfortunately, the Kleibergen-Paap rk LM statistics failed to reject the null hypothesis (p-value = 0.50), suggesting that the fully-specified IV models which included both predicted process measures from the first-stage models were not sufficiently identified. Further investigation was conducted and it was found that the predicted number of counseling sessions and the predicted number of specified-dose antihypertensive medications was significantly correlated (correlation coefficient = 0.24; p-value < 0.001). Moreover, variance inflation factors were estimated and compared with the cut-off point of 10 which is generally used to ascertain whether multicolinearity problems exist. The variance inflation factor of the predicted number of counseling sessions were extremely high (236.62), meaning that the standard error of the predicted number of counseling sessions was 15.4 (square root of 236.62) times larger than it would have been if it was uncorrelated with the other independent variables. The variance inflation factor of the predicted number of specified-dose antihypertensive medications was 101.52.

Each fully-specified IV model (Table 2) showed no associations between the process measures and SBP change, DBP change and treatment costs. Full parameter estimates are available in Appendix D. However, especially notable are the large standard errors associated with each process estimate which signifies a multicollinearity problem. In each fully-specified IV model insufficient variation was available from each predicted process to accurately assess the effect of each process on outcomes.

Post-hoc IV analysis

The results from post-hoc IV models (Table 2) demonstrated that each process measure was significantly associated with every outcome (p-value < 0.001) with coefficient standard errors substantially smaller than in the fully-specified IV models. These results show that an additional counseling session by either a physician or a pharmacist was associated with SBP and DBP reduction by 5.30 mm Hg (SE = 1.13 mm Hg) and 1.65 mm Hg (SE = 0.52 mmHg), respectively. An additional counseling session was also associated with additional total cost of $89.08 (SE = $14.74) over six months. Furthermore, an additional specified-dose antihypertensive medication reduced SBP and DBP by 7.19 mm Hg (SE = 1.57 mm Hg) and 2.68 mm Hg (SE = 0.81 mm Hg), respectively. An added medication was associated with additional total cost of $191.81 (SE = $25.08).

DISCUSSION

This study aimed to estimate the marginal effects of the number of counseling sessions about lifestyle modification and the number of specified-dose antihypertensive medications on SBP change, DBP change and treatment costs. These effects were estimated and compared by using both as-treated methods and IV methods. The as-treated models did not yield statistically significant relationships between the process measures and both SBP and DBP change but showed positive relationships between both processes and total costs. However, since the process choices were discretionary in each study, it is possible that providers applied more of each process to patients with greater unmeasured severity and those patients tended to consume larger healthcare resources. It was expected that this would result in as-treated process effect estimates on SBP and DBP change that were biased low and effects on total cost that were biased high.

When utilizing fully-specified IV models to address unmeasured confounders, the models were unidentified. Even though the instruments significantly explained the variation in each process measure as shown by the F-statistics from the first-stage regressions, the variation in the process measures isolated by the instruments was not sufficient to estimate distinct process effects on each outcome. It appears that, even though the interventions differed between trials, these differences were unable to generate sufficient differences in how the two processes were offered to patients across the studies. In the post-hoc IV models, however, both process measures were associated with reductions in SBP, and DBP and increased total costs. These estimates are potentially biased from the inability to fully control for the portion of the variation in the other process measure that was associated with the instruments. Because both process measures likely reduce BP, it is likely that these post-hoc IV estimates reflect upper bounds of the true effects. However, given the substantially smaller standard errors of the post-hoc IV estimates relative to the fully-specified IV models, the confidence intervals around the post-hoc IV estimates provide a defensible range for the true parameter values.

The signs of the estimates from the post-hoc IV models and the as-treated models were the same. However the magnitudes of the estimates were quite different. The post-hoc IV models revealed considerably larger reductions in SBP and DBP and higher total costs associated with unit changes in each process. These results suggest that relying on as-treated estimates to assess the effects of provider counseling sessions and use of antihypertensive medications will understate the benefits of these processes and overstate their effects on healthcare costs.

In comparing the results of the present study to the previous literature, Inkster et al. (2005) could not find any association between pharmacotherapy processes and BP control.⁹ Their observational study using a sample from eight general practices in the United Kingdom found that three or more BP lowering drugs (vs. one drug) was not associated with BP control (adjusted odds ratio = 1.31, 95% CI = 0.96 to 1.79). This was similar to the present study whereby the association between BP reduction and number of specified-dose antihypertensive agents from the as-treated models was not found. This finding is likely due to the fact that subjects with the most difficult to control BP required more medications and yet, had less of an effect on BP.

In addition, Inkster and colleagues found that a higher number of consultations led to an increased likelihood of having inadequate BP control. In contrast, this present study did not find any significant relationship between number of counseling sessions and BP reduction from the as-treated models. The results from the previous study might have been due to the fact that unmeasured confounders such as severity of BP generally caused physicians to provide more counseling sessions to patients with uncontrolled BP. The present study shows that IV methods may be useful to remove some bias caused by unmeasured confounders driven by health provider discretion.

A study by Brooks et al. observed a disparity between IV and as-treated estimates of the process effects on costs.¹³ Using the data from a randomized controlled trial, their study evaluated the impact of the evidence-based acute pain management practices on inpatient cost changes. The estimate from the IV methods showed that such practices resulted in a drop of inpatient costs by $1,602, which was largely greater than the estimate of the inpatient cost reduction by $58 by using as-treated methods.

The following limitations of this study are acknowledged. Considerable differences in baseline characteristics remained between the groups of patients divided by the instruments according to Table 1 partly due to the cluster randomization of the clinics to avoid contamination of the intervention at the physician level. It may be difficult to fully justify that the instruments were uncorrelated with unmeasured factors affecting outcomes (instrument exogeneity property). If the correlation between the instruments and the unmeasured factors has the opposite direction with the correlation between the instruments and the control variables, the IV estimates will be biased low. Likewise, if those correlations have the same direction, the IV estimates will be biased high.

As stated earlier, the present study was unable to estimate the individual effect of a process of care controlling for other processes due to a limited number of instruments and issues about independent variation of each process measure. Further research may be needed to address the under-identification issue by having interventions which have the same sets of care processes but different focuses on the care. This approach may extract process variation sufficient to estimate the effect on outcomes. For instance, an intervention from one study could primarily focus on changes in pharmacotherapy and an intervention from the second study might heavily emphasize on counseling sessions about lifestyle modifications. Thus, the instrument of distinct characteristics between the two studies should increase variation in each process measure. Moreover, future research should combine more than two studies to attain distinct characteristics between the studies and use that as the instruments. The application of the IV approach and combining multiple randomized studies may be used as a meta-analysis of behavioral interventions to further show the effects of each process embedded in the interventions.

Furthermore, the results may not apply to different settings such as non-community clinics, interventions lacking face-to-face communication between physicians and pharmacists in the same office, and populations with a greater percentage of minorities.

CONCLUSIONS

Instrumental variable methods with combined randomized behavioral studies may be useful to address unmeasured confounders and to evaluate the effects of different care processes. Studies with distinct study designs that create more variation in care processes are needed to address problems of identification. Instrumental variable methods focusing on individual processes provided larger and stronger outcome relationships than those found using as-treated methods which are subject to confounding. Further investigation of the link between care processes such as counseling and drug utilization and outcomes with rigorous methodology will be helpful to improvement on quality of care.