Abstract
Objective
To assess the performance of Diagnostic Cost Groups (DCGs) in explaining variation in concurrent utilization for a defined subgroup, patients with substance abuse (SA) disorders, within the Department of Veterans Affairs (VA).
Data Sources
A 60 percent random sample of veterans who used health care services during Fiscal Year (FY) 1997 was obtained from VA administrative databases. Patients with SA disorders (13.3 percent) were identified from primary and secondary ICD-9-CM diagnosis codes.
Study Design
Concurrent risk adjustment models were fitted and tested using the DCG/HCC model. Three outcome measures were defined: (1) “service days” (the sum of a patient's inpatient and outpatient visit days), (2) mental health/substance abuse (MH/SA) service days, and (3) ambulatory provider encounters. To improve model performance, we ran three DCG/HCC models with additional indicators for patients with SA disorders.
Data Collection
To create a single file of veterans who used health care services in FY 1997, we merged records from all VA inpatient and outpatient files.
Principal Findings
Adding indicators for patients with mild/moderate SA disorders did not appreciably improve the R-squares for any of the outcome measures. When indicators were added for patients with severe SA who were in the most costly category, the explanatory ability of the models was modestly improved for all three outcomes.
Conclusions
Modifying the DCG/HCC model with additional markers for SA modestly improved homogeneity and model prediction. Because considerable variation still remained after modeling, we conclude that health care systems should evaluate “off-the-shelf” risk adjustment systems before applying them to their own populations.
Keywords: Risk adjustment, case-mix, substance abuse, capitation payments
Diagnosis-based risk adjustment systems are widely used to predict resource utilization, provide population-based health management, and make equitable comparisons across providers and facilities (Ettner and Notman 1997; Ettner etal. 1998; Ettner, Frank etal. 1999; Frank etal. 1997; Hendryx, Dyck, and Srebnik 1999; Breckenridge 2000). These population-based “case-mix tools” are increasingly being adopted by health care organizations. An important issue that will be faced by health care managers and providers is how well these tools perform in populations with differing characteristics from the population in which the risk adjustment system was developed. Although researchers have found that diagnosis-based risk adjustment methodologies perform better than simple age–gender adjustments in predicting resource utilization in various populations (Ash, Ellis, Pope, etal. 2000; Pope, Ellis, Ash, etal. 2000; Weiner etal. 1991), several studies have shown that applying “off-the-shelf” systems to new populations for which they were not designed may be problematic (Ettner etal. 1999). Thus, a critical question for health care organizations will be whether simple modifications to existing risk adjustment systems can improve model performance.
Another related question will be how well these diagnosis-based risk adjustment systems perform in clinically meaningful subgroups within new populations. Previous studies suggest that for some subgroups of Medicare beneficiaries (e.g., individuals with functional impairments living in the community), diagnosis-based risk adjusters significantly underpredict Medicare expenses, whereas for other subgroups (e.g., individuals institutionalized in long-term care), Medicare expenses are overpredicted (Riley 2000; Gruenberg etal. 1999). Similarly, among Medicaid beneficiaries, diagnosis-based risk adjustment systems are better able to predict expenditures for individuals with disabilities than for individuals who receive Aid to Families with Dependent Children (AFDC) (Kronick etal. 2000).
We found in previous work that Diagnostic Cost Groups (DCGs), a leading diagnosis-based risk adjustment system developed to predict future costs for Medicare beneficiaries, performed moderately well in explaining variation in concurrent utilization when applied to a different health care setting: the Department of Veterans Affairs (VA) (Rosen etal. 2001). However, the model performed least well among those veterans with the highest utilization, where utilization was significantly underpredicted. We extend our work in this study by evaluating the performance of the DCG system in explaining variation in concurrent utilization among veterans with substance abuse (SA) disorders, many of whom are “dually diagnosed” with both SA and mental health (MH) disorders. Specifically, we examine: (1) the ability of the DCG system to classify VA patients with SA disorders into homogeneous groupings with respect to utilization; (2) the explanatory power of DCGs with respect to modeling three outcomes (total utilization, ambulatory encounters, and MH/SA utilization); and (3) whether the performance of the DCG models can be improved by taking SA disorders more explicitly into account in the models.
Because DCGs were developed in a population with a low prevalence of substance abuse disorders (approximately 50 percent lower than the VA), we expected that the system would not be adequate in classifying VA patients with SA disorders into homogeneous groupings. In particular, because of the variation in the utilization patterns of patients with SA disorders (Horgan and Jencks 1987; Olfson and Pincus 1994; Simon and Unutzer 1999; Fortney, Booth, and Curran 1999), we expected that the DCG models were likely to underestimate the rates of expensive subgroups of SA patients and overestimate the rates of other less expensive subgroups (Pope etal. 2000). Also, because DCGs were designed primarily to predict total costs, we hypothesized that they were likely to perform better in predicting total services utilization (which includes both medical and MH/SA care) than in predicting either MH/SA utilization or ambulatory encounters.
We selected patients with SA disorders as the subgroup of interest for several reasons. First, there are ongoing concerns regarding the quality of care provided to patients with SA disorders within the VA. This is evidenced by the recent establishment of the Quality Enhancement Research Initiative (QUERI) Substance Abuse Module, whose mission is to improve the quality of care for veterans with SA disorders. Second, substance abuse disorders are a significant problem in the VA. For example, 23 percent of inpatients and 8percent of outpatients had a diagnosis of substance abuse in 1998; in addition, substance abuse disorders have become increasingly more severe, complex, and costly in the past few years (Finney, Willenbring, and Moos 2000). Third, studies have shown that patients suffering from SA disorders, conditions persistent and expensive to treat, have twice the average costs and utilization compared to patients with other MH disorders (Schoenbaum, Zhang, and Sturm 1998).
Few studies have focused on the use of diagnosis-based risk adjustment systems in characterizing populations with a high prevalence of patients with substance abuse (SA) disorders. Much of the prior work in this area has been in predicting inpatient admissions of patients with both MH and SA disorders (Fortney, Booth, and Smith 1996; Bauer etal. 1997). More recent studies have examined the prospective ability of existing risk adjustment systems to predict MH/SA expenditures for the purpose of setting capitation rates in the Medicare, Medicaid, and privately insured populations (Ettner and Notman 1997; Ettner etal. 1998; Ettner etal. 1999). None of the models accounted for more than 10 percent of the variance in future expenditures (Ettner etal. 1998). Each of the risk adjustment models would have led to underpayment for patients with psychiatric disability and overpayment for patients without psychiatric disability.
The VA operates the largest mental health service delivery system in the nation, providing specialty MH/SA services to over 650,000 veteransannuallyat a cost of almost $2 billion (Rosenheck and DiLella 2000). Because riskadjustment methods have been applied in a limited manner to patients with SA disorders and have had minimal success, VA data provideanopportunity to apply existing risk adjustment systems to a population of patients with a range of SA disorders. The VA databases are useful for studying bothtotal and MH/SA services use, because inpatient and outpatient data can be linked and patients tracked over time. Information required to categorize patients into risk groups that are similar with respect to resource needs, such as individuals’ demographic and clinical characteristics (i.e., ICD-9-CM codes) from a specific time period, is readily available. Thus, this study will provide important information on whether an existing risk adjustment system, developed in one population for a specific purpose, can perform successfully in another setting and also within a specific subgroup inthatsetting. Or, similar to results with prospective models (Ettner andNotman 1997; Ettner etal. 1998; Ettner etal. 1999), modifications maybe necessary to improve the performance of DCGs for concurrent modeling.
Methods
Databases
Our primary data sources were two inpatient administrative databases: the Patient Treatment File (PTF) and the Extended Care File (ECF), and one outpatient administrative file, the Outpatient Clinic File (OPC). The PTF has records on all individuals discharged from VA acute care hospitals as well as on all patients residing in VA acute care facilities on September 30 (the last day) of each fiscal year. Demographics and ICD-9-CM codes from each episode of care are available (Lamoreaux 1996). Diagnostic codes include the primary diagnosis; up to nine secondary diagnoses; the primary bedsection diagnosis (i.e., site of care); and up to four secondary bedsection diagnoses. The ECF has data similar to the PTF, except on long-term care residents.
The OPC file describes all outpatient care provided at VA facilities. Each outpatient visit may consist of 1 to 15 “clinic stops,” a VA term indicating the variety of clinical and nonclinical encounters that are delivered for patient care. Information related to each visit day includes demographics, eligibility for care, site and purpose of clinic stop(s), CPT-4 procedure codes, and diagnosis codes associated with each clinic stop (one primary and up to nine secondary diagnosis codes).
To create one complete file containing diagnostic and demographic information on veteran patients, we merged records from all these files using veterans’ encrypted social security numbers. As a final step, we merged this file with the Beneficiary Identification and Record Locator Subsystem (BIRLS) file, an administrative database that contains date of death information on veterans. This latter information is useful for constructing annualized utilization outcomes for veterans who died during the study period.
Sample
We selected a 60 percent random sample of veterans from the inpatient and outpatient Fiscal Year 1997 (FY ‘97) files (October 1, 1996 to September 30, 1997). We included all veterans who used acute, long-term, or outpatient care services during this period, excluding only nonveterans and individuals with dental or telephone service use exclusively. A split-sample technique was used to obtain a 40 percent sample (n = 1,046,803) for development and a 20 percent sample (n = 524,461) for validation.
The DCG/HCC Model
We used a specific DCG model, the DCG/HCC (Hierarchical Condition Category) model, a “multiple-condition model,” because it recognizes the cumulative effect of multiple conditions in predicting total medical expenditures. The HCC model groups both ambulatory and inpatient diagnosis codes into 543 mutually exclusive diagnostic clusters (DxGroups), which are closely related medical conditions. These DxGroups are further grouped into diagnosis-based condition categories (CCs) based on costliness and clinical relation. Although each ICD-9-CM code maps into only one CC, an individual may have multiple CCs. Hierarchies within conditions are imposed to prevent additional minor diagnoses from adding to cost predictions. Finally, the “hierarchicalized” CCs (HCCs) are used, in an additive model, to predict costs (Ellis and Ash 1995; Ellis etal. 1996; Ash etal. 2000).
The DCG system places patients with SA and MH disorders together into a Mental Disorders Hierarchy (Figure 1). The 37 MH/SA DxGroups are grouped into five “Mental Disorders” CCs (CC 31–35); each contains a range of clinically related DxGroups that are similar in levels of service use. The hierarchy contains both chronic and acute manifestations of MH/SA. A simple hierarchy orders these CCs into HCCs. For example, HCC 31 (Drug/Alcohol) is directly above HCC 32 (Psychosis). An individual with an alcoholic psychosis is expected to be more costly than an individual with a schizophrenic disorder. Also, an individual with both an MH and an SA disorder, such as a schizophrenic disorder (CC 32) and an alcoholic psychosis (CC 31), respectively, is placed in the higher relevant category. In this example, the individual is placed into HCC 31, the highest cost category within the Mental Disorders Hierarchy.
Figure 1.
Mental Disorders Hierarchy: 5 HCCs and 37 DxGroups
Severity and Type of Substance Abuse Classification
Because SA disorders are not separated from MH disorders within the Mental Disorders Hierarchy, it is difficult to ascertain whether an individual is placed into a specific category because of SA, MH, or a combination of MH/SA disorders. Therefore, to supplement the Mental Disorders Hierarchy, we developed a classification scheme to identify patients with SA disorders, and to further classify them into different levels of severity. This was useful not only in differentiating among patients with SA disorders, but in identifying subgroups of patients whose utilization was under- or overpredicted by the model. This classification also made it possible to identify specific disorders within diagnostic categories that could be used as model indicators because of their association with substantially high levels of impairment or utilization (Ettner and Notman, 1997; Ettner etal. 1998).
Based on clinical input, the literature, and empirical analyses, we classified primary and secondary ICD-9-CM diagnostic codes as SA if they fell into any of the following categories: 291.**, 292.**, 303.**, 304.**, and 305.** Our clinical team divided these diagnostic codes into two broad levels of severity.1“Mild/moderate” SA disorders included irregular use or psychological dependence (e.g., tobacco use disorder); “severe” SA disorders included those with persistent use, physiological effects, or likelihood of reoccurrence (e.g., alcoholic or drug psychoses). Similar to the DCG/HCC model, we imposed a hierarchy for identification purposes so that if an individual had both a moderate and a severe SA disorder, only the severe disease was counted.
Study Variables
The DCG/HCC model uses demographics and diagnoses generated from patient encounters over a one-year time period to describe the medical problems of patients and their likely effect on health care resource consumption (Ash etal. 2000). We obtained the necessary data elements for implementing DCGs from VA administrative data. Explanatory variables from FY ‘97 files included patient's demographic information (encrypted social security number, age, and gender) and diagnostic information (ICD-9-CM codes). We obtained diagnoses from all inpatient, outpatient, and extended care files, although we limited ambulatory diagnoses to those from face-to-face provider visits in order to exclude “tentative” or “rule-out” diagnoses. Diagnoses from laboratory, x-ray, and other types of diagnostic/screening visits were excluded.
Because the VA does not have claims databases, traditional resource utilization measures, such as costs, are not yet available at the patient level (Barnett 1999). Therefore, we selected two outcome measures that reflect overall health care utilization patterns of veterans: (1) “service days,” the sum of a patient's ambulatory visit days and inpatient days of care during the same 12-month period; and (2) the number of provider-related “face-to-face” ambulatory encounters during FY ‘97 (“ambulatory provider encounters”). We also constructed a third measure, “MH/SA service days,” in order to understand how much service day utilization related to MH/SA specifically. We used mortality information to generate “annualized” outcome measures and to weight utilization based on eligibility (i.e., the number of months in FY ‘97 that the patient was alive).
The first outcome, service days, reflects total utilization, medical and MH/SA, by each “active” patient (i.e., the number of days of contact with the system during the study period). Although it is possible for an individual to have both a hospitalization and outpatient clinic stop recorded on the same day, we considered this to represent one “service day.” As a result, the maximum possible number of service days for an individual in FY ‘97 is 365.
The second measure, ambulatory provider encounters, has frequently been used as an outcome measure with the Adjusted Clinical Groups (ACGs) case-mix system (Weiner etal. 1991; Weiner etal. 1996; Chang and McCracken 1996; Salem-Schatz etal. 1994). Here it summarizes all VA outpatient clinical care. We counted each individual outpatient clinic stop that was associated with specific Evaluation and Management (E/M) CPT-4 codes selected from the 1997 American Medical Association (AMA) listing of CPT-4 codes (American Medical Association 1997). This measure differs from service days in that it accounts for multiple provider encounters on a single date. Ambulatory encounters that were not provider-related included laboratory, x-ray, admission/screening stops, and other miscellaneous clinic stops.
The third outcome, MH/SA service days, summarizes only those services related to MH/SA care. It represents the sum of a patient's ambulatory visit days (i.e., those clinic stops that were specifically related to MH/SA) and inpatient days of care in an MH/SA bedsection during the same 12-month time period (Appendix A). Similar to service days, the maximum possible number of MH/SA days for an individual is 365.
Data Analysis
Analyses were performed using the Statistical Analysis System (SAS) software package, version 6.12 (SAS 1995). We constructed two analytic files required for applying the DxCG software, Release 4.2, (Guide 1999) to the data. We obtained means and standard deviations (SDs) of all dependent variables. We examined the distribution and utilization of patients with and without SA disorders in the HCCs within the Mental Disorders Hierarchy (HCCs 31–35). We also compared the utilization of patients within the Mental Disorders Hierarchy to that of all other VA patients.
We fit weighted least squares regression models in the development sample. Four models were constructed: The first, based on the standard DCG/HCC model, included 19 age/gender categorical variables2 and 116 HCCs as predictors (Model 1). Models 2–4 (“expanded models”) included the standard DCG/HCC model variables plus additional severity indicators for subgroups of patients with SA disorders in specific HCCs within the Mental Disorders Hierarchy. Indicators for these expanded models were defined based on the numbers of patients with mild/moderate and severe SA disorders in the Mental Disorders HCCs as well as their utilization patterns across the HCCs. We ran these four models to explain annualized health care utilization (i.e., service days, ambulatory provider encounters, and MH/SA service days). In all, a total of 12 weighted regression models were fit to the 40 percent development sample.
We compared the overall explanatory ability of the models using R-squares. We also performed cross-validation on Model 1 for each outcome variable separately. The process includes applying the fitted model to the 20 percent validation sample, then refitting the model on the 20 percent validation sample and applying it to the 40 percent development sample. We report validated R-squares and average cross-validated R-squares for all three outcomes. The average cross-validated R-square represents the summary cross-validation measure of each sample fit to the other.
To assess the incremental effect of SA disorders on predicted utilization, we examined the values of regression coefficients in the models. We compared the significance and direction (positive or negative) of the coefficients obtained for subgroups of patients with SA disorders in specific HCCs (e.g., Models 2–4) to those coefficients obtained for the same HCCs in the absence of additional markers (Model 1) for all three outcomes.
In addition to assessing overall model performance, we examined the accuracy of Model 1 and Model 4 prediction for the subgroups of patients with and without SA disorders in HCCs 31–35 within the Mental Disorders Hierarchy. We computed coefficients of variation (CVs) for each of the three annualized outcomes to measure relative discrepancy between observed and expected utilization within each subgroup. CVs were calculated as the ratio of weighted root mean square prediction error to weighted actual utilization mean. This reduces to the sample CV when no model has been fitted.
Results
Descriptive Results
Our 40 percent sample was 95.5 percent male, with a mean age of 59.1 years (SD = 15.2). The average number of service days was 17.6 (SD = 41.1), while the average number of MH/SA service days was 5.6 (SD = 28.9). Among the 15.8 percent of patients who were hospitalized, the average number of hospitalizations was 1.7. Eighty-seven percent of all patients had an ambulatory provider encounter. The average number of ambulatory provider encounters per patient was 7.7 (SD = 15.9). Most patients (97.4 percent) had 12 months of eligibility.
Of the total sample, 13.3 percent had SA disorders. Of these 139,032 patients, 52.6 percent had alcohol dependence/psychoses, 30.8 percent had drug dependence/psychoses, and 37.5 percent had only mild/moderate substance abuse. Almost 21.0 percent had both alcohol and drug dependence/psychoses, while 14.9 percent had a combination of alcohol and drug dependence/psychoses and mild/moderate abuse. Compared to the total population, patients with SA disorders had higher utilization; their average number of service days was 37.8 (SD = 57.1), their average number of MH/SA service days was 23.2 (SD = 50.1), and their average number of ambulatory provider encounters was 17.9 (SD = 31.9).
As shown in Table 1, 300,423 patients (28.7 percent of the total population) were assigned to the Mental Disorders Hierarchy (HCCs 31–35). Patients with SA disorders comprised 46.3 percent of the patients within this hierarchy; however, they were unevenly distributed within these HCCs. Approximately 7 percent of the patients in HCC 34 (Anxiety) had SA disorders, compared to 57.8 percent of the patients in HCC 35 (Lower Cost), and 100.0 percent of the patients in HCC 31 (Drug/Alcohol). Of the patients with mild/moderate SA disorders, 67.9 percent were classified into HCC 35 (Lower Cost), 21.2 percent into HCC 32 (Psychosis), and the remainder into HCC 33 (Depression) and HCC 34 (Anxiety). Patients with severe SA disorders (86,919 out of 139,032 patients, or 62.5 percent of all patients with SA disorders) were classified into HCC 31 (Drug/Alcohol) only, the clinical grouping representing the most severe (i.e., highest cost) MH/SA patients.
Table 1.
Distribution of Patients with and without Substance Abuse (SA) Disorders within the Mental Disorders Hierarchy (HCCs 31–35)
| HCC Totals | No SA Disorders | Mild/Moderate SA Disorders | Severe SA Disorders | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Hierarchical Condition Categories (HCCs) | N | % | %Total Population | N | % | %Total Population | N | % | %Total Population | N | % | %Total Population |
| HCC 31 Drug/alcohol | 86,919 | 28.9 | 8.4 | 0 | 0.0 | 0.0 | 0 | 0.0 | 0.0 | 86,919 | 100.0 | 8.4 |
| HCC 32 Psychosis | 91,284 | 30.4 | 8.8 | 80,221 | 49.7 | 7.7 | 11,063 | 21.2 | 1.1 | 0 | 0.0 | 0.0 |
| HCC 33 Depression | 54,877 | 18.3 | 5.3 | 49,658 | 30.8 | 4.8 | 5,219 | 10.0 | 0.5 | 0 | 0.0 | 0.0 |
| HCC 34 Anxiety | 6,263 | 2.1 | 0.6 | 5,821 | 3.6 | 0.6 | 442 | 0.9 | 0.0 | 0 | 0.0 | 0.0 |
| HCC 35 Lower cost | 61,080 | 20.3 | 5.9 | 25,691 | 15.9 | 2.5 | 35,389 | 67.9 | 3.4 | 0 | 0.0 | 0.0 |
| TOTAL | 300,423 | 100.0 | 28.9 | 161,391 | 100.0 | 15.6 | 52,113 | 100.0 | 5.0 | 86,919 | 100.0 | 8.4 |
Note: 746,380 patients were not in HCCs 31–35 (71.3% of total population).
There was a range over the three outcomes across the HCCs, with the highest and lowest utilization occurring among patients in HCC 31 (Drug/Alcohol) and in HCC 35 (Lower Cost) respectively, except for service days, where patients in HCC 34 (Anxiety) had slightly lower utilization than those in HCC 35 (Table 2). Patients in HCC 31 had more than twice the mean number of service days compared to those in HCC 35 (average service day means and [SDs] were 45.6 [62.8] and 19.7 [35.8], respectively). Similar trends occurred for ambulatory provider encounter means, but the differences were even greater between HCC 31 and HCC 35.
Table 2.
Utilization by Patients with and without Substance Abuse (SA) Disorders in the Mental Disorders Hierarchy (HCCs 31–35) and All Other HCCs
| Service Days | Ambulatory Provider Encounters | Mental Health/Substance Abuse (MH/SA) Days | ||||||
|---|---|---|---|---|---|---|---|---|
| Hierarchical | Condition Categories (HCCs) | n | Mean | (SD) | Mean | (SD) | Mean | (SD) |
| HCC 31 | Total patients | 86,919 | 45.6 | (62.8) | 21.9 | (37.0) | 32.7 | (57.4) |
| Drug/Alcohol | Patients with alcohol and drugs | 28,923 | 60.3 | (64.3) | 35.0 | (47.3) | 50.3 | (60.7) |
| Patients with drugs | 13,857 | 44.9 | (66.7) | 17.7 | (31.5) | 33.3 | (62.9) | |
| Patients with alcohol | 44,139 | 36.3 | (58.5) | 14.7 | (27.2) | 21.0 | (49.9) | |
| HCC 32 | Total patients | 91,284 | 38.0 | (66.5) | 16.9 | (28.6) | 22.3 | (57.9) |
| Psychosis | Patients with mild/moderate SA | 11,063 | 44.9 | (64.0) | 22.2 | (36.1) | 28.7 | (58.0) |
| HCC 33 | Total patients | 54,877 | 24.9 | (47.7) | 11.5 | (13.5) | 6.9 | (25.2) |
| Depression | Patients with mild/moderate SA | 5,219 | 27.1 | (41.2) | 13.3 | (14.7) | 7.9 | (23.2) |
| HCC 34 | Total patients | 6,263 | 16.8 | (29.0) | 9.2 | (8.4) | 3.3 | (11.9) |
| Anxiety | Patients with mild/moderate SA | 442 | 21.4 | (30.9) | 11.7 | (10.2) | 3.6 | (6.9) |
| HCC 35 | Total patients | 61,080 | 19.7 | (35.8) | 8.3 | (8.6) | 1.8 | (15.5) |
| Lower cost | Patients with mild/moderate SA | 35,389 | 17.9 | (31.5) | 7.6 | (7.7) | 1.2 | (11.2) |
| All other HCCs | Total patients | 746,380 | 11.1 | (30.0) | 4.6 | (5.8) | 0.7 | (12.8) |
Note: Utilization reported is annualized based on the number of months a patient was alive during the 12-month period; means and standard deviations (SDs) are unweighted. Total number of patients with ambulatory provider encounters in HCC 31 is 86,235.
Within each HCC except for HCC 35 (Lower Cost), patients with SA disorders had more than average utilization than patients without SA disorders; utilization also varied by SA disorder. Patients with drug abuse had higher utilization than patients with alcohol abuse (e.g., service day means [SDs] were 44.9 [66.7] and 36.3 [58.5], respectively). Within HCC 31, where all patients had some type of SA disorder, patients who had all three SA disorders (drugs, alcohol, and mild/moderate SA abuse) had substantially higher utilization than patients with fewer SA disorders (i.e., alcohol abuse only). Patients classified into all other HCCs (nonmedical and medical) (71.3 percent of all patients) had much less utilization on average that those in the Mental Disorders HCCs.
Regression Results for Models with and without Substance Abuse Markers
We used results from Tables 1 and 2 to guide the construction of variables for the expanded models. Since substantial numbers of patients with mild/moderate SA disorders were classified into HCC 32 (Psychosis), HCC 33 (Depression), and HCC 35 (Lower Cost), we constructed three severity indicators that identified the patients with mild/moderate SA within each of these three HCCs. Because patients with severe SA disorders were classified into HCC 31 (Drug/Alcohol) only, we created additional severity indicators to identify subsets of patients with severe SA disorders. The five mutually exclusive indicator variables were: (1) alcohol plus other (mild/moderate) abuse; (2) drugs plus other (mild/moderate) abuse; (3) both alcohol and drugs but no other (mild/moderate) abuse; (4) alcohol, drugs, and other (mild/moderate) abuse; and (5) drug abuse only. Alcohol abuse in the absence of other (mild/moderate) abuse served as the reference category.
In addition to Model 1, the standard DCG/HCC model (19 age/gender categorical variables and 116 HCCs), we constructed three expanded models, each with different combinations of severity indicators. Model 2 included three indicators for patients with mild/moderate SA disorders in each of HCC 32 (Psychosis), HCC 33 (Depression), and HCC 35 (Lower Cost). Model 3 included five indicators for different types of severe SA disorder in HCC 31 (Drug/Alcohol). Model 4 included both sets of severity indicators (8 in all) from Models 2 and 3.
Table 3 summarizes the performance of the models in explaining variance in annualized utilization. We found that the standard DCG/HCC model (Model 1) performed almost as well as the model with indicators for mild/moderate SA disorders (Model 2) across all three outcomes (e.g., R-squares for service days were 0.3153 and 0.3155, respectively). In contrast, Model 3, which included indicators for types of severe SA in HCC 31 (Drug/Alcohol), was superior to Model 1 across all three outcomes (e.g., R-squares for ambulatory provider encounters were 0.1810 for Model 1 and 0.2002 for Model 3). Model 4 included both sets of severity indicators. Although its R-squares were almost identical to those of Model 3 for all three outcomes, we selected it as the “improved” or “modified” DCG/HCC model because it was more comprehensive. Therefore, we omitted Models 2 and 3 from subsequent analyses. Validated R-squares, and average cross-validated R-squares, were comparable to development R-squares for Model 1, indicating overall stability of the models.3
Table 3.
Performance of the Models in Explaining Concurrent Utilization (FY ‘97)
| Risk Adjustment Model | Number of Parameters | Service Days R2 | Ambulatory Provider Encounters R2 | Mental Health/Substance Abuse Service Days R2 |
|---|---|---|---|---|
| n = 1,046,803 | n = 1,039,712 | n = 1,046,803 | ||
| Model 1: Standard DCG/HCC Model + Age/Gender | 135 | 0.3153 | 0.2458 | 0.1810 |
| Model 2: Model 1 + 3 SA Indicator Variables (Patients with mild/moderate SA in HCCs 32, 33 and 35) | 138 | 0.3155 | 0.2464 | 0.1813 |
| Model 3: Model 1 + 5 SA Indicator Variables (Patients with severe SA in HCC 31 divided into 5 groups: alcohol only; drugs only; alcohol & drugs; alcohol, drugs & mild/moderate abuse; alcohol & mild/moderate abuse; drugs & mild/moderate abuse) | 140 | 0.3236 | 0.2778 | 0.2002 |
| Model 4: Model 1 + SA Indicator Variables from Models 2 and 3 | 143 | 0.3238 | 0.2785 | 0.2005 |
Note: Utilization outcomes were annualized based on the number of months a patient was alive during the 12-month period. Ambulatory provider encounters=the number of “face-to-face” provider-related ambulatory encounters in FY ‘97 based on the individual clinic stop plus Evaluation and Management (E/M) CPT-4 codes. Service days=sum of the number of ambulatory visit days and number of inpatient days in FY’97 (maximum=365 days). MH/SA Service days=sum of the number of ambulatory MH/SA visit days + number of inpatient days in FY ‘97 in a MH/SA bedsection (maximum=365 days).
We found statistically significant coefficients for each SA severity indicator in Model 4 across the three outcomes except for the marker of mild/moderate SA in HCC 33 (Depression) for service days and MH/SA service days outcomes (Table 4). Although we considered eliminating this marker from the models, we retained it because it was significant and positive (0.63), albeit marginally incremental, in the model explaining ambulatory provider encounters. Patients with SA disorders had substantially higher utilization than patients without SA disorders across all three outcomes. The greatest impact on utilization occurred among patients in HCC 31 with all three SA disorders (alcohol, drugs, and mild/moderate abuse) (e.g., for MH/SA service days, the HCC 31 coefficient was 11.95 and the corresponding coefficient for the SA indicator was 37.11). The impact of drug abuse on utilization, whether alone or in combination with another SA disorder, was also substantial for patients in HCC 31 (e.g., for service days, the HCC 31 coefficient was 10.81 and the SA coefficients for the indicators “drugs only” and “drugs + mild/moderate SA” were 13.06 and 22.95, respectively).
Table 4.
Regression Coefficients for the Mental Disorders Hierarchy (HCCs 31–35) from Models 1 and 4
| Risk Adjustment Model | Service Days n=1,046,803 | Ambulatory Provider Encounters n=1,039,712 | Mental Health/ Substance Abuse (MH/SA) Service Days n=1,046,803 |
|---|---|---|---|
| Independent Variables | Coefficient* | Coefficient | Coefficient |
| Model 1: Standard HCC Model | |||
| HCC 31: Drug/alcohol | 23.02 | 14.99 | 25.44 |
| HCC 32: Psychosis | 15.94 | 9.75 | 16.21 |
| HCC 33: Depression | 5.61 | 4.03 | 3.94 |
| HCC 34: Anxiety | 1.67 | 2.52 | 1.52 |
| HCC 35: Lower cost | 0.57 | 0.60 | −0.15 |
| Model 4: Model 1 + 8 SA Severity Indicators | |||
| HCC 31: Drug/alcohol | 10.81 | 6.47 | 11.95 |
| HCC 31 & alcohol and mild/moderate abuse | 6.24 | 5.22 | 7.96 |
| HCC 31 & drugs and mild/moderate abuse | 22.95 | 10.67 | 24.19 |
| HCC 31 & alcohol and drugs | 14.13 | 11.74 | 15.79 |
| HCC 31 & alcohol, drugs, and mild/moderate abuse | 33.21 | 26.51 | 37.11 |
| HCC 31 & Drugs | 15.79 | 9.52 | 15.96 |
| HCC 32: Psychosis | 13.06 | 2.70 | 12.26 |
| HCC 32 & mild/moderate abuse | 4.20 | 4.43 | 5.48 |
| HCC 33: Depression | 5.83 | 4.13 | 4.08 |
| HCC 33 & mild/moderate abuse | −0.52 (NS) | 0.63 | 0.65 (NS) |
| HCC 34: Anxiety | 1.76 | 2.61 | 1.63 |
| HCC 35: Lower cost | 2.38 | 1.24 | 0.67 |
| HCC 35 & mild/moderate abuse | −2.91 | −0.86 | −1.12 |
Note: Utilization outcomes are defined as in Table 3.
All coefficients from SA severity indicators are statistically significant except for those marked NS.
All model coefficients were positive except for those identifying patients with mild/moderate SA disorders in HCC 35 (Lower Cost). This SA indicator had an additional impact on the estimated utilization of patients in HCC 35, but the coefficient was negative (e.g., −2.91 for service days), indicating that utilization was lower for the subgroup of patients with SA disorders in HCC 35. Results were comparable across all three outcomes.
We examined coefficients of variation (CVs) for patients within the Mental Disorders Hierarchy by SA category (severe, mild/moderate, and none) across all three outcomes. There was considerable variability within each SA category. All CVs exceeded 100 percent; the most variation occurred for MH/SA service days, where the sample (no model) CV ranged from 175 percent for patients with severe SA disorders in HCC 31 (Drug/Alcohol) to 401.7 percent for patients with mild/moderate abuse in HCC 32 (Psychosis), HCC 33 (Depression), HCC 34 (Anxiety), or HCC 35 (Lower Cost). CVs improved (i.e., decreased) for all three outcomes when the standard DCG/HCC model (Model 1) was applied to HCCs 31–35, and further when Model 4 (with the 8 additional indicators for SA disorders) was applied. This indicates that some of the subgroup variance was explained by Model 1, and more was explained by Model 4. As shown in Table 5, the CV for service days was 137.9 percent for patients with severe SA disorders (HCC 31, Drug/Alcohol); it decreased to 127.0 percent and 124.3 percent when Models 1 and 4 were imposed, respectively. Similar trends occurred for both ambulatory provider encounters and MH/SA service days. The CVs for these two outcomes were higher, though, than those for service days for patients with severe SA disorders. CVs also improved across all three outcomes for patients with mild/moderate SA disorders within HCCs 32–35. CVs were generally higher, indicating greater variability relative to the mean, for patients with mild/moderate SA disorders than those for patients with severe SA disorders. The CVs for patients without SA disorders (i.e., the remainder of those in HCCs 32–35) were comparable to those for patients with mild/moderate SA disorders.
Table 5.
Coefficients of Variation (CVs) for Substance Abuse (SA) Subgroups in Mental Disorders Hierarchy before and after Modeling for the Three Outcomes
| n | No Model | Model 1 Standard DCG/HCC Model | Model 4 Model 1 + 8 SA severity indicators | |
|---|---|---|---|---|
| Severe Substance Abuse Disorders (HCC 31) | ||||
| Service days | 86,919 | 137.86 | 127.03 | 124.27 |
| Ambulatory provider encounters | 86,235 | 168.23 | 160.63 | 154.47 |
| Mental health/substance abuse (MH/SA) service days | 86,919 | 175.00 | 169.13 | 164.51 |
| Mild/Moderate Abuse Disorders (HCCs 32–35) | ||||
| Service days | 52,113 | 174.27 | 145.03 | 144.86 |
| Ambulatory provider encounters | 51,823 | 171.60 | 154.82 | 154.20 |
| Mental health/substance abuse (MH/SA) service days | 52,113 | 401.65 | 365.06 | 363.92 |
| No Substance Abuse Disorders (HCCs 32–35) | ||||
| Service days | 161,391 | 191.01 | 165.62 | 165.62 |
| Ambulatory provider encounters | 159,299 | 159.73 | 148.64 | 148.79 |
| Mental health/substance abuse (MH/SA) service days | 161,391 | 337.51 | 319.50 | 319.53 |
Note: Utilization outcomes are defined as in Table 3.
Discussion
We examined whether an “off-the-shelf” diagnosis-based risk adjustment system was adequate in explaining variation in concurrent utilization among veterans with SA disorders. This is important for any health care organization that is considering adoption of a risk adjustment system in a population considerably different from the one in which the system was designed. Without sensitive methods to account for disease burden, adaptation of an existing risk adjustment system may result in inequitable resource allocations to subgroups of the population who may need them the most. Therefore, evaluating whether simple adjustments can improve model performance is important for successful adaptation.
This is the first study to our knowledge that examines the ability of a leading risk adjustment system to explain variation in concurrent utilization within this particular subgroup of the population. Several results emerge from this study. Existing “off-the-shelf” systems, such as DCGs, appear to have external validity when applied to other populations such as the VA. The ability of the DCG system to perform within clinically meaningful subgroups (i.e., patients with SA disorders) is more variable, however, and depends upon the severity of the disease, the outcome measure examined, and the performance measure used. Although we found that small improvements in overall model performance were possible when indicators of SA and severity were added, further improvements may also be possible by exploring interactions between SA and other variables, such as medical comorbidities. Nonetheless, our results suggest that further modifications are needed before these systems can be adapted for specific purposes, such as allocating resources or adjusting MH/SA payments.
Overall, the ability of the DCG system to explain variance in concurrent utilization was moderate, with R-squares ranging from 18.1 percent for MH/SA service days to 31.5 percent for service days. These R-squares are generally lower than those reported in the literature (Ash etal. 2000; Pope etal. 2000). As expected, R-squares were higher for total utilization (i.e., service days) than for either ambulatory provider encounters or MH/SA utilization. Once all SA disorders were taken into account, the models’ explanatory ability across all three outcomes was improved slightly. This improvement was due primarily to the addition of indicators for severe SA disorders in HCC 31 (Drug/Alcohol), indicating the importance of differentiating among this group of patients. Interestingly, there was only a nominal increase in the R-squares when indicators for mild/moderate SA disorders were added. Several factors may account for this, including the lower utilization of patients with mild/moderate SA disorders in HCC 35 (Lower Cost) compared to their counterparts in HCC 35, and the relatively small numbers of patients with mild/moderate SA disorders in HCC 32 (Psychosis) in the overall population.
We found that patients with severe SA disorders were classified appropriately into the most severe or “costly” HCC. The HCC classification scheme did not work as well for identifying patients with mild/moderate SA disorders, who represented a more heterogeneous group and were unevenly distributed into four HCCs, or for examining patients with both SA and MH disorders. The latter is an important area to address since many VA patients are dually diagnosed. One recent study showed that 67 percent of SA outpatients had both alcohol and drug diagnoses, and 52 percent had psychiatric disorders (Moos etal. 1999). The combination of MH and SA disorders can become both physically and emotionally disabling (Hoff and Rosenheck 1999), leading to heavy service use (Kent, Fogarty, and Yellowlees 1995).
Higher utilization was associated with classification into a more severe HCC, and it increased with the presence of each SA disorder, except in HCC 35, where patients with SA disorders had slightly less utilization than other patients. These results suggest that the Mental Disorders Hierarchy has face validity in VA data. In addition, the higher utilization we found for veterans with SA disorders is consistent with other studies (Piette, Baisden, and Moos 1999; Schoenbaum, Zhang, and Sturm 1998; Rosenheck and DiLella 2000).
The addition of severity indicators for patients with SA disorders in HCCs 31–35 decreased the variability within SA groupings across all three outcomes. Considerable variation remained, however, particularly among patients with mild/moderate SA disorders. These results suggest that simple modifications are not enough. The DCG/HCC model needs more refinement than was made in this study.
The studies by Ettner etal. (Ettner and Notman 1997; Ettner etal. 1998; Ettner etal. 1999) are most comparable to ours. Ettner etal. (1998) found that a model composed of seven classes of MH/SA diagnoses and four interactions between psychiatric comorbidities performed better than both an ACG and DCG/HCC model in predicting MH/SA expenditures among employees eligible for private insurance plans. Ettner etal. 1999 showed that the addition of psychiatric disability indicators, as well as a marker for the interaction of SA and MH disorders, improved the ability of both the DCG/HCC and ACG models to predict MH/SA expenditures. Despite these modifications, they concluded that extant risk adjustment models do not demonstrate adequate explanatory power in predicting total or MH/SA expenditures. Because these studies were conducted prospectively, and on different populations, comparisons with our study should be interpreted with caution.
One limitation of our study was that estimation of total health care resource consumption using expenditures or costs was not feasible because VA cost data at the patient level were not readily available. In addition, “service days,” a simple counting of the number of days on which care is provided, may have introduced some bias, by equating care given in the outpatient and inpatient settings.
Another concern is the issue of data reliability and the validity of administrative databases. Variability in coding practices across facilities and “upcoding” of diagnostic information are also problems that may affect model performance (Iezzoni 1997; Hannan et al. 1992; Romano etal. 1994). Nonetheless, VA administrative databases have important strengths that make them a useful source for risk adjustment studies. These include the lack of financial incentives for providers to “upcode” diagnoses, and a high level of data element completion, particularly important in developing clinical profiles (Kashner 1998).
Although our results were robust across different outcome measures, our population is not representative of the U.S. population with SA disorders (Wilson and Kizer 1997). Even so, this study has broader implications beyond the VA. Researchers applying the DCG/HCC model to their own populations/settings can benefit from our study by being aware that existing risk adjustment systems may not perform adequately in other databases or specific subgroups of the population. If these systems are applied without modifications, this may have serious consequences, particularly if they are used for setting capitation rates. Some providers may avoid the treatment of costly patients, while others may be penalized for serving the sickest and neediest population (Ettner etal. 1998). Our results indicate that although small improvements were possible, “off-the-shelf” systems may require further refinements and evaluation when applied to new populations with different disease burdens.
Acknowledgments
The authors are grateful to Laurie Todd for assistance in manuscript preparation, to Carter Rakovski, M.S., Bedford VAMC, for statistical assistance, and to Arlene Ash, Ph.D. (Boston University School of Medicine and DxCG, Inc.) for guidance on risk adjustment methodology and software implementation of DCGs.
Appendix A
A Mental Health/Substance Abuse Utilization
| Inpatient Utilization: | |||
|---|---|---|---|
| Bedsection codes based on major diagnostic categories (MDC) | |||
| MDC 19 | Mental disorders | ||
| MDC 20 | Drugs | ||
| Outpatient Utilization* | |||
| Mental Health | Clinic Stop Code | Mental Health | Clinic Stop Code |
| INDIVIDUAL: | MILIEU TREATMENT: | ||
| Psychiatry/neurobehaviorial | 509, 510, 511, 512 | Residential care | 503 |
| Psychology/mental health | 502, 504, 520, 531, 563 | Day treatment/day hospital | 505, 506, 532, 548, 553, 554 |
| Substance abuse | 137, 507, 508, 513, 514, 519, 523, 547 | Compensated work therapy | 515, 517, 518, 571, 572, 573, 574 |
| Sexual trauma counseling | 524, 525 | Vocational/housing assistance | 522, 535, 575 |
| Post-traumatic stress disorder (PTSD) | 540, 541, 562 | Geriatric psychiatry | 578 |
| Social work | 125 | PTSD day treatment | 580, 581 |
| Geriatric psychiatry | 576 | Outreach community services | 501, 551, 552, 590 |
| GROUP: | Domicilliary care | 725, 726, 727, 728 | |
| PTSD | 516, 561 | ||
| Long-term therapy | 521, 550 | ||
| Substance abuse | 555, 556, 560 | ||
| Psychiatry/neurobehaviorial | 557, 559 | ||
| Psychology | 558 | ||
| Geriatric psychiatry | 577 | ||
An outpatient visit may consist of 1–15 clinic stops reflecting any outpatient activity that occurred during the encounter(s). A clinic stop is a patient encounter with one or more health professionals at a particular clinic. A MH/SA clinic stop, such as Substance Abuse, has several stop codes associated with it (e.g., 137, 507, and 508). These reflect different locations where a patient has received care for SA disorders (e.g., inpatient unit of a medical center, outpatient SA clinic, psychiatry clinic, day treatment center).
Notes
Severe SA was identified by the presence of ICD-9-CM codes 291.**, 292.**, 303.**, or 304.**. Similarly, mild/moderate SA was identified by the presence of ICD-9-CM codes 305.**. As this scheme included codes indicating that the diagnosis was in remission (e.g., 303.x3, where 3 indicates remission), we considered moving those patients classified as severe to the mild/moderate category because a diagnosis in remission is theoretically less serious. However, since 77 percent of patients with remission codes also had other severe SA codes, most of them would have still been classified as severe even if they had been reassigned to the mild/moderate category because of the hierarchy imposed.
Age/gender categories were female aged 18–34, 35–44, 45–54, 55–59, 60–64, 65–69, 70–74, 75–79, 80–84 and male aged 18–34, 35–44, 45–54, 55–59, 60–64, 70–74, 75–79, 80–84, 85 + for all models. Female aged 85 + served as the reference category.
Model 1 (the standard HCC model) was validated for each of the three outcomes (validation sample n = 524,461). For service days, validated R2 equaled 0.3134 and cross-validated R2 equaled 0.3140. For ambulatory provider encounters, validated R2 equaled 0.2437 and cross-validated R2 equaled 0.2444. For MH/SA service days, the validated R2 equaled 0.1808 and cross-validated R2 equaled 0.1806.
This research was supported by grant number MPC 97-009, Office of Research and Development, Health Services Research and Development Service, Department of Veterans Affairs. An earlier version of this paper was presented at the Association of Health Services Research Annual Meeting (June 2000, Los Angeles, CA). The views expressed are solely those of the authors.
References
- American Medical Association. Physicians’ Current Procedural Terminology. 4th ed. Chicago: AMA; 1997. pp. 9–39. CPT Evaluation Management E/M Service Guidelines. [Google Scholar]
- Ash AS, Ellis RP, Pope GC, Ayanian JZ, Bates DW, Burstin H, Iezzoni L, Mackay E, Yu W. “Using Diagnoses to Describe Populations and Predict Costs”. Health Care Financing Review. 2000;21(3):7–28. [PMC free article] [PubMed] [Google Scholar]
- Barnett P. “Review of Methods to Determine VA Health Care Costs”. Medical Care. 1999;37(4):AS9–AS17. doi: 10.1097/00005650-199904002-00003. [DOI] [PubMed] [Google Scholar]
- Bauer MS, Shea N, McBride L, Gavin C. “Predictors of Service Utilization in Veterans with Bipolar Disorder: A Prospective Study”. Journal of Affective Disorders. 1997;44(2–3):159–68. doi: 10.1016/s0165-0327(97)00046-3. [DOI] [PubMed] [Google Scholar]
- Breckenridge J. “Provider Profiling: What Works for Medicine Works for Mental Health?”. Professional Psychology: Research and Practice. 2000;31(5):531–42. [Google Scholar]
- Chang W, McCracken SB. “Applying Case Mix Adjustment in Profiling Primary Care Physician Performance”. Journal of Health Care Finance. 1996;22(4):1–9. [PubMed] [Google Scholar]
- Ellis RP, Ash A. “Refinements to the Diagnostic Cost Group (DCG) Model”. Inquiry. 1995 winter;32:418–29. [PubMed] [Google Scholar]
- Ellis RP, Pope GC, Iezzoni L, Ayanian JZ, Bates DW, Burstin H, Ash A. “Diagnosis-Based Risk Adjustment for Medicare Capitation Payments”. Health Care Financing Review. 1996;17(3):101–28. [PMC free article] [PubMed] [Google Scholar]
- Ettner S, Notman E. “How Well Do Ambulatory Care Groups Predict Expenditures on Mental Health and Substance Abuse Patients”. Administration and Policy in Mental Health. 1997;24(4):339–57. doi: 10.1007/BF02042518. [DOI] [PubMed] [Google Scholar]
- Ettner SL, Frank RG, McGuire TG, Newhouse JP, Notman EH. “Risk Adjustment of Mental Health and Substance Abuse Payments”. Inquiry. 1998;35(2):223–39. [PubMed] [Google Scholar]
- Ettner SL, Frank RG, Mark T, Smith MW. “Risk Adjustment of Capitation Payments to Behavioral Health Care Carve-outs: How Well Do Existing Methodologies Account for Psychiatric Disability?”. Health Care Management Science. 2000;3(2):159–69. doi: 10.1023/a:1019033105715. [DOI] [PubMed] [Google Scholar]
- Finney J, Willenbring M, Moos R. “Improving the Quality of VA Care for Patients with Substance-Use Disorders: The Quality Enhancement Research Initiative (QUERI) Substance Abuse Module”. Medical Care. 2000;38(6):I-105–13. doi: 10.1097/00005650-200006001-00011. [DOI] [PubMed] [Google Scholar]
- Fortney J, Booth B, Smith G. “Variation Among VA Hospitals in Length of Stay for Treatment of Depression”. Psychiatric Services. 1996;47(6):608–13. doi: 10.1176/ps.47.6.608. [DOI] [PubMed] [Google Scholar]
- Fortney J, Booth B, Curran G. “Do Patients with Alcohol Dependence Use More Services? A Comparative Analysis with Other Chronic Disorders”. Alcohol and Clinical Experimental Research. 1999;23(1):127–33. [PubMed] [Google Scholar]
- Frank R, McGuire TG, Bae JP, Rupp A. “Solutions for Adverse Selection in Behavioral Care”. Health Care Financing Review. 1997;18(3):109–22. [PMC free article] [PubMed] [Google Scholar]
- Guide to DCG, Release 4.2. 1999.
- Gruenberg L, Silva A, Corazzini K, Malone J. “An Examination of the Impact of the Proposed New Medicare Capitation Methods on Programs for the Frail Elderly”. Cambridge, MA: Long-Term Care Data Institute; 1999. [Google Scholar]
- Hannan EL, Kilburn H, Lindsey M, Lewis R. “Clinical versus Administrative Databases for CABG Surgery Does it Matter?”. Medical Care. 1992;30(10):892–907. doi: 10.1097/00005650-199210000-00002. [DOI] [PubMed] [Google Scholar]
- Hendryx M, Dyck D, Srebnik D. “Risk-Adjusted Outcome Models for Public Mental Health Outpatient Programs”. Health Services Research. 1999;34(1):171–95. [PMC free article] [PubMed] [Google Scholar]
- Hoff R, Rosenheck R. “The Cost of Treating Substance Abuse Patients With and Without Comorbid Psychiatric Disorders”. Psychiatric Services. 1999;50(10):1309–15. doi: 10.1176/ps.50.10.1309. [DOI] [PubMed] [Google Scholar]
- Horgan C, Jencks S. “Research on Psychiatric Classification and Payment Systems”. Medical Care. 1987;25(9 supplement):S22–S36. [PubMed] [Google Scholar]
- Iezzoni L, editor. Risk Adjustment for Measuring Health Outcomes. Ann Arbor, MI: Health Administration Press; 1997. [Google Scholar]
- Kashner TM. “Agreement between Administrative Files and Written Medical Records: A Case of the Department of Veterans Affairs”. Medical Care. 1998;36(9):1324–36. doi: 10.1097/00005650-199809000-00005. [DOI] [PubMed] [Google Scholar]
- Kent S, Fogarty M, Yellowlees P. “A Review of Studies of Heavy Users of Psychiatric Services”. Psychiatric Services. 1995;46(12):1247–53. doi: 10.1176/ps.46.12.1247. [DOI] [PubMed] [Google Scholar]
- Kronick R, Gilmer T, Dreyfus T, Lee L. “Improving Health-Based Payment for Medicaid Beneficiaries: CDPS”. Health Care Financing Review. 2000;21(3):29–64. [PMC free article] [PubMed] [Google Scholar]
- Lamoreaux J. “The Organizational Structure for Medical Information Management in the Department of Veterans Affairs: an Overview of Major Health Care Databases”. Medical Care. 1996;34(3 supplement):MS31–MS44. doi: 10.1097/00005650-199603001-00004. [DOI] [PubMed] [Google Scholar]
- Moos R, Federman B, Finney J, Suchinsky R. Outcomes Monitoring for Substance Abuse Patients II Cohort I Patients’ 6–12 Month Treatment Outcomes. Palo Alto, CA: U.S. Department of Veterans Affairs Program Evaluation and Resource Center and Health Services Research and Development Center for Health Care Evaluation; 1999. [Google Scholar]
- Olfson M, Pincus HA. “Outpatient Psychotherapy in the United States II: Patterns of Utilization”. American Journal of Psychiatry. 1994;151:1289–94. doi: 10.1176/ajp.151.9.1289. [DOI] [PubMed] [Google Scholar]
- Piette J, Baisden KL, Moos RH. Health Services for VA Substance Abuse and Psychiatric Patients: Comparison of Utilization in Fiscal Years 1995 to 1998. Palo Alto, CA: Program Evaluation and Resource Center and Health Services Research and Development Center for Health Care Evaluation; 1999. [Google Scholar]
- Pope G, Ellis R, Ash A, Liu C, Ayanian J, Bates D, Burstin H, Iezzoni L, Ingber M. “Principal Inpatient Diagnostic Cost Group Model for Medicare Risk Adjustment”. Health Care Financing Review. 2000;21(3):93–118. [PMC free article] [PubMed] [Google Scholar]
- Riley G. “Risk Adjustment for Health Plans Disproportionately Enrolling Frail Medicare Beneficiaries”. Health Care Financing Review. 2000;21(3):135–48. [PMC free article] [PubMed] [Google Scholar]
- Romano PS, Roos L, Luft H, Jullis J, Doliszny K. “A Comparison of Administrative versus Clinical Data: Coronary Artery Bypass Surgery As an Example”. Journal of Clinical Epidemiology. 1994;47(3):249–60. doi: 10.1016/0895-4356(94)90006-x. [DOI] [PubMed] [Google Scholar]
- Rosen A, Loveland S, Anderson J, Rothendler J, Hankin C, Rakovski C, Moskowitz M, Berlowitz D. “Evaluating Diagnosis-Based Case-Mix Measures: How Well Do They Appy to the VA Population?”. [July 2001];Medical Care. 2001 30(7):692–704. doi: 10.1097/00005650-200107000-00006. [DOI] [PubMed] [Google Scholar]
- Rosenheck R, DiLella D. Department of Veterans Affairs National Mental Health Program Performance Monitoring System: Fiscal Year 1999 Report (Draft) West Haven, CT: Northwest Program Evaluation Center VA Health Services Research and Development Service; 2000. [Google Scholar]
- Salem-Schatz S, Moore G, Rucker M, Pearson SD. “The Case for Case-Mix Adjustment in Practice Profiling”. Journal of the American Medical Association. 1994;272(11):871–4. [PubMed] [Google Scholar]
- SAS Version 6.12. SAS Institute Inc. Cary. North Carolina: 1995. [Google Scholar]
- Schoenbaum M, Zhang W, Sturm R. “Costs and Utilization of Substance Abuse Care in a Privately Insured Population Under Managed Care”. Psychiatric Services. 1998;49(12):1573–8. doi: 10.1176/ps.49.12.1573. [DOI] [PubMed] [Google Scholar]
- Simon G, Unutzer J. “Health Care Utilization and Costs Among Patients Treated for Bipolar Disorder in an Insured Population”. Psychiatric Services. 1999;50(10):1303–8. doi: 10.1176/ps.50.10.1303. [DOI] [PubMed] [Google Scholar]
- Weiner J, Starfield B, Steinwachs D, Mumford L. “Development and Application of a Population-Oriented Measure of Ambulatory Care Case-Mix”. Medical Care. 1991;29(5):452–72. doi: 10.1097/00005650-199105000-00006. [DOI] [PubMed] [Google Scholar]
- Weiner JA, Starfield B, Powe N, Stuart M, Steinwachs D. “Ambulatory Care Practice Variation within a Medicaid Program”. Health Services Research. 1996;30(6):751–70. [PMC free article] [PubMed] [Google Scholar]
- Wilson N, Kizer K. “The VA Health Care System: An Unrecognized National Safety Net”. Health Affairs. 1997;16(4):200–4. doi: 10.1377/hlthaff.16.4.200. [DOI] [PubMed] [Google Scholar]