Assessing the representativeness of Medical Expenditure Panel Survey inpatient utilization data for individuals with psychiatric and non-psychiatric conditions

Abstract

Sampling and reporting biases in the Medical Expenditure Panel Survey (MEPS) survey could render data on inpatient utilization that are not representative for individuals with severe psychiatric conditions. The authors assessed the representativeness of MEPS data on psychiatric inpatient utilization, by comparing MEPS estimates of total annual psychiatric and non-psychiatric inpatient admissions and bed days, and mean length-of-stay, for non-elderly U.S. adults in calendar years 2005 to 2010 (N=9288) to estimates from the Nationwide Inpatient Sample (NIS), a nationally representative inpatient care database derived from hospitals’ administrative records (N=21934378). Compared with the NIS, the MEPS indicated 34% as many psychiatric admissions and 86% as many non-psychiatric admissions, while mean psychiatric length-of-stay was greater in MEPS than in NIS. In MEPS data, underrepresentation of psychiatric inpatient utilization at community hospitals may result in measurement distortions for commonly used statistics on psychiatric inpatient utilization and costs.

INTRODUCTION

The Household Component of the U.S. Medical Expenditure Panel Survey (MEPS) survey is a unique data source, designed and financed by the federal government to address critical national questions regarding health care resource use and the social and economic determinants of health care access, utilization, and expenditure in the U.S. However, sampling and reporting biases could render MEPS data on inpatient utilization that are unrepresentative for individuals with severe psychiatric conditions. Individuals with severe psychiatric conditions may be especially hard to locate, recruit into, and then retain in a household-based survey sample, in part because many of them do not reside at an identifiable address and in part because they have a greater propensity for social and behavioral characteristics that are known to negatively influence survey participation (e.g., social isolation and self-stigma) (Gfroerer, Lessler, & Parsley, 1997). In addition, in personal interviews, some respondents may not fully disclose use of inpatient psychiatric services due to a perception of stigma or privacy concerns (Bhandari & Wagner, 2006; Golding, Gongla, & Brownell, 1988; Killeen, Brady, Gold, Tyson, & Simpson, 2004), a phenomenon that has also been observed with other sensitive topics (Gfroerer et al., 1997; Harrison & Hughes, 1997). Finally, respondents who had more severe psychiatric symptoms and multiple psychiatric inpatient stays during the survey reference period may have greater difficulty recalling all of these events, and briefer stays may be more easily forgotten than longer stays (Clark, Ricketts, & McHugo, 1996).

Information regarding the representativeness of MEPS data on psychiatric inpatient utilization is critically needed by the consumers of MEPS estimates. Individuals with severe psychiatric conditions, such as schizophrenia, bipolar disorder, major depression, and chronic substance abuse, account disproportionately for health care spending and disability costs (Dismuke & Egede, 2011; Himelhoch, Weller, Wu, Anderson, & Cooper, 2004; Katon, 1996). They also have above-average prevalence of chronic medical conditions that have large associated costs, such as HIV, hepatitis, and diabetes (Dixon, Postrado, Delahanty, Fischer, & Lehman, 1999; Himelhoch et al., 2007; Jones et al., 2004; Rosenberg et al., 2001; Sokal et al., 2004). As a result, individuals with severe psychiatric conditions are often a focus of reforms aimed at improving health care access or quality and reducing illness-related morbidity and expenditures.

NEW CONTRIBUTION

Although the validity of MEPS data on utilization and expenditures for various categories of health care services has been examined previously (Aizcorbe et al., 2012; S. H. Zuvekas & Olin, 2009a; S. H. Zuvekas & Olin, 2009b), prior assessments did not specifically assess data on psychiatric services utilization. Results from previous assessments suggest that health care utilization is moderately underrepresented in MEPS compared with administrative claims data (Aizcorbe et al., 2012; S. H. Zuvekas & Olin, 2009a). However, the magnitude of discrepancy between MEPS and administrative health care utilization data could be greater for psychiatric conditions than for non-psychiatric conditions. This study examines the magnitude of the discrepancy between MEPS-based estimates of aggregate U.S. inpatient utilization and estimates based on the Nationwide Inpatient Sample (NIS) (Agency for Healthcare Research and Quality, 2013b). The NIS is a nationally representative inpatient utilization database derived from hospitals’ administrative records. Although NIS and MEPS have somewhat different sampling frames, and may consequently not produce identical estimates of overall inpatient utilization, NIS data are not subject to the sampling and reporting biases that we hypothesize may disproportionately affect MEPS data for psychiatric inpatient utilization. As a result, comparisons of the same psychiatric and non-psychiatric inpatient utilization statistics derived from MEPS with those derived from NIS may help identify whether psychiatric inpatient utilization is differentially underrepresented in MEPS.

CONCEPTUAL FRAMEWORK

The MEPS’ sample is constructed from household addresses and depends on voluntary participation, and therefore may underrepresent homeless individuals and individuals who may have greater propensity not to volunteer (Gfroerer et al., 1997). Moreover, the MEPS’ services data comes from voluntary self-reports provided during in-person and telephone interviews. Psychiatric inpatient care may not be fully reported under these conditions, as a result of stigma or confidentiality concerns (Bhandari & Wagner, 2006; Golding et al., 1988; Killeen et al., 2004), or as a result of respondents not recalling psychiatric inpatient stays (Clark, Ricketts, & McHugo, 1996). By contrast, the NIS data are representative of hospital patients, irrespective of their propensities for survey participation, and reporting biases are unlikely to be significant, because the data are drawn from hospitals’ administrative records. As a result, discrepancies between MEPS and NIS total inpatient admissions were hypothesized to be greater for psychiatric admissions than for non-psychiatric admissions (i.e., admissions for general medical conditions).

DATA AND METHODS

Sample

Study data were from the 2005–2010 MEPS and the 2005–2010 NIS. The MEPS household sample is a nationally representative stratified survey of the non-institutionalized civilian U.S. population (Cohen et al., 1996). Multiple years of MEPS data were required because a single year of the MEPS contains too few inpatient records for reliable estimation. The NIS is an approximately 20% stratified random sample of U.S. community hospitals in 45 states (Agency for Healthcare Research and Quality, 2013b). Community hospitals in NIS include non-federal short-term hospitals and long-term acute care facilities, excluding hospital units of institutions (Agency for Healthcare Research and Quality, 2002). Also excluded from the NIS are short-term rehabilitation hospitals, long-term non-acute care hospitals, psychiatric hospitals, and alcoholism/chemical dependency treatment facilities. NIS sample weights are designed to reproduce national estimates.

Design differences between the MEPS and NIS samples were expected to result in some inpatient utilization discrepancies, independent of any sampling or reporting issues in MEPS. These sources of discrepancy and their expected impacts on measures of total inpatient admissions and length-of-stay are listed in Table 1. MEPS excludes from its sample most individuals who live primarily in institutions, such as nursing homes and skilled nursing facilities, psychiatric hospitals, residential care institutions, and jails/prisons (Cohen et al., 1996). General hospital inpatient admissions for such individuals would only be included in MEPS if the individual in question had been a member of a MEPS household and had an inpatient stay during a period when the individual was not residing in an institution. MEPS also excludes most inpatient admissions for active duty military personnel. By contrast, NIS data generally include all short-term admissions to community hospitals, regardless of where patients live or their military status. In one respect, MEPS inpatient utilization data are more inclusive than NIS. Unlike the NIS, the MEPS does not specifically exclude reported psychiatric inpatient admissions to psychiatric facilities, psychiatric rehabilitation programs, non-hospital-based residential substance abuse treatment programs, and federal government hospitals. The inclusion of these specialty admissions in MEPS may result in longer psychiatric inpatient lengths-of-stay in MEPS than in NIS, because psychiatric inpatient stays with these particular providers are typically longer than psychiatric inpatient stays at community hospitals.

Table 1.

Differences in MEPS-NIS in-scope categories of hospital inpatient admissions

	In-Scope (Y/N)?		Potential Impact on MEPS/NIS ratios
	MEPS	NIS	Admissions	Length of Stay
• Inpatient admissions to community hospitals of individuals living primarily in institutions (nursing homes, skilled nursing facilities, long-term psychiatric institutions, prisons)	N a	Y	↓	?
•Inpatient admissions to community hospitals of active-duty military personnel	N	Yb	↓	?
•Short-term inpatient admissions to specialized psychiatric hospitals, psychiatric rehabilitation programs, and residential substance abuse treatment programs	Y	N	↑	↑
•Short-term inpatient admissions to federal hospitals (e.g., Veterans Administration hospitals)	Y	N	↑	↑

^aInpatient admissions for MEPS household members are considered in-scope if the individual was living in the household during the survey reference period, regardless of whether the individual had been in an institution at some point in time.

^bIn NIS, inpatient admissions of active-duty military personnel to Department of Defense hospitals and foreign hospitals are out-of-scope.

In order to minimize the impacts of known sample design differences, samples from both sources were limited to inpatient stays for non-elderly adults aged 22 to 64 years in U.S. states and the District of Columbia. The rationale for these restrictions was threefold. First, the age restriction substantially reduces the number of discrepancies that may result from the MEPS’ sample exclusion of individuals residing in institutions. By restricting the sample to non-elderly adults, inpatient admissions are removed for the vast majority of nursing home and skilled nursing facility residents as well as a substantial minority of residential care facility residents (Jordan & Beaghen, 2011). However, community hospital admissions by non-elderly residents of institutions may be a significant source of discrepancy. Consequently, we examine the impact of this design factor in a sensitivity analysis. Second, although the NIS sample excludes short-term inpatient stays in psychiatric hospitals, whereas the MEPS does not, this difference is less consequential for individuals ages 22–64. Medicaid does not cover inpatient admissions to most psychiatric hospitals for individuals in this age range, due to the Institutions for Mental Diseases exclusion (Geller, 2000). Last, inpatient admissions records pertaining to U.S. territories were not included, because these areas are not sampled in the NIS.

The MEPS sample included 9682 inpatient admissions for individuals ages 22 to 64, and the NIS sample included 21034378 admissions. Three-hundred-ninety-four (or 4.1% of) inpatient admissions for individuals in MEPS who have a sample weight of 0 were excluded, which left a final MEPS sample size of 9288 inpatient admissions. MEPS assigns a sample weight of 0 to out-of-scope individuals residing in a MEPS household. These are active duty military personnel and certain individuals who first joined a MEPS household after MEPS households had been selected (Machlin et al., 2010). The exclusion of individuals who joined a MEPS household too late to be included is not a significant exclusion, because these individuals are presumably within the MEPS’ target population. Consequently, the MEPS weights would account for their utilization. The exclusion of active duty military personnel was considered in a sensitivity analysis.

Measures

The measures of inpatient utilization were total (U.S. aggregate) inpatient admissions per year, mean length-of-stay per inpatient admission, and total (U.S. aggregate) inpatient bed days per year. In both data sources, the first recorded condition in each inpatient record was used to classify admissions as either psychiatric or non-psychiatric. Psychiatric conditions were defined as AHRQ Clinical Classification Software (CCS) (Agency for Healthcare Research and Quality, 2013a) categories for mental health or substance use condition diagnoses (CCS 650–670), and all other CCS conditions were categorized as non-psychiatric. In MEPS, household respondents may report up to four conditions that contributed to an individual’s hospitalization, although only one condition is listed in most cases (83%). These responses are reported verbatim, and then later coded into CCS categories (Machlin, Cohen, Elixhauser, Beauregard, & Steiner, 2009). NIS records capture up to 25 separate diagnoses.

The use of only the first listed diagnosis to classify psychiatric admissions, rather than all listed diagnoses is conservative, because using all diagnoses would result in over-counting psychiatric admissions, especially in NIS, because psychiatric conditions often are recorded in hospital admissions records for individuals who received care primarily for a non-psychiatric condition (Slade, Dixon, & Semmel, 2010). However, use of only the first diagnosis could result in an undercount of psychiatric admissions, possibly in both data sources, as there might be cases where a psychiatric condition that led to the admission was not listed first. Consequently, the importance of using only the first diagnosis was examined in a sensitivity analysis.

Inpatient records corresponding to the following specific conditions were identified using first listed diagnosis: HIV and hepatitis infection (CCS 5, 6), cancer (CCS 11–41), diabetes mellitus (CCS 49, 50), epilepsy (CCS 83), headache including migraine (CCS 84), limb fracture (CCS 229, 230), substance use disorders (CCS 660, 661), mood and anxiety disorders (CCS 651, 657), and schizophrenia and other psychotic disorders (CCS 659). Measures of inpatient utilization for these conditions were used to explore whether discrepancies were greater for conditions that raise greater disclosure concerns as a result of stigma and confidentiality issues. Greater discrepancies were expected for HIV, hepatitis, schizophrenia, mood disorders, and substance use disorders than were expected for limb fractures, headaches, and cancer. Epilepsy and diabetes were expected to be somewhat sensitive to these same disclosure concerns, but not to the same extent as sexually transmitted diseases and severe psychiatric conditions. These conditions were also selected because a prior validation study suggested that MEPS respondents’ self-reports of these conditions are comparatively reliable (S. Machlin et al., 2009).

Other individual characteristics examined included age, gender, insurance source, and Census region. For NIS, insurance source was the primary source of payment listed on each discharge record. For MEPS, insurance source was assigned based on the insurance coverage that individuals had for the most months during the year. Although specific sources of payment for each inpatient admission record are reported by the MEPS household respondent, this information may be unreliably reported.

Analyses

All estimates in both samples were weighted using sampling weights provided with the two databases and then averaged across years to obtain annualized statistics. Standard errors were obtained using survey-data estimation routines in STATA 13. The statistical significance of MEPS-NIS discrepancies was tested using standard normal test statistics.

RESULTS

Table 2 shows that compared with the NIS, the MEPS indicated 82% as many total admissions (15009±317 thousands in MEPS and 18233±480 thousands in NIS; z=−11.0, p<.001); 34% as many psychiatric admissions (545±86 thousands in MEPS and 1587±261 thousands in NIS; z=−7.4, p<.001); and 86% as many non-psychiatric admissions (14345±308 thousands in MEPS and 16646±403 thousands in NIS; z=−8.8, p<.001). Compared with NIS, the MEPS also indicated 99% as many bed days for all admissions (78.4±3.4 million days in MEPS and 79.1±4.0 million days in NIS; z=−0.3, p=786); 48% as many inpatient bed days for psychiatric admissions (5.2±1.2 million days in MEPS and 10.7±1.7 million days in NIS; z=−5.3, p<.001), and 104% as many inpatient bed days for non-psychiatric admissions (72.5±3.1 million days in MEPS and 69.9±3.3 million days in NIS; z=−1.1, p=.254). Estimated mean length-of-stay was greater in MEPS than in NIS for all admissions (4.87±0.15 days in MEPS and 4.34±0.22 days in NIS; z=3.9, p<.001), psychiatric admissions (9.25±1.35 days in MEPS and 6.75±1.06 days in NIS; z=2.9, p=.004), and non-psychiatric admissions (4.74±0.15 days in MEPS and 4.20±0.20 days in NIS; z=4.3, p<.001).

Table 2.

Inpatient admissions, length of stay and bed days

	MEPS (N=9288)a			NIS (N=21934378)b
Admissions (thousands)	total	±	95% CI	total	±	95% CI	z	p	MEPS/NIS
All conditions	15009	±	317	18233	±	480	−11.0	<0.001	0.82
Psychiatric	545	±	86	1587	±	261	−7.4	<0.001	0.34
Non-psychiatric	14345	±	308	16646	±	403	−8.8	<0.001	0.86
Not classifiable	119	±	18	na		na	na	na	na
Bed days (millions)	total	±	95% CI	total	±	95% CI	z	p	MEPS/NIS
All Conditions	78.4	±	3.3	79.1	±	4.0	−0.3	0.786	0.99
Psychiatric	5.2	±	1.2	10.7	±	1.7	−5.3	<0.001	0.48
Non-psychiatric	72.5	±	3.1	69.9	±	3.3	−1.1	0.254	1.04
Not classifiable	.7	±	0.1	na		na	na	na	na
Length of stay (days)	mean	±	95% CI	mean	±	95% CI	z	p	MEPS/NIS
All Conditions	4.87	±	0.15	4.34	±	0.22	3.9	<0.001	1.12
Psychiatric	9.25	±	1.35	6.75	±	1.06	2.9	0.004	1.37
Non-psychiatric	4.74	±	0.15	4.20	±	0.20	4.3	<0.001	1.13
Not classifiable	6.25	±	1.67	na		na	na	na	na

^aMEPS had 349 psychiatric admissions and 8852 non-psychiatric admissions, and 87 admissions could not be classified because no condition or reason for inpatient admission was reported.

^bNIS had 1903024 psychiatric admissions and 20025084 non-psychiatric admissions. n=6270 admissions with no listed diagnosis, representing 0.03% of all NIS admissions, were dropped.

Table 3 shows total inpatient admissions for each of the nine selected condition categories, sorted in order of the ratio of MEPS totals to NIS totals. MEPS admissions totals were nominally less than corresponding NIS totals for all conditions except limb fracture admissions. However, MEPS totals did not differ significantly from the NIS totals for limb fracture admissions (p=0.575), cancer admissions (p=0.634), headache admissions (p=0.632), or epilepsy admissions (p=0.152). MEPS totals for the other five conditions were significantly less than NIS totals (all p<.05): diabetes (73% of the NIS total), mood & anxiety disorders (55% of the NIS total), hepatitis & HIV (33% of the NIS total), schizophrenia & other psychotic disorders (19% of the NIS total), and substance use disorders (15% of the NIS total).

Table 3.

Total inpatient admissions (1000’s), for selected conditions

		MEPS				NIS
Condition category	n	total	±	95% CI	n	total	±	95% CI	z	p	MEPS/NIS
Limb fracture	132	238	±	68	258944	216	±	35	0.56	0.575	1.10
Cancer	260	480	±	83	597030	502	±	41	−0.48	0.634	0.96
Headache, including migraine	30	54	±	23	73161	61	±	14	−0.48	0.632	0.89
Epilepsy	74	113	±	37	175414	146	±	27	−1.43	0.152	0.77
Diabetes	192	241	±	30	398954	331	±	43	−2.47	0.014	0.73
Mood & anxiety disorders	243	381	±	69	840147	699	±	165	−3.49	<0.001	0.55
Hepatitis & HIV	16	29	±	17	107849	89	±	30	−3.41	0.001	0.33
Schizophrenia & other psychotic disorders	37	61	±	28	380267	317	±	131	−3.86	<0.001	0.19
Substance use disorders	37	65	±	32	520924	436	±	153	−4.64	<0.001	0.15

Table 4 shows the MEPS and NIS population distributions of age category, gender, insurance source, and region for all inpatient and for psychiatric inpatients. Regarding all inpatients, statistically significant (p<0.05) differences were found for gender, insurance source, and region. However, discrepancies for insurance source were larger than discrepancies for gender and region. Compared with the NIS inpatients, MEPS inpatients were less likely to be Medicare enrollees (11.3% in MEPS and 14.4% in NIS; z=−3.0, p=.002), Medicaid enrollees (16.7% in MEPS and 21.9% in NIS; z=−5.8, p<0.001), and enrollees in other public insurance programs (2.1% in MEPS and 6.0% in NIS; z=−2.3, p=.020). MEPS inpatients also were more likely to be privately insured (57.6% in MEPS and 49.0% in NIS; z=11.4, p<.001) and self-pay or uninsured individuals (12.1% in MEPS and 8.7% in NIS; z=4.5, p<.001). Among psychiatric inpatients (MEPS N=349, NIS N=1903024) there was only one significant difference. MEPS inpatients were less likely to be male (43.9% in MEPS and 54.1% in NIS; z=−2.3, p=.021).

Table 4.

Characteristics of inpatients ages 22 to 64

	MEPS (N=9288)			NIS (N=21934378)
	n	total (1000s)a	%a	n	total (1000s)a	%a	z	p	MEPS%/NIS%
All Inpatients
Age group
22–44	4642	7279	48.5	10706659	8898	48.8	−0.5	0.586	1.00
45–64	4646	7730	51.5	11227719	9335	51.2			1.00
Gender
Male	2863	5185	34.5	8397835	6983	38.3	−4.2	<0.001	0.92
Female	6425	9824	65.5	13536543	11250	61.7			1.05
Insurance Sourceb
Medicare	1073	1694	11.3	3154410	2626	14.4	−3.0	0.002	0.78
Medicaid	1914	2512	16.7	4810108	3993	21.9	−5.8	<0.001	0.76
Other public	179	312	2.1	1316316	1094	6.0	−2.3	0.020	0.35
Private insurance	3965	8649	57.6	10744793	8934	49.0	11.4	<0.001	1.18
Self-Pay/Uninsured	1255	1816	12.1	1908751	1586	8.7	4.5	<0.001	1.39
Region
Northeast	1385	2569	17.1	4160695	3464	19.0	−1.8	0.072	0.90
Midwest	2078	3682	24.5	4828271	4011	22.0	2.8	0.006	1.12
South	3879	5903	39.3	8619383	7166	39.3	<0.1	0.999	1.00
West	1946	2855	19.0	4326029	3592	19.7	−0.8	0.438	0.97
Psychiatric Inpatientsc
Age group
22–44	181	326	59.8	1066920	890	56.1	1.0	0.309	1.07
45–64	168	219	40.2	836104	697	43.9			0.91
Gender
Male	127	239	43.9	1029116	858	54.1	−2.3	0.021	0.81
Female	222	306	56.1	873908	729	45.9			1.22
Insurance Sourceb
Medicare	67	110	20.2	426140	355	22.4	−0.4	0.664	0.90
Medicaid	108	150	27.5	581940	485	30.6	−0.7	0.491	0.90
Other public	6	8	1.5	132621	111	7.0	−0.5	0.597	0.21
Private insurance	91	179	32.8	493629	412	25.9	1.5	0.133	1.27
Self-Pay/Uninsured	75	96	17.6	268694	224	14.1	0.9	0.385	1.25
Region
Northeast	56	100	18.3	504431	421	26.5	−1.4	0.166	0.69
Midwest	93	156	28.6	488271	407	25.7	0.7	0.513	1.12
South	150	200	36.7	621134	518	32.6	1.1	0.289	1.12
West	50	89	16.3	289088	241	15.2	0.2	0.822	1.07

^aTotals and percentages are weighted estimates.

^bInsurance source was unavailable in MEPS data for 16 individuals who accounted for 22 admissions.

^cResults for non-psychiatric inpatients (not shown) were virtually identical to results for all inpatients.

As a sensitivity analysis, Table 5 provides information about the potential impact of non-elderly individuals living in long-term care institutions and prisons on the MEPS-NIS discrepancies in total inpatient admissions (see Table 2). Part or all of these discrepancies could be attributable to these individuals’ short-term hospitalizations, which generally are not included in the MEPS totals. The table shows the weighted distribution from the 2010 NIS of discharge destinations for individuals aged 22 to 64 years. Destination frequencies were aggregated into categories for discharges to institutional organizations and other destinations, respectively, based on category descriptions provided with the NIS data. Only 163 thousands (or 10.6% of) NIS psychiatric admissions and only 683 thousands (or 4.7% of) NIS non-psychiatric admissions were to institutional care providers. For psychiatric care, the added number is an overestimate, because it encompasses admissions for individuals who had short-term stays at psychiatric hospitals, which presumably are already reflected in the MEPS totals (i.e., MEPS does not exclude such admissions). Adding these admissions to the MEPS totals would somewhat reduce the size of the discrepancies with the NIS: the updated MEPS totals would be 708 thousands psychiatric admissions (or 44.6% of the NIS total) and 15028 thousands non-psychiatric admissions (or 90.3% of the NIS total).

Table 5.

Discharge destination in the 2010 NIS, by condition (N=3627942)^a

	Psychiatric Admissions			Non-Psychiatric Admissions
	n	Freq (1000s)b	%	N	Freq (1000s)b	%
Discharged to an institution/agency	32387	162.7	10.6	135418	683.2	4.7
Court/Law Enforcement	827	4.1	0.3	4289	21.7	0.2
Skilled nursing, long term care, nursing home, or intermediate care facility	10668	53.2	3.5	119605	603.5	4.2
Psychiatric hospital or specialty unit bed	17255	86.8	5.6	8220	41.6	0.3
Other institutions (types not identified)	3637	18.5	1.2	3304	16.3	0.1
Other known destinations	272605	1377.5	89.4	2724045	13751.1	95.3
Discharged to home or self-care, or left against medical advice	259250	1310.2	85.1	2583696	13044.3	90.4
Transferred to a different acute care hospital or setting	11500	57.8	3.8	89184	447.8	3.1
Died or transferred to hospice	1855	9.5	0.6	51165	259.1	1.8
Total	304992	1540.2	100.0	2859463	14434.3	100.0

^aDischarge destination was “unknown” or “missing” for 39328 (or 11.4% of) psychiatric discharges and 424159 (or 12.9% of ) non-psychiatric discharges.

^bWeighted frequencies.

DISCUSSION

This study is the first to show evidence that psychiatric inpatient utilization may be underrepresented in Medical Expenditure Panel Survey (MEPS) data. The MEPS estimate of total psychiatric admissions for non-elderly adults was only one-third (or 66% less than) the comparable total derived from national hospital discharge data. The discrepancy for total psychiatric bed days (52% fewer days in MEPS) was smaller than the discrepancy for psychiatric admissions, because estimated mean length-of-stay for psychiatric admissions was greater (by 2.5 days or 37%) in MEPS than in the hospital discharge data. Data on admissions discrepancies by diagnosis (Table 3) suggest that these discrepancies are larger for individuals with psychotic disorders and substance use conditions than for individuals with other mental health conditions. In addition, inpatient utilization attributable to publicly insured individuals was underrepresented more than was inpatient utilization attributable to privately insured individuals (Table 4).

The study’s results suggest that underrepresentation of individuals with severe psychiatric conditions, underreporting of psychiatric inpatient use by MEPS respondents, or both factors may affect the representativeness of psychiatric inpatient utilization data in MEPS. Individuals with severe psychiatric conditions may be especially difficult to recruit and retain in the MEPS sample (Gfroerer et al., 1997), reporting of psychiatric inpatient use may be suppressed as a result of stigma or privacy concerns (Bhandari & Wagner, 2006; Gfroerer et al., 1997; Golding et al., 1988; Harrison & Hughes, 1997; Killeen et al., 2004), and respondents who have multiple hospitalizations may fail to report all of them (Clark, Ricketts, & McHugo, 1996). Although individuals with severe psychiatric conditions comprise a small fraction of the total U.S. population, they represent a critically important population for assessments of U.S. health care quality and cost and illness morbidity (Dismuke & Egede, 2011; Himelhoch et al., 2004; Katon, 1996). This population also has a disproportionate share of individuals with chronic medical conditions, especially hepatitis, HIV and other blood-borne infectious diseases; diabetes; hypertension; and smoking-related conditions (Dixon et al., 1999; Jones et al., 2004). Such comorbidities could account for a portion of the unexplained discrepancy in non-psychiatric inpatient admissions for specific medical conditions (see Table 3).

These results are difficult to compare directly with evidence from prior research studies, because prior studies have focused on different methodological issues. Evidence from a prior assessment of Medicare enrollees’ services utilization information in MEPS (S. H. Zuvekas & Olin, 2009a), in which self-reported utilization data were compared to the same individuals’ Medicare claims data, showed that the number of self-reported inpatient admissions per person was 94% the number derived from Medicare claims. By contrast, this study found that 82% of total admissions were reported in MEPS. However, the Medicare study did not specifically examine discrepancies in the reporting of psychiatric inpatient care and did not measure discrepancies resulting from undersampling, factors that are consistent with the larger discrepancies found in this study. Studies comparing mean annual health care expenditures per person in MEPS and in administrative records find discrepancies ranging from 12% to 19% (Aizcorbe et al., 2012; S. H. Zuvekas & Olin, 2009b), with the MEPS’ means being consistently lower especially among individuals with above-average expenditures. Other, not MEPS-based, studies of reporting accuracy for psychiatric inpatient utilization find mixed results (Bhandari & Wagner, 2006; Golding et al., 1988; Killeen et al., 2004). These studies focus on factors affecting discrepancies between self-reported medical utilization and measures derived from medical records abstractions. Their results do not easily generalize to the national inpatient population context in this study.

Limitations of the MEPS data on psychiatric inpatient utilization should not necessarily be interpreted as evidence of broader limitations in MEPS data for psychiatric care. For many questions in psychiatric services and policy research, MEPS is more versatile and comprehensive than any other single health care database. It is national in scope, contains information for individuals who used services as well as for those who did not, and contains a rich set of personal and household characteristics. MEPS is particularly useful for examining receipt of mental health care and psychotropic medications (Marcus & Olfson, 2010), racial disparities in mental health care (Cook et al., 2014), and rates of insurance coverage among individuals with mental health problems (Garfield, Zuvekas, Lave, & Donohue, 2011). However, as a result of how the MEPS sample is designed and how the data are collected, MEPS inpatient data for psychiatric conditions, and for other conditions that frequently co-occur with psychiatric conditions, may be more problematic than data for other types of care.

To some extent, the MEPS’ limited capture of psychiatric inpatient utilization is an intentional consequence of its design. The MEPS was originally designed to capture health care utilization among individuals living in non-institutional “community” settings. Results on inpatient discharge destinations (Table 5) indicated that adjusting MEPS totals for general hospital stays by non-elderly adults residing in prisons, nursing homes, psychiatric hospitals, and other long-term care institutions reduces but does not eliminate the discrepancies in total admissions. The residual discrepancies in admissions totals reported here may help explain why national health care expenditures totals derived from MEPS are substantially less than national health care expenditures totals derived from the National Health Accounts (Bernard et al., 2012), which are based on reports submitted by health care provider organizations.

Other implications of the study’s results are somewhat speculative. First, the greater magnitude of the discrepancies within categories for HIV & hepatitis, psychotic disorders, and substance use disorders compared to other conditions (e.g., limb fracture), suggests that underreporting of services use might be more problematic for conditions that engender a stronger sense of stigma. This interpretation is consistent with a prior finding, replicated in multiple studies, that reporting of substance use is more suppressed the more stigmatized is the drug (Harrison & Hughes, 1997). However, this interpretation could not be confirmed, and there are other plausible explanations for these findings, such as sample underrepresentation of individuals with severe psychiatric and substance use conditions and poorer recall of hospitalizations among individuals who tend to have more frequent hospital episodes. Concerns about underreporting in MEPS of psychiatric services use might be somewhat ameliorated by including a separate audio-computer-assisted self-interviewing (audio-CASI) section on psychiatric services use. Audio-CASI has been shown to result in more complete reporting of HIV risk behaviors, for example (Des Jarlais et al., 1999).

Second, discrepancies in the representation of inpatient utilization by Medicaid and Medicare enrollees (Table 4) might be a signal that individuals with severe psychiatric conditions are underrepresented in the MEPS sample, and, consequently, that MEPS data might underrepresent other types of psychiatric utilization besides inpatient care in this population. Specifically, data on antipsychotic prescriptions as well as emergency department, partial hospitalization, and case management services might all be affected. Assessments of utilization in these categories would require a different reference data source than the NIS. The most direct way to correct sample underrepresentation, namely adding special oversamples of individuals with severe psychiatric conditions to the MEPS sample, could prove infeasible given constraints on survey staff and other survey resources. Obtaining such oversamples would require substantial additional resources to overcome barriers to recruitment and respondent tracking in this population (Gfroerer et al., 1997). Absent an oversample, analysts could use post-stratification (i.e., sample re-weighting) techniques (Little, 1993) to increase the sample representation of underrepresented individuals when estimating total psychiatric inpatient utilization or total psychiatric expenditures in the U.S. However, post-stratification could result in greater inaccuracy if the underlying MEPS sample does not include even a minimal number of individuals from key subgroups or if post-stratification weights are correlated with underreporting of inpatient events.

Third, the finding that the mean of psychiatric inpatient lengths-of-stay was longer in MEPS than in NIS (9.25 days versus 6.75 days) suggests that many shorter-than-average psychiatric inpatient stays were omitted from MEPS. This pattern is consistent with various explanations. Longer stays could be more salient to respondents than shorter stays (Cannell, Miller, & Oksenberg, 1981), some respondents might be unaware of short psychiatric admissions by other household members or confused about whether a short stay would be considered an inpatient stay, and respondents with multiple admissions might be less likely to recall short stays compared with long stays. Respondents may also in some cases report a single hospitalization episode when an individual was actually transferred between hospitals or was discharged and then readmitted within a short time frame. This factor could also account for part of the discrepancy in total admissions. Individuals who tend to have short stays could also have been undersampled. This is consistent with the finding of larger discrepancies in admissions for substance use conditions, which are associated with below-average lengths-of-stay (Tulloch, Fearon, & David, 2011). A final possibility is that psychiatric inpatient admissions reported in MEPS may have included some admissions to specialized psychiatric care programs where lengths-of-stay are typically longer than in community hospitals, such as short-term residential rehabilitation programs and inpatient programs at psychiatric hospitals. These programs are specifically excluded from the NIS sample.

Regarding study limitations, although the study focused on individuals ages 22–64 in order to minimize the impacts of design differences, residual differences between the two data sources may have affected the estimated discrepancies. Community hospital admissions for institutional populations were discussed above. Two additional study design factors require mention. First, inpatient admissions to community hospitals by active-duty military personnel are counted in NIS but not in MEPS, which results in a discrepancy. Second, some admissions could have been misclassified as non-psychiatric in cases where a psychiatric diagnosis was not listed first. A sensitivity analysis suggested that these study design factors are unlikely to account for a majority of the MEPS-NIS discrepancy in total admissions (details are available from the authors by request; reviewers see Table A.1.). Community hospital admissions for active-duty military personnel are probably less than 10330 psychiatric admissions and 108400 non-psychiatric admissions. This adjustment would reduce the psychiatric discrepancy to 65%, from 66%, and the non-psychiatric discrepancy to 13%, from 14%. Re-classification of inpatient admissions in MEPS using all four diagnoses resulted in 57000 additional psychiatric admissions, or an increase of 10.5%, and an equal reduction in non-psychiatric admissions. However, analysis of NIS data indicated that using all listed diagnoses to classify admissions in NIS data would result in a much larger increase in psychiatric admissions, and consequently in an even larger MEPS-NIS discrepancy. This suggests that the first-listed-diagnosis approach yielded a more conservative estimate of the MEPS-NIS discrepancy for psychiatric admissions.

The results of this study raise concerns about using MEPS data to assess psychiatric inpatient utilization or expenditures, especially for individuals with severe psychiatric conditions and substance use disorders. In MEPS data, underrepresentation of psychiatric inpatient utilization at community hospitals may result in measurement distortions for commonly used statistics on psychiatric inpatient utilization and costs, including aggregate admissions and the mean of psychiatric lengths-of-stay. Although reweighting MEPS data could reduce these distortions, adjustments to the MEPS’ sample and methodology for respondent reporting of psychiatric inpatient services utilization should be considered. Undersampling of individuals with severe psychiatric conditions could be partially addressed using supplemental oversamples or post-stratification methods. Underreporting might be reduced by using audio-CASI techniques for questionnaire items on psychiatric inpatient services.

Appendix A: Sensitivity Analysis

Table A1 presents possible revisions of the Table 1 MEPS’ estimates of total admissions and corresponding MEPS-NIS discrepancies adjusted for three design factors:

• the MEPS’ omission community hospital admissions by individuals residing in long term care institutions;

• the MEPS’ omission of community hospital admissions for active-duty military personnel; and

• this study’s use of only the first listed diagnosis to classify admissions as psychiatric.

The table considers what would be the impacts on MEPS-NIS discrepancies of including community hospital admissions for individuals in long-term care (Line 2) and active-duty military personnel (Line 3) and reclassifying an admission as psychiatric if any of up to four diagnoses listed in MEPS admissions records is for a mental health or substance use condition (Line 4). A fourth set of estimates (in Line 5) shows the impacts of all three adjustments applied together. The notes below Table A1 provide more information about the assumptions that were used to develop the table.

The results in Table A1 indicate that the three adjustments applied together would result in somewhat smaller MEPS-NIS discrepancies. However, the revised MEPS total for psychiatric admissions (775 thousand) would still be only 49% of the NIS total, and the revised MEPS total for non-psychiatric admissions (15079 thousand) would be only 91% of the MEPS total. This suggests that most of the large MEPS-NIS discrepancy in psychiatric admissions and most of the moderate MEPS-NIS discrepancy in non-psychiatric admissions are not attributable to the three design factors.

For reference purposes, we also used NIS 2010 data to reclassify admissions as psychiatric if any of up to 25 listed Clinical Classification Software codes were for psychiatric conditions, and found that this increased the psychiatric admissions total by 38.2%, compared to an increase of 10.5% in the MEPS.

Table A.1.

Sensitivity of MEPS estimates of total inpatient admissions to study design changes

	Psychiatric		Non-Psychiatric MEPS
Adjustment	MEPS (1000s)	MEPS/NIS	(1000s)	MEPS/NIS
Line 1: Estimates from Table 1	545	0.34	14345	0.86
Line 2: Line 1 plus community admissions for individuals residing in institutions	708	0.45	15028	0.90
Line 3: Line 1 plus admissions for active-duty military	555	0.35	14453	0.87
Line 4: Reclassify admissions as psychiatric if any secondary psychiatric diagnoses	602	0.38	14288	0.86
Line 5: Line 1 plus adjustments in lines 2–4	775	0.49	15079	0.91

Notes

Line 1: Same estimates as were reported in Table 1

Line 2: Totals include an estimated 163 thousand psychiatric and 683 thousand non-psychiatric community hospital admissions for individuals residing in institutions. These estimates were developed using the NIS discharge data and are reported in the text corresponding to Table 4.

Line 3: Includes estimated admissions to community hospitals for active-duty military personnel. These estimates were deduced from the total number of active military personnel and an assumed hospitalization rate for this population. According to the U.S. Statistical Abstract (http://www.census.gov/compendia/statab/), in 2010, there were 1134000 active duty military personnel. We then estimate, using a U.S. Census estimate of 174137000 U.S. adults ages 18–64 in 2010, that the annual number of community hospitalizations per person in the population was .00911 for psychiatric admissions and .09559 for non-psychiatric admissions. Application of these rates to active duty military personnel results in an estimated 10330 psychiatric hospitalizations and 108400 non-psychiatric admissions. These numbers probably overestimate actual community hospital admissions among active-duty military personnel, because military personnel probably have a lower rate of admissions to community hospitals than do non-elderly adults in general. For one, many military admissions are to federal government hospitals (i.e., to military hospitals or the VA health care system), which are outside the community hospital system. In addition, active duty military personnel are on average younger than the general population, and therefore would be expected to have a lower rate of non-military-related hospitalizations than the general population. Finally, many active duty military personnel are deployed to bases in foreign countries, and therefore would not be admitted to a U.S. community hospital.

Line 4: Psychiatric total includes all admissions that had a psychiatric diagnosis listed in any position, not just first. This resulted in 57 thousand additional psychiatric admissions and 57 thousand fewer non-psychiatric admissions.

Line 5. These totals were calculated by summing the adjustments reflected in lines 2-4.