Abstract
Objective
Many states expanded their Medicaid managed care programs during the 1990s, causing concern about impacts on health care for affected populations. We investigate the relationship between Medicaid managed care enrollment and health care for children.
Data Sources and Measures
Repeated cross-sections of Medicaid-covered children under 18 years of age from the 1996/1997 and 1998/1999 Community Tracking Study Household Surveys (n=2,602) matched to state-year CMS Medicaid managed care enrollment data. For each individual, we constructed measures of health care utilization (provider and emergency department visits, hospitalizations, surgeries); health care access (usual source of care, unmet medical needs, put-off needed care); and satisfaction (satisfaction overall, with doctor choice, and with last visit).
Study Design
Regression analysis of the relationship between within-state changes in Medicaid managed care enrollment rates and changes in mean utilization, access, and satisfaction measures for children covered by Medicaid, controlling for a range of potentially confounding factors.
Principal Findings
Increases in Medicaid health maintenance organization (HMO) enrollment are associated with less emergency room use, more outpatient visits, fewer hospitalizations, higher rates of reporting having put off care, and lower satisfaction with the most recent visit. Medicaid primary care case management (PCCM) plans are associated with increases in outpatient visits, but also with higher rates of reporting unmet medical needs, putting off care, and having no usual source of care.
Conclusions
Both Medicaid HMO and PCCM plans can have important impacts on health care utilization, access, and satisfaction. Some impacts may be positive (e.g., less ED use and more outpatient provider use), although concern about increasing challenges in access to care and satisfaction is also warranted.
Keywords: Medicaid, managed care, primary care case management, utilization, access
Faced with rising expenditures for health care, hoping to improve access to care for their beneficiary population, and impressed by the apparent successes of managed care in the private sector, many state governments moved significant portions of their Medicaid populations into managed care over the last two decades (Holahan and Liska 1997; Holahan et al. 1998), with a substantial portion of the shift occurring during the mid- to late-1990s. Between 1996 and 1998, for example, the number of Medicaid recipients covered by managed care plans grew by nearly 3.5 million, from 40 percent of all recipients to nearly 54 percent (Kaiser Commission on Medicaid and the Uninsured 2001). This study investigates the impact of late-1990s’ growth in Medicaid managed care on children's health care utilization, access, and satisfaction. Children make up a substantial portion of the population covered by Medicaid, and improving care for children has been an important goal of the Medicaid program.
Background and Previous Literature
The adoption of Medicaid managed care typically entails shifts away from traditional fee-for-service (FFS) Medicaid to alternate financing and delivery models that attempt to use resources more effectively and control costs. Many efforts revolve around Medicaid health maintenance organizations (HMOs), which, like private sector HMOs, typically include some combination of financial incentives to providers, direct management of utilization, and restrictions on patient choices of providers. These plans also often engage in the collection and monitoring of utilization and quality information, the production and dissemination of guidelines, and other activities designed to improve the care delivered to their patient population and reduce costs. The other major Medicaid managed care approach, often referred to as primary care case management (PCCM), retains many features of FFS Medicaid, but aims to improve care by assigning beneficiaries to a primary care case manager to provide and help coordinate care.
Voluntary managed care programs, in which Medicaid recipients could choose to enroll in managed care plans, have been a part of Medicaid in some states for many years, but have seldom attracted large numbers of enrollees. During the recent expansions, most states adopted mandatory managed care in one form or another, requiring at least some portion of their Medicaid population to enroll in managed care plans. The timing and extent of mandatory Medicaid managed care implementations varied across states. This study focuses on the period 1996–1999, a period in which a number of mandatory managed care plans began or expanded. Thirty-nine states showed increases in the share of Medicaid enrollees in managed care plans during this period, 17 of which exceeded 20 percentage points, and 10 of which exceeded 30 percentage points. Twelve states showed declines during this period, only three of which exceeded 5 percentage points.
Although there are multiple reasons for growth in Medicaid managed care, the primary reason is legislative and regulatory action by state governments, typically in conjunction with federal government action on section 1915(b) or 1115 waivers, which are required for states to implement mandatory managed care programs. Differences in the timing of state legislative action, and in the timing of application for and approval of waivers, generate differences in the timing of managed care growth. These differences in timing, along with differences in the extent of the populations covered by the plans, generate differences in enrollment rates across states over time that are used here to identify the effects of Medicaid managed care.
Mandatory managed care plans vary across states in a number of dimensions. States vary in the requirements for enrollment, with some requiring managed care enrollment of all or nearly all of their Medicaid population and other states requiring enrollment only of a subset (e.g., those eligible because of their receipt of Aid to Families with Dependent Children (AFDC) or Temporary Assistance for Needy Families (TANF) aid). States may vary in the extent to which they rely on health plans that accept risk, as opposed to other models of health plan reimbursement, and in the extent to which they use PCCM models. States can also vary in a range of other ways, including the extent to which they rely on private plans as opposed to quasi-public or entirely public plans, the regulations they impose on contracting plans, and whether there are voluntary options for nonmandatory populations. For analysis, HMO-based plans are grouped together, as are PCCM-based plans. While there are differences between HMO-based plans and between PCCM-based plans, within these groups, plans typically share similar major features, general structures, and activities.
There are a number of previous studies on which this work builds. Within the broader literature on the impacts of managed care (e.g, Freund and Lewit 1993; Miller and Luft 1994; Miller and Luft 1997; Miller and Luft 2002; Szilagyi 1998), a number of studies have examined health care for children other than newborns in Medicaid managed care plans. Hurley, Freund, and Paul (1993), Freund and Lewit (1993), Kaiser Commission on the Future of Medicaid (1995), and Szilagyi (1998) summarize the main findings of studies conducted on early implementations of Medicaid managed care in the late 1980s and early 1990s. In general, these studies produced evidence of significant reductions in emergency department use, mixed and inconclusive evidence on access to care for children and primary care visits, and no evidence of consistent changes in hospitalization rates. Early studies also tended to report reductions in specialty care visits among children.
A handful of studies address the more recent experience of Medicaid managed care, frequently with stronger methods. Garrett, Davidoff, and Yemane (2003) studied expansions of Medicaid managed care using data from 1991 to 1995, and found consistent evidence that both mandatory HMOs and combined HMO/PCCM programs were associated with less emergency room use, although their results for other measures are less clear. Mandatory HMO programs, but not combined HMO/PCCM programs, were associated with increases in specialist visits. Combined HMO/PCCM programs, but not HMO programs, were associated with higher rates of reporting having a usual source of care, reporting unmet needs for prescription drugs, more physician visits, and more hospitalizations. Zuckerman, Brennan, and Yemane (2002) used a cross-sectional approach, and also found Medicaid HMOs to be associated with reductions in emergency room use and improvements in primary care use among children. They also found that PCCM programs also had some benefits for children. Long and Coughlin (2001) studied Medicaid managed care in Minnesota, and reported little differences in access to, use of, or satisfaction with health care services for children under Medicaid managed care relative to FFS.
This paper examines the effects of Medicaid managed care during a period of growth subsequent to that examined in most prior studies. Given the mixed findings on the overall effects in previous work, we also hope that expanding the evidence on the effects of these important policies will be valuable.
Identification Strategy
One approach to studying the effects of Medicaid managed care would be to compare the subset of Medicaid enrollees in managed care plans with the subset enrolled in FFS Medicaid. This type of comparison is appealing in some ways, but could be biased by selection if the characteristics of individuals in Medicaid managed care plans differ from the characteristics of individuals who are not.
We attempt to avoid this difficulty by examining average changes for the entire population covered by Medicaid when enrollment in Medicaid managed care plans rises.
This is done by estimating models of the form:
| (1) |
where Yi,s,t is a utilization, access, or satisfaction measure for individual i in state s at time t, HMO is the share of the state Medicaid population enrolled in a Medicaid HMO plan, PCCM is the share of the Medicaid population enrolled in a Medicaid PCCM plan, X is a set of individual characteristics or time-varying state characteristics that may be correlated with Y, STATE is a set of dummies for states to capture fixed state characteristics, YEAR is a dummy for years to capture time trends, and e is an error term.
This specification essentially asks whether changes over time in Medicaid managed care enrollment rates within states are associated with changes in the mean level of the utilization, access, or satisfaction measure of interest. This approach has important strengths. If moving individuals from FFS Medicaid into Medicaid managed care influences the dependent variables, this effect should be reflected in the average levels of the dependent variables. But, because the method tracks changes in the mean of the dependent variables across all Medicaid enrollees, and because the characteristics of the Medicaid population as a whole evolve relatively slowly over time (conditional on the control variables), this approach will greatly minimize the effects of selection bias because of variations in the characteristics of the subsets of the Medicaid population that do and do not end up in managed care. Looking at changes over time within states will also largely account for a wide range of other factors like urbanization and general features of the health care system that evolve slowly over time. Moreover, as the main reason for changes in Medicaid managed care enrollment rates within states over this time period are changes in state policies, and not the health status of the state Medicaid population as a whole, variations in Medicaid managed care enrollment rates should be largely exogenous.
Even with these strengths, the potential for confounding from variations in the characteristics of individuals is not entirely eliminated. For example, if the most aggressive Medicaid managed care states also have independent trends toward healthier Medicaid populations, results could be biased. To account for the potential for bias of this type, we rely on an extensive set of individual-level controls for demographics and health status in addition to the state and year fixed effects.
CTS Household Survey Data
Our data come from the public use versions of both the 1996/1997 and 1998/1999 CTS Household Surveys. The structure of the surveys is such that individual households are sampled, and if there are children under age 18 years in the household, one child is randomly sampled for in-depth data collection with data provided by a family respondent or the person who took the child to their last doctor visit. Unmarried children under age 18 years with no parent or guardian in the household are excluded. The CTS Household Survey mainly sampled households in 60 sites, supplemented with a nationwide sample of households. We use combined data from both the 60-site sample and the nationwide supplemental sample.
For analysis, we identified respondents who were under age 18 years and reported to be covered by Medicaid. Attention is restricted to individuals from states in which there were survey data from at least 30 Medicaid-covered children so that state-level fixed effects could be effectively included. In all, the 1996/1997 survey included 11,603 individuals under age 18 years. Of these, 1,331 (11.5 percent) were covered by Medicaid, and of these, 1,221 (91.7 percent) were in eligible states. The 1998/1999 survey contained information about 11,266 individuals under age 18 years. Of these, 1,465 (13 percent) were covered by Medicaid, and of these, 1,381 (94.3 percent) were in eligible states. Combining the 2 years, the sample includes 2,602 individuals, representing 24 states (AL, AR, AZ, CA, FL, GA, IL, IN, LA, MA, MD, ME, MI, MO, NC, NJ, NY, OH, OR, PA, SC, TX, UT, and WA).
These surveys have complex designs, so analyses are weighted to account for survey design and nonresponse. SUDAAN is used to compute standard errors where possible, although, as discussed further below, SUDAAN routines are not available for some of the estimators.
Health Care Utilization Measures
We define the following utilization measures, based on data provided by an adult family informant about the child's health care utilization: (1) an indicator of whether or not the child had an emergency room visit (other than a visit that led to an admission) in the past 12 months and the total number of visits; (2) an indicator for whether or not the child had a visit to a physician in the past 12 months, not including doctors seen in emergency rooms or during inpatient hospitalizations, and the total number of visits; (3) an indicator of whether or not the child had a visit to a nurse practitioner, physician assistant, or midwife in the past 12 months, not including visits as part of emergency room visits or inpatient hospitalizations, and the total number of visits; (4) an indicator for whether or not the child had an overnight hospital stay in the past 12 months excluding deliveries, the total number of hospitalizations, and the total number of hospital days; (5) an indicator for whether the child had outpatient surgery in the past 12 months, and the number of outpatient surgeries; and (6) an indicator for whether or not the child had inpatient surgery in the past 12 months, and the number of surgeries.
The number of emergency room visits, hospitalizations, inpatient, and outpatient surgeries are topcoded at 5. The number of visits to a physician assistant, nurse practitioner, or midwife is topcoded at 4. The number of doctor visits is topcoded at 20. The estimation procedures explicitly incorporate this censoring, as discussed below. Imputed values for missing responses are available for these measures, and are incorporated in the analysis.
Access to Care Measures
We constructed four measures of access to care, based on information provided by an adult family informant about the child's health care: (1) an indicator for whether or not the child had unmet medical needs in the past 12 months, from a question that asked whether there was any time when the child “didn't get the medical care [he or she] needed”; (2) an indicator for whether or not there was any time in the past 12 months when getting needed medical care had been put off or postponed; (3) an indicator for whether or not the child has a usual source of care (i.e., “a place that [the child] usually goes to when he/she is sick or needs advice about their health”); and (4) an indicator for having a usual source of care that is a doctor's office or an HMO, as opposed to a hospital outpatient clinic, some other clinic or health center, emergency room, or some other place. Imputed data are not available for these measures.
Measures of Perceived Quality and Satisfaction
We constructed five measures of satisfaction: (1) a measure of the family informant's overall satisfaction with the health care his or her family has received, available for all children in families where at least one person had a hospital or health care provider visit in the last 12 months, indicated on a five-point scale from very satisfied to very dissatisfied; (2) a measure of the family informant's satisfaction with the choice that they have of primary care physicians for the child, indicated on a five-point scale from very satisfied to very dissatisfied; (3) a measure of the perceived thoroughness and carefulness of the examination and treatment received at the child's last doctor visit, by the adult who accompanied the child to the visit, available for those who had a visit in the past 12 months, expressed on a five-point scale from excellent to poor; (4) a rating of how well the doctor listened at the child's last visit, by the adult who accompanied the child to the visit, available for those who had a visit in the past 12 months, expressed on a five-point scale from excellent to poor; and (5) a rating of how well the doctor explained things in an understandable way at the last visit, by the adult who accompanied the child to the visit, available for those who had a visit in the past 12 months, expressed on a five-point scale from excellent to poor. Imputed data are not available for these measures.
Control Variables
A number of additional variables were constructed from use as control variables in the regression analyses. These are: age (in categories: 0, 1, 2–3, 4–6, 7–10, 11–14, 16–18 years), sex, race (white, black, other race), the highest level of education reached by any member of the household in years (in categories, <12, 12, 13–15, 16, >16 years), whether or not any member of the household is a current smoker, family income (in categories: $0–$4,999, $5,000–$9,999, $10,000–$19,999, $20,000–$29,999, $30,000–$39,999, $40,000–$49,999, $50,000 and higher), family type (married or single without children, married parents with children, single parent with children, nonnuclear family), family size (in categories, 1–2, 3–5, or 6–8 persons), child's health status as reported by the family informant (excellent, very good, good, fair, poor), and parent's health status (excellent, very good, good, fair, or poor).
Medicaid Managed Care Data
For each child, we matched data on the level of enrollment in Medicaid managed care in his or her state each year to the CTS data. These data were obtained from publicly available CMS reports of Medicaid managed care enrollment, compiled as of June 30 each year. These reports provide information about total enrollment, as well as enrollment in HMOs and enrollment in PCCM plans. Because the CTS surveys are conducted over time, and some questions ask about previous experiences, it is not always clear whether the current or the prior year of Medicaid managed care enrollment data is the most appropriate match to a given survey year. Enrollment information as of June 30, 1997 is matched to the 1996/1997 survey and enrollment information as of June 30, 1999 to the 1998/1999 survey. The results are robust to varying the year of the enrollment data matched to the CTS data.
Results
Tables 1 and Table 2 report means of the utilization, access, and satisfaction measures in the 2 years. There are only a few cases in which there are statistically significant changes in the mean response patterns between the 2 years. Table 3 reports means of other sample characteristics. Most measures are stable over time, with somewhat better child health status and more married parents in the later survey. State HMO market share is somewhat higher in the later year, as expected with the expansion of Medicaid managed care.
Table 1.
Summary of Utilization and Access Measures
| 1996/1997 | 1998/1999 | ||||
|---|---|---|---|---|---|
| Measure | N | Value | N | Value | p-Value |
| Emergency room visits in the last 12 months | |||||
| % with any visit | 1,221 | 32.3 | 1,381 | 30.8 | .941 |
| Mean number of visits | 1,221 | 0.68 | 1,381 | 0.61 | .990 |
| Physician visits in the last 12 months | |||||
| % with any visit | 1,221 | 86.7 | 1,381 | 85.5 | .551 |
| Mean number of visits | 1,221 | 4.30 | 1,381 | 3.99 | .137 |
| NP/PA/midwife visits in the last 12 months | |||||
| % with any visit | 1,221 | 15.9 | 1,381 | 15.4 | .530 |
| Mean number of visits | 1,221 | 0.32 | 1,381 | 0.30 | .286 |
| Hospitalizations in the last 12 months | |||||
| % any stay | 1,221 | 15.3 | 1,381 | 13.0 | .136 |
| Mean number of stays | 1,221 | 0.21 | 1,381 | 0.21 | .458 |
| Outpatient surgery in the last 12 months | |||||
| % any surgery | 1,221 | 6.1 | 1,381 | 6.2 | .121 |
| Mean number of surgeries | 1,221 | 0.07 | 1,381 | 0.08 | .225 |
| Inpatient surgeries in the last 12 months | |||||
| % any surgery | 1,221 | 1.9 | 1,381 | 2.7 | .006 |
| Mean number of surgeries | 1,221 | 0.02 | 1,381 | 0.03 | .006 |
| % with unmet medical needs in the last 12 months | 1,220 | 4.7 | 1,375 | 6.0 | .516 |
| % put off care in the last 12 months | 1,219 | 7.5 | 1,376 | 7.9 | .565 |
| % with usual source of care | 1,218 | 96.1 | 1,377 | 93.1 | .007 |
| % with usual source of care that is MD office or HMO | 1,221 | 55.5 | 1,381 | 52.8 | .450 |
Note: p-value from tests for equality across years. χ2 tests were used for the categorical (percent) variables, and t-tests were used for continuous (mean) variables.
HMO, health maintenance organization; NP, nurse practitioner; PA, physician assistant.
Table 2.
Summary of Satisfaction Measures
| Measure | 1996/1997 | 1998/1999 | p-Value |
|---|---|---|---|
| Overall satisfaction with health care received | |||
| N | 1,196 | 1,339 | .077 |
| % very satisfied | 54 | 58 | |
| % somewhat satisfied | 32 | 30 | |
| % neither satisfied nor dissatisfied | 1 | 1 | |
| % somewhat dissatisfied | 6 | 6 | |
| % very dissatisfied | 7 | 6 | |
| Satisfaction with choice of physician | |||
| N | 1,202 | 1,369 | .125 |
| % very satisfied | 68 | 71 | |
| % somewhat satisfied | 21 | 21 | |
| % neither satisfied nor dissatisfied | 1 | 1 | |
| % somewhat dissatisfied | 5 | 4 | |
| % very dissatisfied | 4 | 3 | |
| Thoroughness/carefulness of exam and treatment at last doctor visit | |||
| N | 977 | 1,082 | .193 |
| % excellent | 37 | 40 | |
| % very good | 30 | 31 | |
| % good | 21 | 20 | |
| % fair | 9 | 8 | |
| % poor | 3 | 2 | |
| How well the doctor listened at last visit | |||
| N | 972 | 1,078 | .185 |
| % excellent | 40 | 42 | |
| % very good | 28 | 30 | |
| % good | 20 | 19 | |
| % fair | 8 | 6 | |
| % poor | 4 | 3 | |
| How clearly the doctor explained things in an understandable way at last visit | |||
| N | 975 | 1,080 | .164 |
| % excellent | 43 | 44 | |
| % very good | 27 | 28 | |
| % good | 19 | 19 | |
| % fair | 7 | 6 | |
| % poor | 4 | 3 | |
Note: p-values from χ2 tests of the hypothesis that the 1996/1997 distribution is equal to the 1998/1999 distribution.
Table 3.
Summary of Individual Characteristics
| 1996/1997 | 1998/1999 | ||||
|---|---|---|---|---|---|
| Characteristic | Mean | Standard Deviation | Mean | Standard Deviation | p-Value |
| N | 1,221 | 1,381 | |||
| Age (years) | |||||
| 0 | 0.095 | 0.293 | 0.083 | 0.276 | .274 |
| 1 | 0.093 | 0.291 | 0.080 | 0.271 | |
| 2–3 | 0.170 | 0.375 | 0.135 | 0.342 | |
| 4–6 | 0.179 | 0.383 | 0.176 | 0.381 | |
| 7–10 | 0.196 | 0.397 | 0.222 | 0.415 | |
| 11–14 | 0.144 | 0.351 | 0.159 | 0.366 | |
| 15–18 | 0.124 | 0.329 | 0.146 | 0.353 | |
| Sex | |||||
| Male | 0.495 | 0.500 | 0.497 | 0.500 | .759 |
| Female | 0.505 | 0.500 | 0.503 | 0.500 | |
| Race | |||||
| Race 1 (white) | 0.407 | 0.491 | 0.414 | 0.493 | .532 |
| Race 2 (black) | 0.321 | 0.467 | 0.337 | 0.473 | |
| Race 3 (other) | 0.272 | 0.445 | 0.248 | 0.432 | |
| Highest education level (years) | |||||
| Less than 12 | 0.210 | 0.407 | 0.198 | 0.399 | .796 |
| 12 | 0.460 | 0.499 | 0.478 | 0.499 | |
| 13–15 | 0.243 | 0.429 | 0.235 | 0.424 | |
| 16 | 0.062 | 0.242 | 0.067 | 0.249 | |
| 17 or more | 0.025 | 0.155 | 0.022 | 0.146 | |
| Household smokers | |||||
| Any smokers | 0.491 | 0.500 | 0.492 | 0.500 | .308 |
| No smokers | 0.509 | 0.500 | 0.508 | 0.500 | |
| Family income | |||||
| <$5,000 | 0.170 | 0.376 | 0.140 | 0.348 | .053 |
| $5,000–$9,999 | 0.209 | 0.407 | 0.185 | 0.388 | |
| $10,000–$19,999 | 0.294 | 0.456 | 0.299 | 0.458 | |
| $20,000–$29,999 | 0.143 | 0.351 | 0.159 | 0.365 | |
| $30,000–$39,999 | 0.063 | 0.243 | 0.090 | 0.286 | |
| $40,000–$49,999 | 0.042 | 0.200 | 0.046 | 0.210 | |
| $50,000 or more | 0.079 | 0.269 | 0.080 | 0.272 | |
| Family type | |||||
| Married/single, no children | 0.012 | 0.110 | 0.014 | 0.120 | .044 |
| Married parents, w/children | 0.241 | 0.428 | 0.273 | 0.446 | |
| Single parent, w/children | 0.622 | 0.485 | 0.586 | 0.493 | |
| Other, nonnuclear | 0.125 | 0.331 | 0.127 | 0.333 | |
| Family size | |||||
| 1–2 persons | 0.301 | 0.459 | 0.298 | 0.458 | .890 |
| 3–5 persons | 0.642 | 0.480 | 0.627 | 0.484 | |
| 6–8 persons | 0.057 | 0.233 | 0.075 | 0.263 | |
| Child's health status | |||||
| Excellent | 0.412 | 0.492 | 0.439 | 0.496 | .014 |
| Very good | 0.301 | 0.459 | 0.274 | 0.446 | |
| Good | 0.188 | 0.391 | 0.196 | 0.397 | |
| Fair | 0.084 | 0.277 | 0.075 | 0.264 | |
| Poor | 0.015 | 0.121 | 0.015 | 0.122 | |
| Parent's health status | |||||
| Excellent | 0.143 | 0.350 | 0.154 | 0.361 | .113 |
| Very good | 0.283 | 0.451 | 0.280 | 0.449 | |
| Good | 0.328 | 0.470 | 0.311 | 0.463 | |
| Fair/poor | 0.247 | 0.431 | 0.255 | 0.436 | |
| Medicaid HMO market share | 0.281 | 0.205 | 0.329 | 0.262 | .056 |
| Medicaid PCCM market share | 0.178 | 0.216 | 0.152 | 0.227 | .789 |
Note: p-value from tests for equality across years. χ2 tests were used for the categorical (percent) variables, and t-tests were used for continuous (mean) variables.
HMO, health maintenance organization; PCCM, primary care case management.
Table 4 and Table 5 report regression results for utilization measures. We use logistic regressions for dichotomous utilization measures (e.g., any doctor visits), and SUDAAN to compute standard errors adjusting for design effects. To analyze continuous count measures (e.g., number of doctor visits), we use censored Poisson models, which account for topcoding. All observations are used in the Poisson models, including those with no utilization for the given measure. There do not appear to be statistical routines available that allow for the computation of survey-design-adjusted standard errors for the censored Poisson model. We thus compute robust standard errors (White 1982) for the censored Poisson models. We hope that these are conservative. While robust errors need not always be conservative with data from surveys with complex designs, in several test cases, using logistic regression models where we could compute both design-adjusted and robust standard errors, the robust standard errors were larger than the design-adjusted standard errors.
Table 4.
Results from Analysis of Visit Utilization
| Emergency Department Visits | Physician Visits | NP/PA/Midwife Visits | ||||
|---|---|---|---|---|---|---|
| Any (Logit) | Number (Censored Poisson) | Any (Logit) | Number (Censored Poisson) | Any (Logit) | Number (Censored Poisson) | |
| Medicaid HMO market share | −1.015** | −0.741* | −0.449 | −0.095 | 0.030 | 0.832* |
| (0.394) | (0.309) | (0.666) | (0.103) | (0.913) | (0.413) | |
| Medicaid PCCM market share | 0.716 | 0.285 | 1.093 | 0.415** | 1.744** | 1.505** |
| (0.389) | (0.250) | (0.721) | (0.081) | (0.433) | (0.260) | |
| Age (years) | ||||||
| 0 | −0.342 | −0.255 | 1.366** | 0.430** | 0.860* | 0.709** |
| (0.322) | (0.156) | (0.366) | (0.049) | (0.389) | (0.161) | |
| 1 | 0.490 | 0.275* | 1.512** | 0.670** | 1.070** | 0.548** |
| (0.262) | (0.137) | (0.449) | (0.048) | (0.345) | (0.172) | |
| 2, 3 | 0.414* | 0.345** | 1.496** | 0.363** | 0.402 | 0.333* |
| (0.212) | (0.113) | (0.270) | (0.044) | (0.298) | (0.150) | |
| 4–6 | −0.224 | −0.195 | 0.845** | 0.126** | 0.352 | 0.047 |
| (0.181) | (0.118) | (0.256) | (0.043) | (0.330) | (0.150) | |
| 7–10 | −0.288 | −0.028 | 0.805** | 0.194** | 0.171 | 0.120 |
| (0.188) | (0.108) | (0.242) | (0.041) | (0.333) | (0.140) | |
| 11–14 | −0.175 | −0.189 | 0.738** | −0.018 | −0.107 | −0.220 |
| (0.215) | (0.121) | (0.243) | (0.045) | (0.299) | (0.157) | |
| Male | 0.100 | 0.096 | −0.122 | −0.057** | −0.056 | 0.020 |
| (0.112) | (0.060) | (0.130) | (0.021) | (0.131) | (0.076) | |
| Year=1998/1999 | −0.051 | −0.047 | −0.085 | −0.008 | 0.002 | −0.025 |
| (0.123) | (0.062) | (0.173) | (0.022) | (0.154) | (0.079) | |
| Black | −0.370** | −0.327** | −0.327 | −0.370** | −0.222 | −0.132 |
| (0.133) | (0.079) | (0.234) | (0.029) | (0.151) | (0.098) | |
| “Other” race | −0.651** | −0.478** | −0.393 | −0.171** | −0.619** | −0.576** |
| (0.149) | (0.089) | (0.237) | (0.029) | (0.236) | (0.111) | |
| Highest education (years) | ||||||
| ≤12 | −0.081 | −0.055 | 0.570** | 0.138** | 0.321 | 0.293** |
| (0.168) | (0.080) | (0.196) | (0.029) | (0.189) | (0.101) | |
| 13–15 | 0.363** | 0.258** | 0.533 | 0.168** | 0.140 | 0.097 |
| (0.133) | (0.089) | (0.278) | (0.033) | (0.190) | (0.121) | |
| 16 | 0.104 | −0.098 | 0.986* | 0.117* | −1.040** | −0.868** |
| (0.254) | (0.157) | (0.387) | (0.051) | (0.405) | (0.258) | |
| >16 | −0.541 | −0.666* | 0.903 | 0.351** | 0.019 | 0.380 |
| (0.398) | (0.317) | (0.616) | (0.076) | (0.307) | (0.281) | |
| Any current smoker in household | 0.113 | 0.035 | −0.285 | −0.021 | −0.200 | −0.218** |
| (0.106) | (0.064) | (0.158) | (0.022) | (0.162) | (0.081) | |
| Family income | ||||||
| $5,000–$9,999 | 0.120 | 0.098 | 0.023 | 0.118** | 0.288 | 0.128 |
| (0.155) | (0.093) | (0.242) | (0.035) | (0.216) | (0.121) | |
| $10,000–$19,999 | −0.111 | −0.169 | 0.114 | 0.056 | −0.003 | 0.101 |
| (0.143) | (0.095) | (0.263) | (0.034) | (0.233) | (0.118) | |
| $20,000–$29,999 | −0.246 | −0.270* | −0.062 | 0.024 | −0.089 | −0.116 |
| (0.190) | (0.117) | (0.309) | (0.041) | (0.254) | (0.150) | |
| $30,000–$39,999 | −0.330 | −0.066 | −0.210 | −0.121* | −0.388 | −0.116 |
| (0.229) | (0.139) | (0.327) | (0.053) | (0.395) | (0.186) | |
| $40,000–$49,999 | −0.105 | 0.030 | −0.447 | 0.169** | −0.223 | −0.393 |
| (0.294) | (0.168) | (0.374) | (0.060) | (0.403) | (0.258) | |
| ≥$50,000 | −0.254 | −0.253 | 0.038 | 0.011 | −0.020 | −0.075 |
| (0.256) | (0.151) | (0.376) | (0.053) | (0.364) | (0.190) | |
| Family type | ||||||
| Married/single, no children | 1.033* | 0.634* | 0.037 | −0.219 | −0.234 | −0.531 |
| (0.464) | (0.248) | (0.549) | (0.139) | (0.596) | (0.457) | |
| Married parents, w/children | 0.123 | 0.014 | −0.469 | −0.087* | −0.604* | −0.446** |
| (0.253) | (0.111) | (0.315) | (0.038) | (0.268) | (0.125) | |
| Single parent, w/children | 0.051 | 0.001 | −0.208 | −0.013 | −0.300 | −0.416** |
| (0.178) | (0.102) | (0.339) | (0.036) | (0.200) | (0.113) | |
| Family size | ||||||
| 3–5 | −0.156 | −0.179* | −0.020 | −0.073* | −0.084 | −0.152 |
| (0.123) | (0.085) | (0.165) | (0.030) | (0.138) | (0.107) | |
| 6–8 | −0.550* | −0.299* | −0.048 | −0.316** | 0.077 | −0.047 |
| (0.247) | (0.125) | (0.266) | (0.044) | (0.320) | (0.146) | |
| Child's health status | ||||||
| Very good | 0.076 | 0.117 | 0.236 | 0.119** | −0.244 | −0.359** |
| (0.163) | (0.077) | (0.184) | (0.027) | (0.169) | (0.106) | |
| Good | 0.377 | 0.282** | 0.151 | 0.426** | 0.336 | 0.354** |
| (0.213) | (0.085) | (0.204) | (0.029) | (0.203) | (0.101) | |
| Fair | 0.477* | 0.481** | 0.013 | 0.640** | 0.321 | 0.401** |
| (0.230) | (0.106) | (0.299) | (0.037) | (0.236) | (0.133) | |
| Poor | 0.524 | 0.405 | 0.025 | 0.924** | 0.853 | 0.889** |
| (0.655) | (0.231) | (0.596) | (0.066) | (0.490) | (0.203) | |
| Parent's health status | ||||||
| Very good | 0.260 | −0.069 | 0.418 | 0.143** | −0.667** | −0.538** |
| (0.192) | (0.105) | (0.231) | (0.037) | (0.229) | (0.128) | |
| Good | 0.353 | 0.055 | 0.563** | 0.105** | −0.451 | −0.149 |
| (0.200) | (0.102) | (0.215) | (0.037) | (0.301) | (0.120) | |
| Fair/poor | 0.566* | 0.282** | 0.366* | 0.141** | −0.440 | 0.001 |
| (0.244) | (0.103) | (0.188) | (0.038) | (0.276) | (0.124) | |
| Constant | −0.819* | −0.623** | 0.679 | 0.779** | −1.426* | −1.462** |
| (0.349) | (0.235) | (0.497) | (0.086) | (0.739) | (0.314) | |
| Observations | 2,602 | 2,602 | 2,602 | 2,602 | 2,602 | 2,602 |
| State dummies | Yes | Yes | Yes | Yes | Yes | Yes |
| Predictions, holding covariates at their sample means | ||||||
| HMO market share | ||||||
| 0.30 | 0.304 | 0.434 | 0.875 | 3.611 | 0.138 | 0.230 |
| 0.40 | 0.283 | 0.403 | 0.870 | 3.577 | 0.139 | 0.250 |
| 0.50 | 0.263 | 0.374 | 0.865 | 3.543 | 0.139 | 0.271 |
| 0.60 | 0.244 | 0.348 | 0.860 | 3.510 | 0.140 | 0.295 |
| PCCM share | ||||||
| 0.30 | 0.324 | 0.449 | 0.890 | 3.816 | 0.169 | 0.283 |
| 0.40 | 0.340 | 0.462 | 0.900 | 3.977 | 0.195 | 0.329 |
| 0.50 | 0.357 | 0.475 | 0.907 | 4.146 | 0.224 | 0.383 |
| 0.60 | 0.374 | 0.489 | 0.918 | 4.322 | 0.255 | 0.445 |
Note: Standard errors are in parentheses.
.01<p<.05.
p<.01.
NP, nurse practitioner; PA, physician assistant; HMO, health maintenance organization; PCCM, primary care case management.
Table 5.
Results from Analysis of Hospital and Surgery Utilization
| Hospital Stays | Inpatient Surgeries | Outpatient Surgeries | ||||
|---|---|---|---|---|---|---|
| Any (Logit) | Number (Censored Poisson) | Any (Logit) | Number (Censored Poisson) | Any (Logit) | Number (Censored Poisson) | |
| Medicaid HMO market share | −1.107 | −1.520** | −0.666 | −1.081 | 0.160 | 0.419 |
| (0.859) | (0.541) | (1.521) | (1.509) | (0.995) | (0.728) | |
| Medicaid PCCM market share | −1.017 | −0.886 | −0.268 | −1.165 | 0.937 | 0.363 |
| (0.883) | (0.493) | (1.185) | (1.332) | (0.749) | (0.544) | |
| Observations | 2,602 | 2,602 | 2,602 | 2,602 | 2,602 | 2,602 |
| Predictions, holding covariates at their sample means | ||||||
| HMO market share | ||||||
| 0.30 | 0.080 | 0.093 | 0.007 | 0.006 | 0.052 | 0.053 |
| 0.40 | 0.072 | 0.080 | 0.006 | 0.005 | 0.053 | 0.056 |
| 0.50 | 0.065 | 0.068 | 0.006 | 0.004 | 0.054 | 0.058 |
| 0.60 | 0.058 | 0.059 | 0.005 | 0.004 | 0.054 | 0.061 |
| PCCM share | ||||||
| 0.30 | 0.070 | 0.081 | 0.006 | 0.005 | 0.058 | 0.056 |
| 0.40 | 0.063 | 0.074 | 0.006 | 0.004 | 0.064 | 0.058 |
| 0.50 | 0.057 | 0.068 | 0.006 | 0.004 | 0.070 | 0.061 |
| 0.60 | 0.052 | 0.062 | 0.006 | 0.003 | 0.076 | 0.063 |
Note: Standard errors are in parentheses.
p<.01. Models also include the other covariates shown in Table 4.
HMO, health maintenance organization; PCCM, primary care case management.
Columns 1 and 2 of Table 4 show the results from models of emergency department utilization. For the probability of having an emergency department visit, there is a statistically significant negative effect of the share of the population in Medicaid HMOs. The bottom half of Table 5 reports predicted probabilities of a visit calculated from the regression results, holding the covariates fixed at their sample means. An increase in Medicaid HMO market share from 30 to 40 percent is associated with a decline of about 2.1 percentage points in the probability of having an emergency department visit. The second column reports results from a censored Poisson model of the number of emergency department visits. Here, there is also a negative and statistically significant coefficient for Medicaid HMO market share, consistent with a decline of 0.03 visits per person for every 10 percentage point increase in Medicaid HMO market share. There are no statistically significant effects of Medicaid PCCM market share on emergency department visit rates.
For physician visits, coefficients for Medicaid HMO market share are negative but statistically insignificant. Coefficients for Medicaid PCCM share are positive, and increases in PCCM share are statistically significantly associated with increases in the number of visits. Increases in Medicaid HMO share are also associated with increases in the number of nurse practitioner, physician assistant, and midwife visits. Increases in PCCM share are associated with increases in the probability of having any visit and in the number of visits.
Table 5 reports the results for hospitalization and surgery measures. Increases in HMO market share are associated with decreases in the number of hospitalizations. While coefficients for surgeries tend to go in expected directions—more managed care decreasing inpatient surgeries and increasing outpatient surgeries—the results are insignificant.
Table 6 reports the results from logistic regression analyses of the access measures. Increases in Medicaid HMO market share are associated with increases in the probability that needed medical care is reported to have been put off. Increases in PCCM share are significantly associated with an increase in the probability of reporting unmet medical needs and with a decrease in the probability of reporting having a usual source of care. About half of the effect on usual source of care is related to reductions in the share of people who say that their usual source of care is a physician office or HMO, suggesting that some of the decline is also associated with reductions in other usual sources of care.
Table 6.
Results from Analysis of Access Measures
| Had Unmet Medical Needs in the Last 12 Months | Put Off Care in the Last 12 Months | Has Usual Source of Care | Has Usual Source of Care That Is MD Office or HMO | |
|---|---|---|---|---|
| Medicaid HMO market share | −0.423 | 1.447* | −1.071 | 0.611 |
| (0.790) | (0.704) | (1.355) | (0.424) | |
| Medicaid PCCM market share | 1.415** | 1.003 | −2.336** | −0.145 |
| (0.556) | (0.539) | (0.635) | (0.517) | |
| Observations | 2,595 | 2,595 | 2,589 | 2,602 |
| Predictions, holding covariates at their sample means | ||||
| HMO market share | ||||
| 0.30 | 0.033 | 0.053 | 0.973 | 0.555 |
| 0.40 | 0.032 | 0.060 | 0.970 | 0.570 |
| 0.50 | 0.031 | 0.069 | 0.966 | 0.585 |
| 0.60 | 0.030 | 0.079 | 0.962 | 0.599 |
| PCCM share | ||||
| 0.30 | 0.041 | 0.061 | 0.963 | 0.551 |
| 0.40 | 0.047 | 0.067 | 0.953 | 0.546 |
| 0.50 | 0.054 | 0.074 | 0.942 | 0.542 |
| 0.60 | 0.062 | 0.082 | 0.928 | 0.538 |
Note: Results shown are coefficients from logistic regression models. Standard errors are in parentheses.
.01<p<.05
p<.01. Models also include the other covariates shown in Table 4.
HMO, health maintenance organization; PCCM, primary care case management.
For each of the five satisfaction variables, we conducted two analyses. First, we used logistic regression in which the dependent variable was a dichotomous indicator measuring either “very satisfied” and “somewhat satisfied” or “excellent” and “very good,” depending on the question. In addition, each question is also examined using an ordered probit model, with a dependent variable running from 1, the most favorable, to 5, the least favorable. SUDAAN was used to compute standard errors for the logistic regression models. For the ordered probit models, Stata's survey adjustment routines were used as SUDAAN routines were not available. These also account for the complex survey design, although less completely than SUDAAN. In principle, this could lead to an overestimate or underestimate of the standard errors, although when results from SUDAAN and Stata's survey routines for the logistic satisfaction models were computed, the SUDAAN standard errors were always lower than the Stata standard errors, consistent with a low potential for biased inference.
Table 7 reports the results. There is no evidence for effects on overall satisfaction with care or on satisfaction with doctor choice. Increases in HMO market share are associated with declines in the rate of favorable perceptions about thoroughness, the doctor's listening, and doctor's explaining at the last visit. Increases in PCCM share are not associated with changes in satisfaction.
Table 7.
Results from analysis of satisfaction measures
| Overall Satisfaction | Doctor Choice | Thoroughness of Last Visit | Doctor Listening at Last Visit | Doctor Explaining at Last Visit | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Very/Somewhat Satisfied (Logit) | Ordered Probit | Very/Somewhat Satisfied (Logit) | Ordered Probit | Excellent/Very Good (Logit) | Ordered Probit | Excellent/Very Good (Logit) | Ordered Probit | Excellent/Very Good (Logit) | Ordered Probit | |
| Medicaid HMO market share | 0.218 | −0.032 | −0.660 | 0.490 | −0.917 | 0.401** | −0.837 | 0.510** | −0.500 | 0.401** |
| (0.829) | (0.208) | (0.658) | (0.401) | (0.548) | (0.140) | (0.440) | (0.172) | (0.418) | (0.119) | |
| Medicaid PCCM market share | −0.539 | −0.016 | −0.041 | −0.273 | −0.322 | 0.009 | 0.193 | −0.225 | −0.029 | −0.242 |
| (0.515) | (0.218) | (0.570) | (0.287) | (0.579) | (0.178) | (0.404) | (0.184) | (0.438) | (0.197) | |
| Observations | 2,535 | 2,535 | 2,571 | 2,571 | 2,059 | 2,059 | 2,050 | 2,050 | 2,055 | 2,055 |
| Predictions, holding covariates at their sample means | ||||||||||
| HMO market share | ||||||||||
| 0.30 | 0.881 | 1.576 | 0.922 | 1.873 | 0.694 | 2.297 | 0.717 | 2.218 | 0.726 | 1.518 |
| 0.40 | 0.883 | 1.571 | 0.917 | 1.967 | 0.674 | 2.391 | 0.699 | 2.334 | 0.716 | 1.580 |
| 0.50 | 0.885 | 1.566 | 0.912 | 2.065 | 0.653 | 2.489 | 0.681 | 2.457 | 0.705 | 1.644 |
| 0.60 | 0.887 | 1.561 | 0.906 | 2.169 | 0.631 | 2.591 | 0.662 | 2.585 | 0.694 | 1.712 |
| PCCM share | ||||||||||
| 0.30 | 0.872 | 1.572 | 0.921 | 1.812 | 0.682 | 2.307 | 0.720 | 2.160 | 0.724 | 1.474 |
| 0.40 | 0.865 | 1.570 | 0.921 | 1.763 | 0.675 | 2.309 | 0.723 | 2.112 | 0.723 | 1.438 |
| 0.50 | 0.858 | 1.567 | 0.920 | 1.715 | 0.667 | 2.311 | 0.726 | 2.065 | 0.721 | 1.404 |
| 0.60 | 0.851 | 1.565 | 0.919 | 1.669 | 0.659 | 2.313 | 0.729 | 2.019 | 0.720 | 1.370 |
Note: Standard errors are in parentheses.
.01<p<.05.
p<.01. Models also include the other covariates shown in Table 4. Ordered probit models are coded 1=most favorable to 5=least favorable.
HMO, health maintenance organization; PCCM, primary care case management.
Discussion
There is some evidence that Medicaid managed care is associated with changes in health care utilization, access to care, and satisfaction. For utilization, Medicaid HMOs are associated with reductions in emergency room use, increases in outpatient visits, particularly to nonphysician providers, and reductions in hospitalizations. These kinds of changes are consistent with effects that have been attributed to HMOs in many settings. In the case of emergency department visits, these results are consistent with previous work on Medicaid managed care (Garrett, Davidoff, and Yemane 2003). Previous work has, however, not typically reported increases in outpatient visits or reductions in hospitalizations associated with Medicaid HMOs.
The relationship between Medicaid HMOs and access measures is less clear. In one case, there is evidence for worse access—rates of reporting having put off care are higher with higher Medicaid HMO market share—but in other cases, there is no significant association. Previous work has not reported reductions in the rate of putting off care associated with Medicaid managed care. Finally, in some analyses, increases in HMO market share are associated with reductions in perceived satisfaction with patients' most recent visit.
There are also relationships with PCCM market share. These are somewhat surprising, as previous work has tended not to find notable effects of PCCM programs. Specifically, there are increases in the number of visits to doctors and nonphysician providers with increases in PCCM share. At the same time, there are also higher rates of reporting having unmet medical needs, putting off care, and having no usual source of care with higher PCCM share. PCCM programs are not associated with satisfaction. Because these results vary in their direction—more visits yet higher rates of reporting access issues—their interpretation is not entirely clear. One possibility is that the reported access difficulties reflect perceived rather than real difficulties in obtaining care because of the uncertainties surrounding the shift into PCCM plans. If so, rates of difficulties could dissipate over time. At the same time, these results could suggest that further attention is needed to the effects of PCCM plans on access to care. For example, difficulties with perceived access may reflect problems dealing with PCCM gatekeepers.
On the whole, this study, in combination with previous work, does not present a clear picture of the effects of Medicaid managed care. There are some potentially positive messages. One concern is that children lacking appropriate sources of care overuse emergency departments, and results suggest increases in provider visits and reductions in ED use. (Of course, we cannot rule out that Medicaid managed care has led to underuse of EDs with these data.) At the same time, this paper and others also find some evidence for worse self-reported access to care, and reductions in satisfaction. This could reflect disruptions associated with transitions to new managed care arrangements, or structural problems with the programs.
With mixed results, it is important to push further to understand reasons for variations and try to clarify the ways in which Medicaid managed care influences health care. One natural next step would be to devote more energy to examining individual plans, where specific characteristics can be examined more closely. Nationwide studies like this provide important baseline evidence about effects of Medicaid managed care and a valuable overall view, but are correspondingly unable to identify the particular features that make for the most successful implementations.
It is not clear what the future holds for Medicaid managed care. While well entrenched in many states, Medicaid managed care programs have not always achieved the savings that had been expected, and in an era of continued pressure on costs, some states may seek to change their programs in the coming years. As they do, it is important to continue to monitor the effects on health care for the affected populations.
Acknowledgments
This project was supported by a grant from the Robert Wood Johnson Foundation's Health Care Financing and Organization Program. We thank Robert Hurley for particularly helpful comments, and Robert Osterhoff and Jia Chan for assistance with the analysis.
References
- Freund DA, Lewit EM. “Managed Care for Children and Pregnant Women: Promises and Pitfalls.”. The Future of Children. 1993;3(2):92–122. [Google Scholar]
- Garrett B, Davidoff AJ, Yemane A. “Effects of Medicaid Managed Care Programs on Health Services Access and Use.”. Health Services Research. 2003;32(2):575–94. doi: 10.1111/1475-6773.00134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Holahan J, Liska D. “The Slowdown in Medicaid Spending Growth.”. Health Affairs. 1997;16(2):157–63. doi: 10.1377/hlthaff.16.2.157. [DOI] [PubMed] [Google Scholar]
- Holahan J, Zuckerman S, Evans A, Rangarajan S. “Medicaid Managed Care in 13 States.”. Health Affairs. 1998;17(3):43–63. doi: 10.1377/hlthaff.17.3.43. [DOI] [PubMed] [Google Scholar]
- Hurley RE, Freund DA, Paul JE. Managed Care in Medicaid: Lessons for Policy and Program Design. Ann Arbor, MI: Health Administration Press; 1993. [Google Scholar]
- Kaiser Commission on the Future of Medicaid . Medicaid and Managed Care: Lessons from the Literature. Washington, DC: Kaiser Commission on Medicaid and the Uninsured; 1995. [Google Scholar]
- Kaiser Commission on Medicaid and the Uninsured . Medicaid Facts: Medicaid and Managed Care. Washington, DC: Kaiser Commission on Medicaid and the Uninsured; 2001. [Google Scholar]
- Long SK, Coughlin TA. “Impacts of Medicaid Managed Care on Children.”. Health Services Research. 2001;36(1):7–23. Part 1. [PMC free article] [PubMed] [Google Scholar]
- Miller RH, Luft HS. “Managed Care Plan Performance Since 1980. A Literature Analysis.”. Journal of the American Medical Association. 1994;271(19):1512–19. [PubMed] [Google Scholar]
- Miller RH, Luft HS. “Does Managed Care Lead to Better or Worse Quality of Care?”. Health Affairs. 1997;16(5):7–25. doi: 10.1377/hlthaff.16.5.7. [DOI] [PubMed] [Google Scholar]
- Miller RH, Luft HS. “HMO Plan Performance Update: An Analysis of the Literature 1997–2001.”. Health Affairs. 2002;21(4):63–86. doi: 10.1377/hlthaff.21.4.63. [DOI] [PubMed] [Google Scholar]
- Szilagyi PG. “Managed Care for Children: Effect on Access to Care and Utilization of Health Services.”. The Future of Children. 1998;8(2):39–59. [PubMed] [Google Scholar]
- White H. “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.”. Econometrica. 1982;50:483–99. [Google Scholar]
- Zuckerman S, Brennan N, Yemane A. “Has Medicaid Managed Care Affected Beneficiary Access and Use?”. Inquiry. 2002;39(3):221–42. doi: 10.5034/inquiryjrnl_39.3.221. [DOI] [PubMed] [Google Scholar]