Abstract
Objectives
To evaluate the following: (1) baseline knowledge, attitudes, and behavioral intentions about Medicaid managed care (MMC) among seniors and people with disabilities (SPD) receiving Medicaid benefits; (2) SPD Medicaid beneficiaries' use of and satisfaction with a user-designed MMC guidebook; and (3) guidebook effects on changes in MMC knowledge, attitudes, and intended behaviors of SPD beneficiaries.
Data Sources/Study Setting
Survey data collected between February and May 2008 from a random sample of SPD receiving Medicaid benefits in three California counties.
Study Design
This randomized controlled trial of 319 intervention and 373 control SPD Medicaid beneficiaries used pre- and postintervention telephone surveys to compare changes in MMC knowledge, attitudes, and intended behaviors.
Data Collection Methods
Baseline and follow-up telephone interviews were conducted in English, Spanish, Cantonese, and Mandarin.
Principal Findings
Seventy-seven percent of intervention participants reported using the guidebook. Nearly all (97.9 percent) found it somewhat or very useful. Intervention participants showed gains in knowledge, positive attitudes, and intentions to enroll in MMC that are statistically significant compared with control participants. However, knowledge levels remained low even among intervention participants.
Conclusions
Findings suggest that the guidebook is an effective way to improve recipients' MMC knowledge, confidence, and behavioral intentions.
Keywords: Medicaid, people with disabilities, seniors, managed care, health communication
A critical health services challenge is reaching the estimated 55.6 million Medicaid beneficiaries with empowering information about their health care options (Centers for Medicare and Medicaid Services 2007). Medicaid beneficiaries often face serious communication barriers resulting from limited literacy, limited English proficiency, cultural differences in beliefs about medical care and disease, and sometimes disability (Weiss et al. 1994; Coughlin et al. 2008;). Within this population, the subgroups most at risk include over 13.5 million seniors and people with disabilities (SPD) (Centers for Medicare and Medicaid Services 2007). Many have limited health literacy skills (difficulty accessing, understanding, and acting on health information) (Rudd, Moeykens, and Colton 2000; Kutner et al. 2006;), limited English proficiency (Weiss et al. 1994; Root and Stableford 1999; Hoy, Kenney, and Talavera 2004;), or physical/cognitive conditions that impede information access.
With the rapid proliferation of state “consumer choice models,” millions of Medicaid beneficiaries need to make complex decisions about health plan options and navigate a complicated system of care (Coughlin et al. 2008). These decisions impact access to needed clinical services (Ireys, Thornton, and McKay 2002; Long, Coughlin, and Kendall 2002; Chimento et al. 2005; Carbaugh, Elias, and Rowland 2006; Health Research for Action 2007;). However, this population may have difficulty making decisions—in part because of poor information quality (Ireys, Thornton, and McKay 2002; Long, Coughlin, and Kendall 2002; Hoy, Kenney, and Talavera 2004; Carbaugh, Elias, and Rowland 2006; Health Research for Action 2007; Coughlin et al. 2008;). In a study of Medicaid beneficiaries using printed materials about health care choices, fewer than half of the recipients reported that they understood the information (Coughlin et al. 2008). In another study limited to SPD on Medicaid, only a quarter understood the information (Health Research for Action 2007). More than 250 studies indicate that the reading levels of most health materials greatly exceed the average literacy skills of adults in the United States. Medicaid beneficiaries typically have lower health literacy skills than those of most Americans (Rudd, Moeykens, and Colton 2000; Kutner et al. 2006;).
Few studies have examined the effects of mass health communication interventions used to educate beneficiaries of federally funded health programs about health plan choices. We found none for Medicaid populations in our literature review. A study of the 1999 Medicare & You handbook found that Medicare beneficiaries who received the handbook improved knowledge and confidence about health plan choices (McCormack et al. 2001; McCormack et al. 2002;).
Theoretical guidance from social-ecological (Stokols 2000), social-cognitive (Bandura 1985), and transtheoretical (Prochaska and Velicer 1997) models, and from empirical research, suggests that health communication is more effective when it is relevant to people's personal and social contexts (Vaiana and McGlynn 2002; Institute of Medicine 2003; Neuhauser and Kreps 2003;). Evidence-based strategies to tailor printed mass communication resources to the needs of population subgroups include matching readability to users' literacy levels, using “clear communication” formatting criteria, incorporating culturally relevant concepts and graphics, and adapting rather than simply translating material into other languages (CDC 1999; Hoy, Kenney, and Talavera 2004; Calderon and Beltran 2005; Osborne 2005; U.S. Department of Health and Human Services 2008;).
In California an estimated 920,800 seniors and 935,800 people with disabilities (18 percent of total Medicaid enrollment in the state) participated in the state's Medicaid program (Medi-Cal) in 2005, accounting for 67 percent of the U.S.$28.6 billion in total Medi-Cal spending (Henry J. Kaiser Family Foundation 2005). Like nearly every other state, California offers Medicaid managed care (MMC) to SPD on Medicaid. Unlike in some other states, however, MMC is not mandatory for SPD in California (Cohen 2009). SPD are defaulted to fee-for-service (FFS) Medi-Cal unless they actively choose MMC. The California Department of Health Care Services (DHCS) was concerned that most SPD beneficiaries were not actively choosing between MMC and FFS. DHCS reasoned that MMC may be a better and more cost-effective choice than FFS for these vulnerable beneficiaries because MMC plans are required to provide important services like case management, disability accommodations, linguistic translation, and paratransit services. As our earlier study showed that only a small proportion of SPD beneficiaries understood their choices, DHCS decided an educational intervention was needed for this population (Health Research for Action 2007).
In 2006, DHCS commissioned Health Research for Action to use participatory processes to develop a low-literacy guidebook to inform SPD Medi-Cal beneficiaries about choices between FFS and MMC. In the first phase of the Medi-Cal Access Project (MAP), we conducted extensive formative research with beneficiaries in five languages. The participatory processes included usability testing, key informant interviews, readability testing, focus groups, and linguistic and cultural adaptation. The final full-color, 38-page guidebook, What Are My Medi-Cal Choices? (see Appendix SA2 for an excerpt), was developed in English, Spanish, and Chinese at a sixth to seventh grade reading level with large font, abundant white space, vignettes with photos of real people, and included county-specific inserts about health plan choices. The results of the formative research and the participatory processes used to develop the guidebook are presented elsewhere (Neuhauser et al. 2009).
In the second phase of the MAP study, a telephone survey was designed to evaluate the guidebook's effectiveness for English-, Spanish-, Mandarin-, and Cantonese-speaking SPD Medi-Cal beneficiaries in three pilot counties. Objectives included the following: (1) assessing SPD beneficiaries' baseline knowledge, attitudes, and behavioral intentions about Medi-Cal managed care (MMC); (2) assessing their use of and satisfaction with the guidebook; and (3) evaluating the guidebook's effects on MMC knowledge, attitudes, and intended behaviors. DHCS was especially interested in knowing whether beneficiaries would look over the guidebook soon after receiving it, find it easy to use, consider it a valuable resource, and express confidence in making Medi-Cal decisions. To our knowledge, this is the first study of a communication intervention about health plans for Medicaid beneficiaries who are seniors or people with disabilities.
RESEARCH DESIGN AND METHODS
We designed a randomized controlled trial using a pre- and postintervention telephone survey to assess the efficacy of the guidebook and gathered data from intervention and control groups in three counties. The participants randomized to the intervention group were sent the MAP guidebook between the baseline interview and the follow-up interview. The participants randomized to the control group received no guidebook between baseline and follow-up interviews.
The UC Berkeley institutional review board approved all study protocols and instruments.
Sampling Design and Participant Recruitment
DHCS provided a list of the telephone numbers of SPD Medi-Cal beneficiaries from the three counties who were not dually eligible for Medicare, had language codes of English, Spanish, Cantonese, Mandarin, or “not valid,” and were age 18 or older (n=69,176). We eliminated telephone numbers that were invalid, out of area, or assigned to multiple beneficiaries from the list. We then ascertained a random sample from that list. The sample was stratified by language group, with nested proportional stratification by county, city, and area code. Each potential participant was then randomly assigned to intervention or control.
Screening
Of the 2,740 potential participants, we identified 1,486 as ineligible during the screening process, for example, those dually eligible for Medicare or who had a primary language other than English, Spanish, Cantonese, or Mandarin. Of the remaining 1,254 beneficiaries, a screening interview identified 959 as eligible for the study, of whom 865 completed the baseline interview, and 295 beneficiaries were of unknown eligibility (e.g., reached an answering machine at multiple contact attempts or refused the screening interview). Based on the ratio of known eligible to known ineligible cases in the sample, we estimated that 162 of these were eligible. The total of eligible plus estimated eligible cases came to 1,121. Following the guidelines of the American Association for Public Opinion Research (2008), we calculated a baseline response rate of 77 percent.
Interviewers received special training to address potential communication, cognitive, and stamina barriers with people with disabilities (Mitchell et al. 2006). At several points during screening, potential participants could opt to designate as a “proxy” for the interview someone who had assisted the Medi-Cal beneficiary with health care decisions and was not a professional care provider. If a proxy was interviewed at baseline, the same proxy was interviewed at follow-up. We modified a protocol developed for a large study of people with disabilities to screen for cognitive impairments that prevent participants from giving true informed consent (Mitchell et al. 2006). After reading the consent aloud to the potential recruit, interviewers asked the person to paraphrase one point from the consent, the meaning of “voluntary.” Medi-Cal beneficiaries unable to give an adequate response (n=6) were asked to provide a proxy in order to continue the interview; because none were able to provide a proxy, all six were excluded.
Approximately equal numbers of participants were recruited for the baseline interviews in English, Spanish, and Chinese subsamples. We combined the Cantonese (n=147) and Mandarin (n=85) speakers into a single “Chinese language” group to increase statistical power. Intervention group participants were mailed a guidebook in the language of baseline interview unless they requested one in a different language (n=25). Control participants did not receive a guidebook. About 6 weeks after baseline interviews and guidebook mailing, we reinterviewed intervention and control participants.
Instruments and Protocols
Survey Instrument Development
We selected the following key survey domains from themes that emerged from our literature review and formative research (Neuhauser et al. 2009): knowledge about Medi-Cal choices; attitudes, behavioral intentions, and confidence related to Medi-Cal choices; and guidebook use and perceived helpfulness. We included sociodemographic indices (for interviewee, whether beneficiary or proxy), and beneficiary self-rated (or proxy-rated) health status. DHCS provided data about each Medi-Cal beneficiary's county, type of Medi-Cal, age, and sex.
Where possible, we adapted previously validated items to include in this survey. Questions related to guidebook use were adapted from our previous guidebook evaluations (Neuhauser et al. 2007). We used Lorig's self-efficacy questions (Stanford Patient Education Research Center) as a model to create two new self-efficacy questions (referred to as “confidence”) related to making Medi-Cal choices. We also developed questions to measure participants' attitudes about MMC and their intentions to switch between FFS and MMC to address the state's interest in these domains.
Although the guidebook was designed as a resource book rather than a book of facts to be memorized, we wanted to explore whether participants would exhibit initial knowledge changes after receiving the guidebook. Therefore, the baseline survey included 14 items about participants' knowledge of FFS Medi-Cal and MMC. For example, participants were asked to name at least two Medi-Cal plans in their county; which type of Medi-Cal (FFS or MMC) must provide a provider directory; and to state the maximum number of prescriptions covered without authorization under FFS versus MMC (see Appendix SA3 for a complete list of items). These items were combined into a “core knowledge scale.” To assess scale reliability, we calculated Cronbach coefficient α, which measures the consistency of respondents' answers across the 14 items. The coefficient α was 0.56 (Schmitt 1996).
The follow-up survey repeated all questions in the baseline survey except sociodemographic and health status items. In addition, we asked only the intervention group questions about use of and satisfaction with the guidebook.
Survey instruments were translated into Spanish, Cantonese, and Mandarin. The English and Spanish telephone survey instruments were programmed into computer-assisted telephone interview (CATI) software. Chinese surveys were conducted using a paper Chinese survey and an English CATI screen for simultaneous data entry.
Language Protocols
Interviewers initiated phone calls in the participant's language indicated in DHCS records, and switched into English as needed. Chinese-speaking interviewers spoke both Cantonese and Mandarin. The interview was terminated if the prospective interviewee spoke a language other than English, Spanish, Cantonese, or Mandarin.
Fatigue and Encouragement Probes
Interviewers were trained to detect signs of fatigue, and instructed to use a structured “fatigue probe” or “encouragement probe” or offer participants an opportunity to take a break or continue the interview at a later time. We adapted fatigue and encouragement probes from Mitchell et al. (2006) and translated into Spanish, Cantonese, and Mandarin.
Data Analysis
We used SAS software (version 9.1, SAS, Cary, NC) for quantitative analyses. We calculated frequencies for categorical and ordinal variables. We used cross-tabulations to examine and compare categorical variables by group (i.e., intervention versus control assignment) and by interview language. The χ2-test was used to evaluate associations (i.e., group differences) between categorical variables. Student's t and analysis of variance (ANOVA) were used to evaluate intergroup differences between means that were calculated for interval variables. The Mann–Whitney U test or the Kruskal–Wallis test was used to evaluate group differences for ordinal variables. The Wilcoxon signed ranks test was used to test the statistical significance of differences in core knowledge scores between baseline and follow-up. We ran repeated measure ANOVAs with post hoc testing and Bonferroni adjustment for multiple testing to evaluate differences among all language groups and between pairs of language groups in the effect of receipt of the guidebook on change in knowledge scale score between T1 and T2.
Description of Participants
Of the 865 participants interviewed at baseline, 416 were randomized to the intervention group and 449 to the control group. We mailed intervention group participants a guidebook, and 6 weeks later we reinterviewed 319 of the initial intervention participants (77 percent retention) and 373 of the initial control participants (83.1 percent retention). Overall, 692 of the 865 baseline participants were reinterviewed (80 percent retention).
Entire Sample
Demographic characteristics of the sample appear in Table 1. Forty-four percent of participants were proxies for a Medi-Cal recipient, rather than the beneficiary. Most Medi-Cal beneficiaries (including those represented by proxies) were under 65 years of age (61 percent), female (62 percent), had FFS Medi-Cal (75 percent), and had generally fair or poor health (self- or proxy-reported) (72.1 percent). Over half of participants (beneficiary or proxy) were married (54.6 percent).
Table 1.
Demographics by Group (Control versus Intervention Assignment)
| Group |
||||
|---|---|---|---|---|
| Control % (n) | Intervention % (n) | All % (n) | Statistical Significance p-Value | |
| Interview participant category | ||||
| Medi-Cal beneficiary | 56.6 (211) | 55.2 (176) | 55.9 (387) | |
| Proxy | 43.4 (162) | 44.8 (143) | 44.1 (305) | .7124 |
| Medi-Cal beneficiary | ||||
| Age above or below 65 | ||||
| 64 and under | 58.5 (218) | 63.9 (204) | 61.0 (422) | |
| 65 and above | 41.6 (155) | 36.1 (115) | 39.0 (270) | .1393 |
| Sex | ||||
| Female | 64.6 (241) | 58.9 (188) | 62.0 (429) | |
| Male | 35.4 (132) | 41.1 (131) | 38.0 (263) | .1251 |
| Type of Medi-Cal (per Department of Health Care Services [DHCS]) | ||||
| Fee-for-service (“regular Medi-Cal”) | 72.1 (269) | 78.4 (250) | 75.0 (519) | |
| Medi-Cal managed care | 27.9 (104) | 21.6 (69) | 25.0 (173) | .0583 |
| General health status | ||||
| Excellent | 3.0 (11) | 4.1 (13) | 3.5 (24) | |
| Very good | 7.8 (29) | 6.9 (22) | 7.4 (51) | |
| Good | 18.3 (68) | 15.5 (49) | 17.0 (117) | |
| Fair | 46.0 (171) | 42.0 (133) | 44.1 (304) | |
| Poor | 25.0 (93) | 31.6 (100) | 28.0 (193) | .1441 |
| Interview participant (either Medi-Cal beneficiary or proxy) | ||||
| Marital status | ||||
| Married | 54.2 (202) | 55.1 (174) | 54.6 (376) | |
| Living with partner | 5.9 (22) | 4.8 (15) | 5.4 (37) | |
| Widowed | 6.7 (25) | 7.6 (24) | 7.1 (49) | |
| Divorced | 14.2 (53) | 10.1 (32) | 12.3 (85) | |
| Separated | 4.8 (18) | 7.9 (25) | 6.2 (43) | |
| Never married | 14.2 (53) | 14.6 (46) | 14.4 (99) | .3498 |
| Race/ethnicity | ||||
| White | 14.2 (53) | 14.6 (46) | 14.4 (99) | |
| Black/African American | 9.1 (34) | 8.9 (28) | 9.0 (62) | |
| Asian | 35.1 (131) | 34.8 (110) | 35.0 (241) | |
| American Indian/ Alaska Native | 0.5 (2) | 0 (0) | 0.3 (2) | |
| Other Pacific Islander | 0.3 (1) | 0 (0) | 0.2 (1) | |
| Two or more races | 1.9 (7) | 3.2 (10) | 2.5 (17) | |
| Latino | 38.9 (145) | 38.6 (122) | 38.8 (267) | .7137 |
| Education | ||||
| Completed eighth grade or less (middle school or less) | 25.8 (96) | 26.9 (85) | 26.3 (181) | |
| Grades 9 through 11 (some high school) | 21.0 (78) | 18.4 (58) | 19.8 (136) | |
| Grade 12 or GED (high school graduate) | 23.1 (86) | 23.7 (75) | 23.4 (161) | |
| 1 or more years of college | 30.1 (112) | 31.0 (98) | 30.5 (210) | .8807 |
| Interview language | ||||
| English | 31.4 (117) | 35.1 (112) | 33.1 (229) | |
| Spanish | 34.6 (129) | 32.0 (102) | 33.4 (231) | |
| Chinese (Mandarin/ Cantonese) | 34.0 (127) | 32.9 (105) | 33.5 (232) | .5643 |
Intervention versus Control Groups
Overall, intervention and control group participants had similar sociodemographic characteristics. No sociodemographic differences between the intervention and control groups reached statistical significance (Table 1).
By Language Group
Sociodemographic characteristics differ by interview language. English speakers were younger (p<.0001), meaning they were more likely to receive Medi-Cal due to disability, rather than age. Self-reported (or proxy-reported) health status was more likely to be poor among the English speakers, compared with the Spanish or Chinese speakers (p<.0001). Spanish speakers were more likely than English or Chinese speakers to be female (p=.002). Chinese speakers were more likely than those in other language groups to be seniors (p<.0001), be in good, very good, or excellent health (p<.0001), and have FFS Medi-Cal (p=.03). English-speaking beneficiaries and proxies were more educated (p<.0001) and less likely to be married (p<.0001) than Spanish and Chinese speakers.
Proxy versus Medi-Cal Beneficiary
Proxies differed from Medi-Cal beneficiaries on several important characteristics. Medi-Cal beneficiaries who represented themselves were less likely to be seniors; to have FFS Medi-Cal; to report good, very good, or excellent general health; to be married; to be Asian or Latino; but more likely to be interviewed in English (p<.05 for all).
RESULTS
Guidebook Use
Entire Sample
Seventy-seven percent of the participants receiving a guidebook said at the follow-up interview they had “read, referred to, or looked at” it. Overall, the majority of participants who received and reported looking at the guidebook thought it was somewhat easy (34.6 percent) or very easy (48.1 percent) to understand (Table 2). About two-thirds (66.5 percent) said they had learned something new about their Medi-Cal choices from the guidebook. Almost all found the guidebook information either somewhat useful (40.2 percent) or very useful (57.7 percent). Those who found the guidebook useful were asked to name one thing they found useful. The most common responses were information about choosing between FFS and MMC, Medi-Cal benefits, physicians, enrolling in MMC, and prescriptions, and resource contact information. More than one-third (35.5 percent) of those who read the guidebook reported that they referred to it to find answers to questions. Of those who had looked up a question, 83.5 percent said they were able to find an answer.
Table 2.
Guidebook Use Questions by Language
| Interview Language |
|||||
|---|---|---|---|---|---|
| Percent of Participants Who Received and Looked at the Guidebook Who | English % (95% CI) (n=87) | Spanish % (95% CI) (n=60) | Chinese % (95% CI) (n=90) | All Languages (n=237) | Statistical Significance p-Value |
| Thought the guidebook was easy or somewhat easy to understand* | 87.4 (78.5, 93.5) | 76.7 (64.0, 86.6) | 82.2 (72.7, 89.5) | 82.7 (77.9, 87.5) | .2394 |
| Referred to the guidebook to find answers to questions† | 35.3 (25.2, 46.4) | 50.0 (36.8, 63.2) | 25.8 (17.1, 36.2) | 35.5 (29.3, 41.6) | .0104 |
| Learned something new about their Medi-Cal choices from the guide† | 51.7 (40.8, 62.6) | 78.2 (65.0, 88.2) | 73.9 (63.4, 82.7) | 66.5 (60.4, 72.6) | .0009 |
| Found the information in the guidebook very useful or somewhat useful* | 94.2 (87.0, 98.1) | 100 (94.0, 100.0) | 100 (95.9, 100.0) | 97.9 (96.0, 99.7) | .0123 |
| Had questions about Medi-Cal that were not answered by the guide† | 24.7 (16.0, 35.3) | 12.7 (5.3, 24.5) | 16.7 (9.4, 26.4) | 18.8 (13.6, 23.9) | .1714 |
Kruskal–Wallis test.
χ2-test.
By Language Group
Of participants who received a guidebook, there were statistically significant differences among language groups in the percentage who said that they “read, referred to, or just looked at” any part of it: English, 80.6 percent; Spanish, 61.9 percent; and Chinese, 86.5 percent (p<.0001). Table 2 summarizes responses to guidebook use questions by language.
Proxy versus Medi-Cal Beneficiary
None of the differences in guidebook use between Medi-Cal beneficiaries who were interviewed themselves and proxies reached statistical significance.
Knowledge about Medi-Cal Choices
Entire Sample
The percentage of participants who could correctly answer knowledge questions regarding FFS Medi-Cal and MMC at baseline was low. The mean knowledge scale score for the entire sample at baseline was 20.3 out of a possible 100, indicating participants on average correctly answered only 20.3 percent of the knowledge questions.
Intervention versus Control Group
Baseline mean knowledge scale scores for the intervention group (20.3) and control group (20.4) were virtually the same at baseline (Table 3). Both groups showed significant knowledge gains between the baseline and follow-up interviews (p<.0001), but the intervention group gained significantly more knowledge (increase of 12.5) than did the controls (increase of 5.9) (p<.0001).
Table 3.
Mean Knowledge Scale Scores at T1 and T2 by Language and Group (Control versus Intervention Assignment)
| Knowledge Scale Score (Mean) |
|||||
|---|---|---|---|---|---|
| Interview Language | Group | T1 | T2 | Mean Change (T1–T2) | Statistical Significance between T1 and T2 Score p-Value* |
| English | Control | 26.1 | 27.0 | 0.9 | .3861 |
| Intervention | 25.4 | 33.9 | 8.5 | <.0001 | |
| Spanish | Control | 18.5 | 21.9 | 3.3 | .0032 |
| Intervention | 17.8 | 25.4 | 7.6 | <.0001 | |
| Chinese | Control | 17.0 | 30.1 | 13.1 | <.0001 |
| Intervention | 17.2 | 39.0 | 21.8 | <.0001 | |
| All languages | Control | 20.4 | 26.3 | 5.9 | <.0001 |
| Intervention | 20.3 | 32.8 | 12.6 | <.0001 | |
Wilcoxon signed rank test.
By Language Group
The mean knowledge scale scores at baseline by language group were as follows: English 25.8, Spanish 18.2, and Chinese 17.1 (Table 3). These differences among the mean baseline knowledge scale scores were statistically significant (p<.0001). Among the linguistic groups, English speakers had the highest mean knowledge scale score at baseline.
At follow-up, all groups—except for the English control group—showed statistically significant knowledge increases from baseline (Table 3). English (p=.002) and Chinese (p=.0005) intervention participants demonstrated higher mean knowledge scores than control participants at follow-up. Intergroup differences were not statistically significant for Spanish-speaking participants at follow-up (p=.10, NS). We tested for significant differences in the effect of receipt of the guidebook on change in knowledge scale score between T1 and T2 by language. No significant differences were found among all language groups or between pairs of language groups before or after we conducted the Bonferroni adjustment of target p-values for multiple testing.
Proxy versus Medi-Cal Beneficiary
Proxies demonstrated a higher mean core knowledge score at baseline (T1) (p=.004) at 22.1 (versus 19.0 among beneficiaries). There was no statistically significant difference in mean increase in knowledge scale score between Medi-Cal beneficiaries and proxies in the intervention group (p=.36, NS).
Attitudes and Confidence Regarding Medi-Cal Managed Care
Entire Sample
At baseline, the majority of participants (56.4 percent) had positive impressions about Medi-Cal managed care. However, only 16.1 percent of participants who had FFS Medi-Cal reported at baseline that they were very or somewhat likely to switch to a MMC plan within 6 months. The majority of participants at baseline (79.5 percent) reported being very or somewhat confident that they understood how to get the health care they needed. However, fewer participants at baseline reported being very or somewhat confident that they understood the different MMC plans (58.2 percent).
Intervention versus Control Group
At baseline, the percentage of participants in the control and intervention group who had positive impressions of MMC was the same (56 percent). At follow-up, 61.6 percent of the intervention participants reported a positive impression compared with 56.1 percent among controls, although this difference did not reach statistical significance. In a separate question asked only of intervention participants at T2, 53.6 percent (95 percent CI, 48.0–59.2) reported having a more positive impression of MMC than before they received the guidebook.
At baseline, none of the differences between control and intervention group on either measure of confidence—confidence in understanding how to get needed health care and confidence in understanding the different MMC plans—reached statistical significance (Table 4). However, at follow-up, the percentage of intervention participants who were somewhat or very confident that they understood the different MMC plans was higher (79.9 percent) than the percentage in the control group (62.3 percent) (p<.0001). The intervention group also showed statistically significant changes between baseline and follow-up in confidence in getting health care (80.6–86.2 percent very or somewhat confident, p=.01) and confidence in understanding different MMC plans (58.0–79.9 percent somewhat or very confident, p<.0001). Increases seen in the control group between baseline and follow-up for these two questions were not statistically significant.
Table 4.
Confidence Regarding Health Care and Intention to Change Plans at T2
| English |
Spanish |
Chinese |
All Languages |
|||||
|---|---|---|---|---|---|---|---|---|
| Control (%) | Intervention (%) | Control (%) | Intervention (%) | Control (%) | Intervention (%) | Control (%) | Intervention (%) | |
| How confident in getting the health care you need?—Very confident or somewhat confident | ||||||||
| T1 (n=620) | 85.5 | 86.6 | 79 | 83.7 | 70.3 | 67.6 | 78.7 | 80.6 |
| T2 (n=671) | 82.9 | 85.7 | 88.7 | 87.0 | 78.0 | 86.0 | 83.3 | 86.2 |
| Statistical significance between T1 and T2* | 0.4913 | 0.8185 | 0.0343 | 0.2482 | 0.2230 | 0.0011 | 0.0915 | 0.0126 |
| How confident are you that you understand the different Medi-Cal Managed Care plans?—Very confident or somewhat confident | ||||||||
| T1 (n=608) | 62.6 | 56.9 | 61.8 | 67.1 | 49.0 | 49.4 | 58.4 | 58.0 |
| T2 (n=656) | 58.6 | 78.9 | 70.3 | 84.4 | 57.8 | 76.5 | 62.3 | 79.9 |
| Statistical significance between T1 and T2* | 0.5271 | <0.0001 | 0.0771 | 0.0006 | 0.1282 | 0.0004 | 0.147 | <0.0001 |
| How likely are you to switch to a Medi-Cal Managed Care plan in the next 6 months? (asked only of people on fee-for-service Medi-Cal)—Very likely or somewhat likely | ||||||||
| T1 (n=477) | 16.2 | 8.6 | 22.5 | 18.4 | 11.5 | 18.1 | 17.0 | 15.1 |
| T2 (n=507) | 14.7 | 16.9 | 23.6 | 32.9 | 17.5 | 28.1 | 19.2 | 26.7 |
| Statistical significance between T1 and T2* | 0.6547 | 0.1573 | 0.6374 | 0.0522 | 0.3173 | 0.0833 | 0.2888 | 0.0039 |
| How likely are you to switch back to regular (fee-for-service) Medi-Cal in the next 6 months? (asked only of people on a Medi-Cal Managed Care plan)—Very likely or somewhat likely | ||||||||
| T1 (n=68) | 15.8 | 6.7 | 30.0 | 40.0 | 11.1 | 0 | 18.4 | 16.7 |
| T2 (n=98) | 10.3 | 0 | 9.1 | 22.2 | 5.3 | 0 | 8.5 | 5.1 |
| Statistical significance between T1 and T2* | 0.0005 | † | 0.3173 | 0.1573 | 0.0143 | † | <0.0001 | <0.0001 |
McNemar test.
Unable to calculate test statistic due to uniformity in one of the measures.
Intervention and control group participants who were on FFS Medi-Cal did not differ at baseline on their likelihood of switching to a MMC plan in the next 6 months. However, at the follow-up interview, intervention participants more often reported being somewhat or very likely to enroll in a MMC plan than did controls (26.7 versus 19.2 percent, p=.04) (Table 4). The intervention group also showed a statistically significant change between baseline and follow-up in the likelihood of switching to a MMC plan in the next 6 months (increasing from 15.1 to 26.7 percent somewhat or very likely, p=.004). The control group did not show an increase in likelihood of changing to MMC.
By Language Group
There were differences in characteristics of and change among language groups. At baseline, Spanish and Chinese speakers were more likely to report positive impressions of MMC than were English speakers (p<.0001). At follow-up, Spanish and Chinese speakers continued to be more likely to report positive impressions of MMC compared with English speakers (p<.0001). At follow-up, Chinese speakers in the intervention group were more likely to report positive impressions of MMC than their counterparts in the control group (p=.03).
At baseline, Chinese speakers were more likely than English or Spanish speakers to report they were not at all confident of their ability to obtain health care (31 percent) (p=.0001) and to understand the different types of Medi-Cal plans (51 percent) (p=.01). Spanish control (p=.03) and Chinese intervention (p=.001) participants showed increased confidence in obtaining health care between T1 and T2. The increase in confidence in understanding of the MMC plans between T1 and T2 among intervention participants was found for all three language groups (English p<.0001, Spanish p=.0006, Chinese p=.0004). In addition, intervention participants in all languages showed greater confidence at T2 than their control counterparts (English p=.001, Spanish p=.01, Chinese p=.004).
While no statistically significant differences were found at baseline, at follow-up more Spanish (27.7 percent) and Chinese (22.6 percent) speakers with FFS Medi-Cal reported they were likely to switch to MMC in the next 6 months compared with English speakers (15.8 percent) (p=.04).
Proxy versus Medi-Cal Beneficiary
At T2 Medi-Cal beneficiaries in the intervention group had significantly increased confidence in their ability to get the health care they need (p=.04) compared with baseline. No increase in confidence from T1 to T2 was found among Medi-Cal beneficiaries in the control group or among proxies in either group. Proxies in both control and intervention groups showed an increased intention to change from FFS to MMC between T1 and T2 (p=.046 and .02, respectively), with no statistically significant difference between control and intervention groups at T2 (p=.61, NS).
DISCUSSION
Medicaid beneficiaries who are seniors or have a disability (SPD) are among the most vulnerable populations in the United States. Subgroups with limited literacy or limited English proficiency face additional health risks. Although these beneficiaries are increasingly required to make choices about their health plans, available evidence indicates that only a small percentage of them understand their options—a problem with potentially serious health consequences. Despite the critical importance of improved communication for such Medicaid beneficiaries, we found no studies of the effectiveness of communication strategies for this population. Our study was intended to address this knowledge gap by providing evidence about the initial efficacy of user-designed, multilingual mass communication resources for SPD Medicaid beneficiaries in California.
We evaluated this guidebook in a rigorous randomized controlled trial with linguistically diverse beneficiary subgroups. The primary study questions were whether beneficiaries or their proxy decision makers would use a resource about Medi-Cal coverage choices, and whether that would help them improve knowledge and confidence and would affect their decisions about coverage. The evaluation was intended to inform state policy decisions related to supporting vulnerable Medi-Cal beneficiaries. The challenge of recruiting, interviewing, and retaining adequate samples of these beneficiaries benefited from theoretical guidance and prior empirical evidence about surveying vulnerable populations. We were successful in reaching these hard-to-interview groups.
The baseline survey showed low levels of knowledge about Medi-Cal coverage choices among all language groups. Knowledge scores for Spanish and Chinese speakers were lowest. Results were highly positive for guidebook use and satisfaction: 77 percent of the intervention participants reported using it within the first 6 weeks. Users were also very satisfied with the resource; 97.9 percent found the information somewhat or very useful. The majority thought it was somewhat or very easy to understand (82.7 percent). Most reported that they had learned something new about their Medi-Cal choices (66.5 percent). The rates of use and satisfaction with the guidebook are considered high for a mass communication intervention without mediated support from a provider. We believe the positive results reflect the year-long participatory process to collaborate with SPD Medi-Cal beneficiaries and stakeholders in designing and testing the guidebook (Neuhauser et al. 2009).
Although the follow-up telephone survey was conducted only 6 weeks after the intervention group received their guidebook, the confidence of the intervention group participants in understanding MMC plans increased from 58 to 80 percent (p<.0001), while there was no contemporaneous change in the control participants' confidence. Participants' increased confidence in obtaining information and high satisfaction with the guidebook are encouraging indicators of the resource's value. In our view, the increased confidence participants reported after using the guidebook may indicate an increased ability to make informed decisions with the guidebook as a resource.
Knowledge gains were not considered the most critical outcome because the guidebook is intended as a decision-making resource rather than information to be memorized. However, results showed that the guidebook did spark early learning. Intervention group participants showed statistically significant improved knowledge about Medi-Cal choices compared with the control group participants who did not get the guidebook. Among the language subgroups, significant differences in knowledge at follow-up were found between intervention and control groups for English and Chinese speakers. While there was an increase in knowledge among the Spanish-speaking intervention group, the difference from the pattern seen in the control group is not statistically significant due to the surprising increase in knowledge of this control group. The reason for the increase in knowledge among this group may suggest a Hawthorne effect (Parsons 1974) and warrants further investigation.
Over half of intervention group participants at follow-up reported having a more positive impression of MMC than they did before getting the guidebook. This is important because negative impressions or stigma toward MMC may be a barrier to making an informed choice. Likewise, those who received the guidebook reported more confidence about making Medi-Cal and other health care choices than did those in the control group.
Although the baseline-to-follow-up time period was too short to measure whether participants actually made changes in their Medi-Cal health care choice, results showed that participants on FFS Medi-Cal who got a guidebook reported a significantly greater intention to switch to a MMC plan (27 versus 19 percent) than did those on FFS Medi-Cal in the control group.
The study had several limitations. Because the survey was conducted in three counties, the results cannot necessarily be generalized statewide, although the selected counties represent good geographic diversity and include the main MMC models that allow California SPD beneficiaries to choose between FFS and MMC. The knowledge scale showed good discriminant validity, but it needs more refinement to improve the internal reliability across the items. Despite the limitations, the study's randomized controlled design provides good evidence that the guidebook intervention was related to the outcomes measured (internal validity), and the intervention was relevant across diverse language groups (external validity).
This first study of a communication intervention for English-, Spanish- and Chinese-speaking Medicaid beneficiaries who are seniors or people with disabilities found low baseline knowledge, low confidence in understanding plan choices, and negative impressions of health plan choices. The initial findings suggest that the guidebook is an effective way to improve diverse recipients' knowledge, confidence, and intentions about making more informed Medi-Cal choices. Further research will explore differences by language groups and track participants' longer-term Medi-Cal coverage choices.
Acknowledgments
Joint Acknowledgment/Disclosure Statement: This report is based on research conducted by Health Research for Action, University of California, Berkeley, under contract to the Medi-Cal Managed Care Division of the California Department of Health Care Services (DHCS) (contract no. 05-46254 A05). The findings and conclusions in this document are those of the authors, who are responsible for its content, and do not necessarily represent the views of DHCS. No statement in this report should be construed as an official position of DHCS.
The authors thank Quantum Market Research for conducting the interviews, Beccah Rothschild for codirecting the overall Medi-Cal Access Project, Gerald Sumner for conducting sampling and response rate calculation, and Joan Lichterman for editorial support.
Disclosures: None.
Disclaimers: None.
Supporting Information
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Appendix SA2: What Are My Medi-Cal Choices?
Appendix SA3: Knowledge Scale Items.
Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.
REFERENCES
- American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. Lenexa, KS: American Association for Public Opinion Research; 2008. [Google Scholar]
- Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice-Hall; 1985. [Google Scholar]
- Calderon JL, Beltran RA. Culture and Linguistics: Neglected Variables in the Health Communication Equation. American Journal of Medical Quality. 2005;20(4):179–81. doi: 10.1177/1062860605276033. [DOI] [PubMed] [Google Scholar]
- Carbaugh AL, Elias R, Rowland D. Aid to People with Disabilities: Medicaid's Growing Role. Annual Review of Public Health. 2006;27:417–42. doi: 10.1146/annurev.publhealth.27.021405.102145. [DOI] [PubMed] [Google Scholar]
- CDC. Scientific and Technical Information “Simply Put.”. Atlanta, GA: Office of Communication, Centers for Disease Control and Prevention; 1999. [Google Scholar]
- Centers for Medicare and Medicaid Services. Brief Summaries of Medicare and Medicaid as of November 1, 2007. Baltimore, MD: Centers for Medicare and Medicaid Services; 2007. [Google Scholar]
- Chimento L, Forbes M, Sander A, Kailes J, Premo B, Richards C, Perrone C. 2005. Medical Beneficiaries with Disabilities: Comparing Managed Care with Fee-for-Service Systems. Issue Brief. Oakland, CA: California Health Care Foundation.
- Cohen RK. 2009. “Medicaid Managed Care for the Aged, Blind and Disabled.” [accessed on July 6, 2009]. Connecticut General Assembly Office of Legislative Research. Available at http://www.cga.ct.gov/2009/rpt/2009-R-0017.htm.
- Coughlin T, Lon S, Triplett T, Artiga S, Lyons B, Duncan R, Hall A. Florida's Medicaid Reform: Informed Consumer Choice? Health Affairs (Millwood) 2008;27(6):w523–32. doi: 10.1377/hlthaff.27.6.w523. [DOI] [PubMed] [Google Scholar]
- Health Research for Action. Year One Report of the Medi-Cal Access Project. Berkeley: University of California; 2007. [Google Scholar]
- Henry J. Kaiser Family Foundation. State Medicaid Fact Sheet. Mento Park, CA: The Kaiser Commission on Medicaid and the Uninsured; 2005. [Google Scholar]
- Hoy EW, Kenney E, Talavera AC. Engaging Consumers in Designing a Guide to Medi-Cal Managed Care Quality. Oakland, CA: California Healthcare Foundation; 2004. [Google Scholar]
- Institute of Medicine. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, DC: Institute of Medicine, National Academy Press; 2003. [Google Scholar]
- Ireys HT, Thornton C, McKay H. Medicaid Managed Care and Working-Age Beneficiaries with Disabilities and Chronic Illnesses. Health Care Financing Review. 2002;24(1):27–42. [PMC free article] [PubMed] [Google Scholar]
- Kutner M, Greenberg E, Jin Y, Paulsen C. The Health Literacy of America's Adults: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: U.S. Department of Education, National Center for Education; 2006. [Google Scholar]
- Long SK, Coughlin TA, Kendall SJ. Access to Care among Disabled Adults on Medicaid. Health Care Financing Review. 2002;23(4):159–73. [PMC free article] [PubMed] [Google Scholar]
- McCormack LA, Garfinkel SA, Hibbard JH, Keller SD, Kilpatrick KE, Kosiak B. Health Insurance Knowledge among Medicare Beneficiaries. Health Services Research. 2002;37(1):43–63. [PubMed] [Google Scholar]
- McCormack LA, Garfinkel SA, Hibbard JH, Kilpatrick KE, Kalsbeek WD. Beneficiary Survey-Based Feedback on New Medicare Informational Materials. Health Care Financing Review. 2001;23(1):37–46. [PMC free article] [PubMed] [Google Scholar]
- Mitchell S, Ciemnecki A, CyBulski K, Markesich J. Removing Barriers to Survey Participation for Persons with Disabilities. Report to the Rehabilitation Research and Training Center on Disability Demographics and Statistics. Princeton, NJ: Mathematica Policy Research; 2006. [Google Scholar]
- Neuhauser L, Constantine W, Constantine N, Sokal-Gutierrez K, Obarski S, Clayton L, Desai M, Sumner G, Syme S. Promoting Prenatal and Early Childhood Health: Evaluation of a Statewide Materials-Based Intervention for Parents. American Journal of Public Health. 2007;97(10):1813–9. doi: 10.2105/AJPH.2006.089532. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Neuhauser L, Kreps G. Rethinking Communication in the E-Health Era. Journal of Health Psychology. 2003;8(1):7–23. doi: 10.1177/1359105303008001426. [DOI] [PubMed] [Google Scholar]
- Neuhauser L, Rothschild B, Graham C, Ivey S, Konishi S. Participatory Design of Mass Health Communication in Three Languages for Seniors and People with Disabilities on Medicaid. American Journal of Public Health. 2009;99(12):189–95. doi: 10.2105/AJPH.2008.155648. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Osborne H. Health Literacy from A to Z: Practical Ways to Communicate Your Health Message. Sudbury, MA: Jones & Bartlett Publishers; 2005. [Google Scholar]
- Parsons H. What Happened at Hawthorne? New Evidence Suggests the Hawthorne Effect Resulted from Operant Reinforcement Contingencies. Science. 1974;183(4128):922–32. doi: 10.1126/science.183.4128.922. [DOI] [PubMed] [Google Scholar]
- Prochaska J, Velicer W. Behavior Change: The Transtheoretical Model of Health Behavior Change. American Journal of Health Promotion. 1997;12(1):38–48. doi: 10.4278/0890-1171-12.1.38. [DOI] [PubMed] [Google Scholar]
- Root J, Stableford S. Easy-to-Read Consumer Communications: A Missing Link in Medicaid Managed Care. Journal of Health Politics, Policy and Law. 1999;24(1):1–26. doi: 10.1215/03616878-24-1-1. [DOI] [PubMed] [Google Scholar]
- Rudd R, Moeykens B, Colton T. Health and Literacy: A Review of Medical and Public Health Literature. In: Comings J, Garner B, Smith C, editors. The Annual Review of Adult Learning and Literacy. San Francisco, CA: Jossey-Bass; 2000. pp. 158–99. [Google Scholar]
- Schmitt N. Uses and Abuses of Coefficient Alpha. Psychological Assessment. 1996;8(4):350–3. [Google Scholar]
- Stanford Patient Education Research Center. Chronic Disease Self-Efficacy Scales. [accessed on July 16, 2007]. Available at http://patienteducation.stanford.edu/research/secd32.pdf.
- Stokols D. The Social Ecological Paradigm of Wellness Promotion. In: Jammer M, Stokals D, editors. Promoting Human Wellness: New Frontiers for Research, Practice, and Policy. Berkeley, CA: University of California Press; 2000. pp. 21–37. [Google Scholar]
- U.S. Department of Health and Human Services. Quick Guide to Health Literacy. 2008. [accessed on October 31, 2008]. Available at http://www.health.gov/communication/literacy/quickguide/
- Vaiana M, McGlynn E. What Cognitive Science Tells Us about the Design of Reports for Consumers. Medical Care Research and Review. 2002;59(1):3–35. doi: 10.1177/107755870205900101. [DOI] [PubMed] [Google Scholar]
- Weiss B, Blanchard J, McGee D, Hart G, Warren B, Burgoon M, Smith K. Illiteracy among Medicaid Recipients and Its Relationship to Health Care Costs. Journal of Health Care for the Poor and Underserved. 1994;5(2):99–111. doi: 10.1353/hpu.2010.0272. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.