Abstract
Purpose
Examine associations between readability of survey items and missing data rates in a sample of White and African American (AA) Medicare enrollees in managed care plans.
Methods
Consumer Assessment of Healthcare Provider and Systems (CAHPS®) 2.0 health plan survey data collected from 139,284 respondents (127,524, Whites and 11,760, AAs) in 321 health plans. Product-moment correlations were computed between Flesch-Kincaid (F-K) readability estimates and the CAHPS® item missing data rates.
Results
F-K reading levels for items ranged from 4.8 to 17.7 with a mean of 8.9 across items. Missing data rates ranged from 1 to 10%, with AAs having significantly higher missing data rates. Correlations between missing data rates and item-level readability were statistically significant for Whites (r = 0.33, p = 0.0515) and AAs (r = 0.37, p =0.0284).
Conclusions
The significant associations between missing data rates and item-level readability estimates indicate that the completion of survey items varies by their readability. Enhancing the readability of survey items can improve the inclusion of survey data collected from different respondents.
Keywords: Flesch-Kincaid readability estimates, CAHPS survey, African Americans, Medicare enrollees
Introduction
The validity of self-reported data depends on the ability of the respondent to comprehend the questions asked1. Readability is the semantic and syntactic attributes of the written word.2,3 Readability measures text complexity and is operationalized as the number of syllables per word and the average number of words per sentence in a passage of text.4,5 Readability is often expressed as the number of years of education required for understanding. Educational level may not always be consistent with literacy level but literacy skills acquired through formal education are important for navigating daily activities in life.6
Health surveys need to be designed so that the literacy level required for completing them matches the literacy skills of the target respondents. Calderon et al.3 wrote that the context provided by prose can help the reader comprehend the meaning of a particular passage or sentence whereas in a survey, the reader is expected to comprehend each item independently without the benefit of context. If survey respondents tend to skip over items that are difficult to comprehend, then difficult to understand survey questions may have higher rates of non-response.
Patient health care reports and ratings offer opportunities for quality improvement.7,8 However, missing items on such surveys may also reflect problems with understanding, relevance, and/or acceptability of the questions.9,10 Fongwa et al.11 found that African American Medicare enrollees had significantly higher missing data rates than their White counterparts. We examined the associations between readability of survey items and missing data rates using data from a sample of Medicare managed care beneficiaries who completed the CAHPS® Medicare 2.0 survey. We hypothesize that missing data rates will be positively correlated with the readability level of the CAHPS® survey items.
Methods
The CAHPS® 2.0 data, collected in 2002, came from a random sample of Medicare enrollees (n = 151,214 respondents with 82% response rate) from 321 health plans. The data were collected through the mail (82%) and telephone calls (18%). The gender of those who responded to the survey (n = 151,214) and those who did not (n = 33,568) was similar but significantly different because of the large sample size (58% [respondents] versus 57% [non-respondents] female; χ2 = 10.99, p = 0.0009). In this paper, we examine the subset of White (n = 127,524) and African-American (n = 11,760) respondents from the overall sample of 151, 214. The study protocol was approved by the University of California Los Angeles, Institutional Review Board (UCLA-IRB # 02-450, 11/05/02).
Readability formula
The Flesch-Kincaid (F-K) formula rates text on a U.S. grade-school level such as that an average eight grader would be able to read a document that scores 8.0. Score generated by using the F-K formula are highly correlated with scores from other commonly used readability formulae.3,12 The F-K formula generates scores based on the average number of syllables per word and number of words per sentence. The correspondence between scores obtained by the F-K formula and the reading difficulty rating are found in Table 1. The formula for calculating F-K is as follows: F-K reading grade level score (0.039 × ASL) + (11.8 × ASW) – 15.59, where ASL is the average sentence length (the number of words divided by the number of sentences) and ASW is the average number of syllables per word (the number of syllables divided by the number of words).
Table 1. Reading Difficulty Rating of Flesch-Kincaid Grade Level Scores.
| Reading Difficulty Rating | Flesch-Kincaid Grade Level Score |
|---|---|
| Very easy | 5th |
| Easy | 6th |
| Fairly easy | 7th |
| Standard | 8th – 9th |
| Fairly difficult | 10th – 12th |
| Difficult | 13th – 16th |
| Very difficult | ≥College graduate |
Adapted from Calderon et al., 2006
CAHPS® items
The CAHPS® 2002 Medicare satisfaction survey consists of 21 core items including five multi-item scales or composites measuring getting care quickly/timeliness, provider-doctor communication, office staff helpfulness, getting needed care, and health plan customer service and four single-item global ratings of care from one's personal doctor or nurse, specialists, overall health care, and health plan.13 Reports of care capture a patient's specific experiences on what did or did not happen from the patient's perspective and ratings consist of personal evaluation of care. Items for the timeliness of care, provider communication, and staff helpfulness composites were administered using a four-point response scale (Never, Sometimes, Usually, Always), whereas items for the getting needed care and plan customer service composites were administered using a three-point response scale (A big problem, A Small problem, Not a problem). The four global rating items were administered using a 0-10 response format, with 0 the worst possible and 10 the best possible rating (see Appendix A).
Analysis
We estimated readability using the F-K formula used in Microsoft Word 2003 for the 21 items constituting the core reporting items of the CAHPS® 2.0 Medicare Survey and 4 global rating items. We also included 14 other items from the survey. These items consisted of 2 global rating items on getting specialty care and health in the last six months (questions 15 and 24), 6 from “Other services” items (questions 39, 41, 43, 44, 45, and 51), and 6 background items (questions 66, 67, 72, 78, 86, and 89) (see Appendix B). Hence, our overall item pool included 35 items. Missing data was defined as “don't know”/refused for telephone interviews and “no response” for applicable items on the mail survey. That is, appropriately skipped items were not counted as missing data. We conducted a paired t-test of the difference in item missing data rates for African Americans and Whites. In addition, we estimated Pearson product-moment correlations between the item-readability estimates and missing data rates for African Americans and Whites separately.
We conducted additional analysis to see if there are differences in item non-response (dependent variable) by level of education, age, gender, race, and self-rated health (independent variables). We created a variable representing the proportion of missingness for applicable CAHPS® items. Then the proportions were multiplied by 100 so the regression coefficients would be in percentage-point units. Ethnicity was coded 0 for White and 1 for African American. For education, two dummy categories were created with < high school education as the reference group. Gender was coded as 0 for male and 1 for female. The excellent response category for the health status variable was selected as the reference group.
Results
Demographics
The analytic sample size (139,284) consisted of 58% females; about 6% were younger than 65 years, 21% were ages 65 to 69, 27% 70 to 74, 22% 75 to 79, and 23% 80 or older (Table 2). Education ranged from less than 8th grade (11%) to more than 4-year college degree (7%) with most people being either a high school graduate (17%) or having a GED (37%). The modal self-rated health was good (38%), followed by fair (26%), very good (23%), excellent (7%), and poor (7%). Mode of survey administration (mail versus telephone) differed significantly by race (χ2 = 124.72, p <.0001) with African Americans being more likely than Whites to respond via telephone (Table 2). African American respondents were more likely to be female, younger, less educated, and have worse health than Whites.
Table 2. Demographics for All Medicare Managed Care Enrollees in Analytic Sample.
| Variable | White/non-Hispanic | African American | Combined Sample and Percentage |
|---|---|---|---|
|
| |||
| Number and Percentage (in parentheses) | Number and Percentage | ||
| SAMPLE | 127,524 (92) | 11,760 (8) | 139,284 (100) |
| SURVEY METHOD | |||
| 106,566 (84) | 8,309 (71) | 114,875 (82) | |
| Telephone | 20,958 (16) | 3,451 (29) | 24,409 (18) |
| GENDER | |||
| Female | 73,844 (58) | 7,574 (64) | 8,1418 (58) |
| Male | 53,680 (42) | 4,186 (36) | 57,866 (42) |
| AGE | |||
| 44 or younger | 833 (1) | 182 (2) | 1,015 (0.7) |
| 45 to 64 | 6,372 (5) | 1,176 (10) | 7,548 (5) |
| 65 to 69 | 27,170 (21) | 2,680 (23) | 29,850 (21) |
| 70 to 74 | 34,536 (27) | 3,372 (29) | 37,908 (27) |
| 75 to 79 | 28,438 (22) | 2,320 (20) | 30,757 (22) |
| 80 or older | 30,175 (24) | 2,030 (17) | 32,205 (23) |
| EDUCATION | |||
| 8th grade or < | 13,166 (11) | 2,375 (21) | 15,541 (11) |
| Some High School | 20,636 (17) | 3,072 (27) | 23,708 (17) |
| Graduate or GED | 47,285 (38) | 3,292 (29) | 50,577 (37) |
| College/2 year degree | 25,989 (21) | 1,757 (15) | 27,746 (20) |
| 4 year college degree | 8,510 (7) | 405 (4) | 8,915 (7) |
| > 4 year college degree | 8,731 (7) | 475 (4) | 9,206 (7) |
| SELF-RATED HEALTH | |||
| Poor | 7,935 (6) | 995 (9) | 8,930 (7) |
| Fair | 30,993 (25) | 4,157 (36) | 35,157 (26) |
| Good | 48,395 (39) | 4,080 (35) | 52,475 (38) |
| Very good | 29,788 (24) | 1,759 (15) | 31,547 (23) |
| Excellent | 8,657 (7) | 536 (5) | 9,193 (7) |
Note: All variables differed significantly between African Americans and whites at p < .0001.
The F-K grade level for items ranged from 4.8 to 17.7 (rate overall health [Q = question, Q66] and rate health care [Q37], respectively) with a mean of 8.9 (Table 3). Missing data rates ranged from 1% (getting care quickly [19]) to 10% (other health services [Q43]) for Whites and 2% (getting care quickly [Q19]) to 14% (global rating [Q16]) for African Americans.
Table 3. Readability and missing data rates for CAHPS® 2.0 Medicare Items.
| Item | Question # | African American | White | Full Sample | |
|---|---|---|---|---|---|
| Missing rate | Missing rate | Missing rate | Readability | ||
| Frequency getting needed care quickly during regular hours in last 6 months | 19 | 1.6 | 0.8 | 1.0 | 8.7 |
| Frequency getting needed care right away in last 6 months | 23 | 2.2 | 1.2 | 1.5 | 10.9 |
| Frequency getting appointment as soon as wanted in last 6 months | 21 | 2.6 | 1.3 | 1.6 | 6.3 |
| Frequency seeing scheduled-to-see person within 15 minutes of appointment | 30 | 7.9 | 4.4 | 6.1 | 8.5 |
| Frequency care providers listened carefully to in last 6 months | 33 | 5.8 | 2.6 | 4.3 | 9.0 |
| Frequency care providers explained things to your understanding in last 6 months | 34 | 5.5 | 2.6 | 4.3 | 8.8 |
| Frequency care providers respected what you said in last 6 months | 35 | 5.5 | 2.7 | 4.4 | 7.7 |
| Frequency care providers spent enough time with you in last 6 months | 36 | 6.5 | 3.1 | 4.9 | 7.3 |
| Frequency office staff treated your with courtesy and respect in last 6 months | 31 | 6.8 | 3.9 | 5.6 | 9.0 |
| Frequency office staff were as helpfulness as expected in last 6 months | 32 | 5.9 | 2.6 | 4.3 | 8.5 |
| Problem you have seeing the care provider you are happy with as a Medicare enrollee | 12 | 3.8 | 2.9 | 3.1 | 9.5 |
| Problem you have seeing a specialist you need in last 6 month | 14 | 1.8 | 1.5 | 1.6 | 7.7 |
| Problem getting needed care, tests or treatment in last 6 months | 27 | 1.5 | 0.9 | 1.0 | 10.2 |
| Problem with delays in health care while you waited for health plan approval in last 6 months | 29 | 1.6 | 1.1 | 1.2 | 9.0 |
| Problem understanding information in last 6 months | 48 | 3.4 | 1.7 | 1.9 | 7.9 |
| Problem getting needed help when you called customer service in last 6 months | 50 | 2.2 | 1.2 | 1.3 | 9.6 |
| Problem you had with health plan paper work in last 6 months | 56 | 4.6 | 4.3 | 4.5 | 6.6 |
| On a scale of 0 (worst) to 10 (best), rate your personal care provider | 07 | 3.0 | 2.5 | 2.8 | 17.1 |
| On a scale of 0 (worst) to 10 (best), rate the specialist you saw | 16 | 13.5 | 6.5 | 8.0 | 14.2 |
| Frequency, gone to specialist for care in last 6 months | 15 | 7.8 | 3.8 | 4.6 | 5.6 |
| On a scale of 0 (worst) to 10 (best), rate your health care in the last 6 months | 37 | 6.8 | 3.2 | 4.9 | 17.7 |
| On a scale of 0 (worst) to 10 (best), rate your health plan | 57 | 6.1 | 4.0 | 5.8 | 12.6 |
| Problem getting needed special medical equipment via Medicare in last 6 months | 39 | 6.1 | 4.8 | 5.0 | 11.1 |
| Problem getting needed therapy via Medicare in last 6 months | 41 | 4.3 | 2.7 | 3.0 | 9.9 |
| Problem getting needed home care or assistance in last 6 months | 43 | 11.0 | 10.0 | 10.4 | 10.3 |
| Frequency you got needed prescribed Medicine in last 6 months | 44 | 2.7 | 1.4 | 2.9 | 6.7 |
| Problem getting needed prescribed medicine in last months | 45 | 2.6 | 1.6 | 3.1 | 8.3 |
| Frequency, Medicare customer service people were as helpful as expected | 51 | 2.6 | 1.4 | 1.6 | 9.0 |
| Overall rating of your health now | 66 | 1.8 | 1.1 | 2.9 | 4.8 |
| Rate your health now compared to a year ago | 67 | 2.2 | 1.2 | 3.1 | 5.0 |
| Rate your overall mental health now | 72 | 2.3 | 1.2 | 3.2 | 5.8 |
| Frequency you walked/exercised for more than 20 minutes each time in last 4 weeks | 78 | 3.2 | 1.9 | 4.0 | 7.2 |
| Frequency you were advised to quit smoking by your care provider in last 6 months | 86 | 8.1 | 6.3 | 6.6 | 9.9 |
| Your highest level of education completed | 89 | 1.9 | 1.0 | 4.1 | 4.9 |
| Frequency you went to emergency for care in last 6 months | 24 | 3.4 | 2.6 | 3.5 | 6.9 |
Readability mean score = 8.9
Question 19 in the getting care quickly composite/category had the lowest missing data rate for African Americans and Whites. Overall, African Americans had an average of 4.5 (standard deviation [SD] = 2.84) missing data per item compared to 2.75 (SD = 1.95) for Whites. The paired t-test of the difference in missing data rates at the item level for African Americans and Whites was significant (t = −7.48 at p = < 0.0001) with African Americans having significantly higher mean item missing data rates. The product-moment correlation between item-level readability and missing data rate correlation was 0.33 (p = 0.0515) for Whites and 0.37 (p = 0.0284) for African Americans.
The results of the regression predicting proportion of missing responses to applicable CAHPS® item are shown in Table 4. Missing data was found to be 1.45 percentage points higher for African Americans than for Whites. In addition, missing data for respondents with < high school education was 1.12 percentage points higher than the rate for those with high school diplomas only, 1.57 percentage points higher than those with some college, and 1.79 percentage points higher than for college graduates. Moreover, women had two-thirds of a percentage point more missinging data than men. Finally, those with excellent self-rated health had more missing data than those with very good, good, fair, and poor health status (difference in missingness rates ranged from about one-third of a percentage point to one-half of a percentage point).
Table 4. Regression of proportion of missing responses to applicable CAHPS® item on race, education, gender, and self-rated health.
| Variable | DF | Estimate | SE | t | P | 95% Confidence Interval | |
|---|---|---|---|---|---|---|---|
| Intercept | 1 | 3.30 | 0.09 | 38.64 | <.0001 | 3.13 | 3.47 |
| Black race | 1 | 1.45 | 0.07 | 20.64 | <.0001 | 1.32 | 1.59 |
| High school graduate | 1 | −1.12 | 0.05 | −23.16 | <.0001 | −1.22 | −1.03 |
| Some college | 1 | −1.57 | 0.06 | −27.88 | <.0001 | −1.68 | −1.46 |
| College graduate | 1 | −1.79 | 0.07 | −27,36 | <.0001 | −1.92 | −1.66 |
| Female | 1 | 0.64 | 0.04 | 16.22 | <.0001 | 0.56 | 0.72 |
| Very good health | 1 | −0.47 | 0.08 | −5.51 | <.0001 | −0.63 | −0.30 |
| Good health | 1 | −0.35 | 0.08 | −4.36 | <.0001 | −0.51 | −0.19 |
| Fair health | 1 | −0.40 | 0.08 | −4.78 | <.0001 | −0.57 | −0.24 |
| Poor health | 1 | −0.55 | 0.11 | −5.14 | <.0001 | −0.76 | −0.34 |
Note: Holdout group was White, less than high school education, male, and excellent health.
Discussion
This study found a positive association between readability and missing data rates for items in the CAHPS® 2.0 Medicare survey. The average estimated readability level for the CAHPS® Medicare survey items was a ninth grade level. This is potentially problematic for the 21% of African Americans and 11% Whites that have eight or fewer years of education. The higher level of missing data among African Americans than Whites in this study was, at least in part, due to African Americans' lower average educational level.
The lowest item missing data rate among African Americans (2%) (Table 3) corresponds to the item with the lowest readability estimate (4.8 for item 66) (see Appendix B). By contrast, item 16 had the highest missing data rate (14%; Table 3) and a readability of 14.2 (see Appendix B). In general, items with a lower readability level had higher response rates. Low reading level among older African Americans may be associated with their inability to comprehend survey items and subsequent performance on health-related surveys. Research findings inform clinical practice14 and incomplete responses from the most vulnerable African American Medicare patients11 may set the stage for misinterpretations of their feedback. Missing data from African Americans raises concerns about possible bias in drawing inferences from the data. Survey items need to be appropriate for the target population.
African Americans had more missing data than Whites. Persons with less than high school level education had greater missing data than those with higher level education. The latter finding supports the hypothesis that missing data rates are positively related to readability of the CAHPS® survey items. Interestingly, people in excellent health had higher missingness rates than those with very good, good, fair, and poor self-rated health. People in excellent health (who might also be younger) might not be as engaged in their health care as those with worse health and this could account for their higher rates of missing CAHPS® data. Low literacy affects the validity of self-administered instruments15,16 and therefore challenges the provision of quality of care.17,18 A lack of feedback from patients challenged by their literacy abilities could hinder quality improvement efforts aiming at patient-centered care for all patients.
Strengths and Limitations
Few readability studies have focused on older adults and specifically on Medicare enrollees. Also, most readability estimates are provided as overall survey average whereas our study uses item-level analysis. We could not examine other study variables such as education or self-rated health in the non-respondents to the survey because gender was the only variable for which we had complete data on. The study findings may not generalize beyond Medicare enrollees. This study used only CAHPS® 2.0 survey and there are other instruments that may have appropriate reading levels for older adults.
Conclusion
The findings from this study link missing data rates of CAHPS® 2.0 survey items to readability of items. Outcomes researchers need to ascertain that survey materials are appropriate for the educational background of potential study participants. Besides observing for informal cues on reading difficulty such as inability to read because of forgotten eye glasses or exhibition of facial signs of frustration while reading a passage19 among older adults, we recommend that researchers assess reading ability in their study population whenever possible. In designing health surveys, enhancing the readability of the items can improve the inclusion of survey data collected from respondents
Researchers might also consider using phone or in-person interviews instead of self-administered surveys. However, an individual with very low level of education may still struggle with an interview due to lack of comprehension of the concepts in survey items. As noted by Morales, Weidmer, and Hays,20 finding a balance between collecting important information and maintaining a reasonable level of survey readability is an important consideration for researchers in future versions of CAHPS® surveys. The CAHPS® survey has been validated in several populations but more work needs to be done on the survey items. The higher mean readability level across items in this study is challenging to those with eight or less years of education (21% African Americans versus 11% Whites). Low readability can limit understanding of crucial health information that can impact patient health outcomes. Survey items meant to assess patients' perceptions of their care and global attitude about their provider, health care and services must be understood by the patients themselves.
Acknowledgments
Supported in part by grant number 5 U18 HS-00924 and 1 U18 HS-016980 from AHRQ, the UCLA/Drew Project EXPORT, NIH, National Center on Minority Health & Health Disparities, (P20-MD00148-01) and the UCLA Center for Health Improvement in Minority Elders/Resource Centers for Minority Aging Research, National Institutes of Health, National Institute of Aging, NIH/NIA/NCMHD, under Grant P30-AG-021684. The authors also acknowledge the assistance with access to data provided by the CMS and staff affiliated with the CAHPS Database effort. Hays was also supported in part by AG20679-01 from the National Institute of Aging and the UCLA Older Americans Independence Center, NIH/NIA Grant P30-AG028748.
Appendix A.
2002 Medicare CAHPS: Reports and Global Ratings of Care.
| Adult Survey Composite | Survey Items | Response Format |
|---|---|---|
| Getting Care Quickly/Timeliness |
|
|
| Provider Communication |
|
|
| Office Staff Helpfulness |
|
|
| Getting Needed Care |
|
|
| Customer Service |
|
|
| Global Rating | ||
| Rate Your personal doctor or nurse |
|
0-10 Scale |
| Rate Getting health care from a specialist |
|
0-10 Scale |
| Rate Your health care in the last 6 months |
|
0-10 Scale |
| Rate Your health plan |
|
0-10 Scale |
Appendix B.
Medicare CAHPS® 2.0: Composite Level Missing Data Rate and Item Level Flesch-Kincaid Readability Estimate Combined for African Americans and Whites.
| Medicare Cahps Item | Item # | Mail Missing Data | Phone Missing Data | Flesch-Kincaid Grade Level Estimate |
|---|---|---|---|---|
| Getting Care Quickly/Timeliness | 8 | 7* | ||
| In the last 6 months, when you called during regular office hours, how often did you get the help or advice you needed? | Q19 | 8.7 | ||
| In the last 6 months, when you needed care right away for an illness, injury, or condition, how often did you get care as soon as you wanted? | Q23 | 10.9 | ||
| In the last 6 months, how often did you get an appointment for health care as soon as you wanted? | Q21 | 6.3 | ||
| In the last 6 months, how often did you see the person you came to see within 15 minutes of your appointment? | Q30 | 8.5 | ||
| Provider Communication | 3 | 4* | ||
| In the last 6 months, how often did doctors or other health providers listen carefully to you? | Q33 | 9.0 | ||
| In the last 6 months, how often did doctors or other health providers explain things in a way you could understand? | Q34 | 8.8 | ||
| In the last 6 months, how often did doctors or other health providers show respect for what you had to say? | Q35 | 7.7 | ||
| In the last 6 months, how often did doctors or other or health providers spend enough time with you? | Q36 | 7.3 | ||
| Office Staff Helpfulness | 4 | 3* | ||
| In the last 6 months, how often did office staff at a doctor's office or clinic treat you with courtesy and respect? | Q31 | 9.0 | ||
| In the last 6 months, how often were office staff at a doctor's office or clinic as helpful as you thought they should be? | Q32 | 8.5 | ||
| Getting Needed Care | 10 | 9* | ||
| Since you joined Medicare, how much of a problem, if any, was it to get a personal doctor or nurse you are happy with? | Q12 | 9.5 | ||
| In the last 6 months, how much of a problem, if any, was it to see a specialist that you needed to see? | Q14 | 7.7 | ||
| In the last 6 months, how much of a problem, if any, was it to get the care, tests or treatment you or a doctor believed necessary? | Q27 | 10.2 | ||
| In the last 6 months, how much of a problem, if any, were delays in health care while you waited for approval from your health plan? | Q29 | 9.0 | ||
| Customer Service | 10 | 10 | ||
| In the last 6 months, how much of a problem, if any, was it to find or understand information? | Q48 | 7.9 | ||
| In the last 6 months, how much of a problem, if any, was it to get the help you needed when you called your health plan's customer service? | Q50 | 9.6 | ||
| In the last 6 months, how much of a problem, if any, did you have with paperwork for your health plan? | Q56 | 6.6 | ||
| Global Rating | ||||
| [Rate Your personal doctor or nurse] Using any number from 0 to 10, where 0 is the worst personal doctor or nurse possible and 10 is the best personal doctor or nurse possible, what number would you use to rate your personal doctor or nurse? | Q7 | 3 | 4* | 17.1 |
| [Rate Getting health care from a specialist] Using any number from 0 to 10 where 0 is the worst specialist possible and 10 is the best specialist possible, what number would you use to rate the specialist? | Q16 | 7 | 6* | 14.2 |
| In the last 6 months, how many times did you go to specialists for care for yourself? | Q15 | 5.6 | ||
| [Rate Your health care in the last 6 months] Using any number from 0 to 10 where 0 is the worst health care possible and 10 is the best health care possible, what number would you use to rate all your health care in the last 6 months from all doctors and other health providers? | Q37 | 3 | 4* | 17.7 |
| [Rate Your health plan] Using any number from 0 to 10 where 0 is the worst health plan possible and 10 is the best health plan possible, what number would you use to rate your health plan? | Q57 | 3 | 5* | 12.6 |
| Other health services | ||||
| In the last 6 months, how much of a problem, if any, was it to get the special medical equipment you needed through your Medicare health plan? | Q 39 | 11.1 | ||
| In the last 6 months, how much of a problem, if any, was it to get the special therapy you needed through your Medicare health plan? | Q 41 | 9.9 | ||
| In the last 6 months, how much of a problem, if any, was it to get the home health care or assistance you needed through your Medicare health plan? | Q 43 | 10.3 | ||
| In the last 6 months, how often did you get the prescription medicine you needed? | Q 44 | 6.7 | ||
| In the last 6 months, how much of a problem, if any, was it to get the prescription medicine you needed? | Q 45 | 8.3 | ||
| In the last 6 months, how often were people at your Medicare health plan's customer service as helpful as they should be? | Q 51 | 9.0 | ||
| About You | ||||
| In general, how would you rate your overall health now? | Q 66 | 4.8 | ||
| Compared to one year ago, how would you rate your health in general now? | Q 67 | 5.0 | ||
| In general, how would you rate your overall mental health now? | Q 72 | 5.8 | ||
| In the past 4 weeks, how often have you walked and/or exercised for more than 20 minutes at a time? | Q 78 | 7.2 | ||
| In the last 6 months, on how many visits were you advised to quit smoking by a doctor or other health provider in your plan? | Q 86 | 9.9 | ||
| What is the highest grade or level of school that you have completed? | Q 89 | 4.9 | ||
| Your health care in the last 6 months | ||||
| In the last 6 months, how many times did you go to an emergency room to get care for yourself? | Q24 | 6.9 |
Chi-square test: difference in missing data rate by method of administration was significant at p <.0001. Q = question number on Medicare CAHPS® survey.
Footnotes
Provided original guidance on design, analysis, and organization of the manuscript. Read all drafts and provided feedback including critiques, edits, and structure
References
- 1.Beckman HT, Lueger RJ. Readability of self-report clinical outcome measures. J Clinic Psych. 1997;53(8):785–789. doi: 10.1002/(sici)1097-4679(199712)53:8<785::aid-jclp1>3.0.co;2-f. [DOI] [PubMed] [Google Scholar]
- 2.Chall JS. The beginning years. In: Zakaluk BL, Samuels SJ, editors. Readability, Its Past, Present and Future. Newark, Del: International Reading Association; 1988. pp. 2–13. [Google Scholar]
- 3.Calderon JL, Morales LS, Liu H, et al. Variation in the readability of items within surveys. Am J Med Qual. 2006;21:49–56. doi: 10.1177/1062860605283572. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Slaughter L, Kelselman A, Kushniruk A, et al. A framework for capturing the interactions between laypersons' understanding of disease, information gathering behaviors and actions taken during an epidemic. J Biomed Info. 2005;38(4):298–313. doi: 10.1016/j.jbi.2004.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kincaid JP, Fishburne RP, Rogers RL, et al. Research Branch Report 8-75. Memphis: Naval Air Station; 1975. Derivation of new readability formulas (Automated Readability Index Fog Count, and Flesch Reading Ease Formula) for Navy enlisted personnel. [Google Scholar]
- 6.Greenberg E, Dunleavy E, Kutner M. U S Department of education. Washington, DC: National Center for Education Statistics; 2007. Literacy behind bars: Results from the 2003 National Assessment of Adult Literacy Prison Survey (NCES 2007-473) [Google Scholar]
- 7.Bost JE. Managed care organizations publicly reporting three years of HEDIS measures. Managed Care Interface. 2001;14(9):50–54. [PubMed] [Google Scholar]
- 8.Shih SC, Bost HE, Pawlson G. Standardized health plan reportings in four areas of preventive health care. Am J Prev Med. 2003;24(4):293–300. doi: 10.1016/s0749-3797(03)00025-4. [DOI] [PubMed] [Google Scholar]
- 9.Cunningham WE, Hays RD, Burton TM, et al. Health status measurement performance and health status differences by age, ethnicity, and gender: Assessment in the Medical Outcomes Study. J Health Care Poor & Underserved. 2000;11(1):58–76. doi: 10.1353/hpu.2010.0638. [DOI] [PubMed] [Google Scholar]
- 10.McHorney CA, Ware JE, Jr, Lu JF, et al. The MOS 36-item short form health survey (SF-36): III Test of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994;32:40–66. doi: 10.1097/00005650-199401000-00004. [DOI] [PubMed] [Google Scholar]
- 11.Fongwa MN, Cunningham W, Weech-Maldonado R, et al. Comparison of data quality for reports and ratings of ambulatory care by African American and White Medicare Managed care enrollees. J Aging & Health. 2003;18(5):701–721. doi: 10.1177/0898264306293264. [DOI] [PubMed] [Google Scholar]
- 12.Meade CD, Smith CF. Readability formulas: cautions and criterion. Pt Educ Counsel. 1991;17:153–158. [Google Scholar]
- 13.Weech-Maldonado R, Morales LS, Elliott M, et al. Race/ethnicity, language, and patients assessments of care in Medicaid managed care. Health Serv Research. 2003;38(3):789–808. doi: 10.1111/1475-6773.00147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Mody L, Miller DK, McGloin JM, et al. Recruitment of older adults in aging research. J Am Geriatrics Soc. 2008;56:2340–48. doi: 10.1111/j.1532-5415.2008.02015.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Mayeaux EJ, Jr, Davis TC, Jackson RH, et al. Literacy and self-reported educational levels in relation to Mini-mental State Examination scores. Fam Med. 1995;27(10):658–62. [PubMed] [Google Scholar]
- 16.MacDiarmid SA, Goodson TC, Holmes TM, et al. An assessment of the Comprehension of the American Urological Association Symptom Index. J Urol. 1998;159(3):873–874. [PubMed] [Google Scholar]
- 17.French KS, Larrabee JH. Relationships among educational material readability, client literacy, perceived beneficence, and perceived quality. J Nurs Care Qual. 1999;13(6):68–82. doi: 10.1097/00001786-199908000-00008. [DOI] [PubMed] [Google Scholar]
- 18.Mayeaux EJ, Jr, Murphy PW, Arnold C, et al. Improving patient education for patients with low literacy skills. Am Fam Physician. 1996;53(1):205–211. [PubMed] [Google Scholar]
- 19.Meade CD, Byrd JC. Patient literacy and the readability of smoking education literature. Am J Public Health. 1989;79(2):204–206. doi: 10.2105/ajph.79.2.204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Morales LS, Weidmer BO, Hays RD. Readability of CAHPS 2.0 child and adult core surveys. Feature Paper. 2001:83–90. [Google Scholar]