Differences in CAHPS Reports and Ratings of Health Care Provided to Adults and Children

Abstract

Background

Consumer assessment of health care is an important metric for evaluating quality of care. These assessments can help purchasers, health plans and providers deliver care that fits patients’ needs.

Objective

To examine differences in reports and ratings of care delivered to adults and children and whether they vary by site.

Research Design

This observational study compares adult and child experiences with care at a large west coast medical center and affiliated clinics and a large mid-western health plan using Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) Clinician & Group 1.0 Survey data.

Results

Office staff helpfulness and courtesy was perceived more positively for adult than pediatric care in the west coast site. In contrast, more positive perceptions of pediatric care were observed in both sites for coordination of care, shared decision making, overall rating of the doctor, and willingness to recommend the doctor to family and friends. In addition, pediatric care was perceived more positively in the Midwest site for access to care, provider communication, and office staff helpfulness and courtesy. The differences between pediatric care and adult care were larger in the mid-western site than the west coast site.

Conclusions

There are significant differences in the perception of care for children and adults with care provided to children tending to be perceived more positively. Further research is needed to identify the reasons for these differences and provide more definitive information at sites throughout the U.S.

INTRODUCTION

Consumer evaluations of health care are an important part of patient-centeredness, consistent with the Institute of Medicine’s (IOM) vision in the National Health Care Quality Report,¹ because they provide direct information on how well providers and health plans meet the needs of patients.^2–3 These perceptions of care are associated with adherence to treatment recommendations,⁴ utilization of services,⁵ and disenrollment.⁶

Health plans and healthcare organizations may provide different infrastructure (facility and environment) and/or personnel (qualification and appearance) for adult and pediatric care.^7–8 Information about experiences with care can help health plans target areas for improvement. Chong analyzed CAHPS data from Iowa and found that timeliness of care and short in-office waiting time was of greater importance to adults, while courtesy and helpfulness of office staff members was of greater importance in parental assessments of the child’s doctor.⁹ Although an existing study reported similar CAHPS scores and plan rankings for adult and pediatric care, these scores can and do differ by specific components – for example, adult care was rated higher than pediatric care for customer service.¹⁰ Information on health care experiences for both adult and pediatric care can help purchasers, health plans, and providers deliver care that fits patient’s needs. In this study, we examine the difference between care experiences of adult and pediatric patients in two large U.S. health care organizations.

METHODS

We analyzed data collected from patients receiving care from a large west coast integrated health system, and from patients in a large health plan in the Midwest. Data were collected in 2009 from the west coast site and in 2008 from the mid-western site.

Survey Instrument

The sample was administered the Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) Clinician & Group 1.0 (CG-CAHPS 1.0) 12-month survey.¹¹ The CG-CAHPS survey includes 5 reporting composites: access to care (5 items), provider communication (5 items), coordination of care (3 items), shared decision making (1 item), and office staff (2 items). Each of these items are administered using a never/ sometimes/ usually/ always response scale. The survey also includes a single global item rating of the doctor (0 to 10 response scale, with 0 being the worst and 10 being the best) and a single item asking whether the patient would recommend the doctor to family or friends (definitely yes/ somewhat yes/ somewhat no/ definitely no).

The survey development process for both the adult and pediatric survey included focus groups, cognitive testing, and field testing in English and Spanish using multiple modes of data collection. The pediatric survey instrument incorporated updates to the survey (developed by the CAHPS consortium members at the Center for Survey Research, UMASS Boston, Harvard Medical School, and Yale School of Public Health).¹² These updates are reflected in the data collected in 2009 and include child development (6 items) and preventive care (5 items) reporting composites assessed using a yes/ no response scale. These new composites are not examined here because they are not included on the adult survey. Abbreviated item stems are provided in the Appendix.

We averaged items within each composite to create scale scores and transformed these averages linearly to a 0–100 possible range, with 100 representing the most positive reports about care. We also transformed the global rating item and the recommend item to a 0–100 possible range.

Survey Administration

The list of doctors eligible for the survey was generated by each participating site. The target size for the number of surveys sent to each adult provider was 100 to 115 unique patients. The target size for the number of surveys sent for each child provider was 120 to 135 unique patients. Doctors who did not meet this eligibility number were excluded. Prior to sampling, patient addresses were standardized using the US Postal Service Coding Accuracy Support System. Records with non-U.S. addressed were removed. The patient file was also processed through the USPS National Change of Address Processing to update addresses. Following address standardization at both sites, all patients were grouped into households based on address alone. No more than one person per “household” was sampled—that is, once a household member has been included in the sample, all other members of the household were ineligible for the survey.

The survey was mailed to patients sampled across the entire cohort of eligible doctors at each participating site. It was introduced by a cover letter written in English and Spanish. An English language version of the survey was enclosed in the mailing. Survey recipients had the option of calling a toll-free number to request the Spanish language version. The cover letter also contained an option to complete the survey online. A follow-up mailing with the survey and the web invitation enclosed was sent to non-respondents four weeks after the initial mailing.

Settings

The west coast site consists of a large multi-specialty practice in an urban setting that is a component of an academic integrated health system. About 83% of the primary care physicians are board certified and the remainder is board eligible. The primary care providers are a mix of family medicine, internal medicine, and pediatrics. The practice operates more than 50 distinct ambulatory clinic locations. Almost all the primary care offices are in small group settings of 4–6 physicians. The practice contracts with all major health plans, including PPOs and HMOs, as well as Medicare and Medicaid. None of the practice sites were NCQA Patient-Centered Medical Home certified.

The mid-western site consists of a network of providers spanning three counties, each of which has at least one mid-sized city and surrounding rural or sub-urban populations. About 91% of all primary care providers are board certified. The primary care providers are also a mix of family medicine, internal medicine, and pediatrics. The practices are predominantly in small group settings, ranging from 1–4 providers. As a health plan, the mid-western site has been NCQA certified as “Excellent” for over a decade.

Participants

The study included responses from 15,051 adults and 2,323 parents or caregivers of children from the west coast site; 7,823 adults and 668 parents or caregivers of children provided responses from the mid-western site. The response rates to the survey were 37% for adults and 25% for children’s parents or guardian at the west coast site. The response rates were 46% for adults and 32% for parents or guardian at the mid-western site. The characteristics of respondents from the two health care organizations were similar (see Table 1). A large proportion of the respondents (adults or parents of children) had some college or four-year college degrees, consistent with the fact that the majority of those in the sample have health insurance coverage. Gender and racial/ethnic distributions differ among the populations, with a lower proportion of racial/ethnic minorities in the mid-western site, reflecting the regional population. The majority of patients who completed the surveys had 1–2 doctor visits during the past year.

Table 1.

Characteristics of the Study Sample Respondents

Respondent Characteristics	West Coast Adults (n=15,051)	West Coast Children (n=2,323)	Mid-west Adults (n=7,823)	Mid-west Children (n=668)
	Percent	Percent	Percent	Percent
Gender
Male	39	15	40	14
Education
8th grade or less	2	2	1	0
Some HS	2	2	4	1
HS grad/GED	11	7	36	14
Some college	26	23	43	44
College grad	23	27	9	25
> 4 years of college	36	38	7	16
Race/Ethnicity
Hispanic	10	25	3	8
African-American	6	5	6	11
Asian/Pacific Islander	12	14	1	2
White	69	52	90	78
Other (including multi- racial and unknown)	3	4	1	1
Age
< 24	2	2	3	6
25–34	8	22	4	27
35–44	10	42	8	38
45–54	15	28	29	24
55–64	21	6	53	4
65+	44	1	4	1
Insurance Type
Self-pay	3	4	0	0
Medicare-FFS	35	0	0	0
Medicaid	2	14	0	0
Commercial(non-HMO)	34	51	0	0
HMO (non-capitated)	6	9	0	0
HMO (capitated)	20	22	100	100
Other	1	1	0	0
Number of MD visits in past year
1	25	25	28	35
2	24	26	27	29
3	17	16	21	18
4	14	14	20	17
5–9	15	16	4	2
10+	6	3	0	0
Usual Source of Care
Yes	46	57	99	97
Length of time with current MD
< 6 mos	21	15	1	3
6 mos- 1 year	18	21	4	5
1 yr - < 3 yr	25	31	16	23
3 yrs – 5 yrs	13	15	13	15
5 yrs or more	23	19	66	55

All CAHPS surveys are collected from adults. This study compared patient-assessed adult care and parent/proxy-assessed pediatric care. Previous studies suggest relatively modest effects of proxies on responses to the CAHPS survey. ^13–14

Statistical Analysis

We estimated the internal consistency reliability for the multi-item scales using coefficient alpha. Then we estimated ordinary least squares regression models separately in the two sites, with the CAHPS composites and ratings as dependent variables and a dummy variable indicator of pediatric care (versus adult care), controlling for child race/ethnicity and respondent (adult) age (in categories: 25–34; 35–44; 45–54; 55–64; 65+; <25 as holdout), education (≤8^th grade; some high school; high school graduate or GED; some college; college graduate; > 4 years of college as holdout), and gender (female as holdout). For the west-coast site we also adjusted for respondent (parent/guardian) self-rated health. (Parent/guardian self-rated health was not collected in the Midwest site for pediatric cases.) Regression results are presented as casemix-adjusted means from recycled predictions. An omnibus test of the potential significance of two-way Interactions between adult versus pediatric care and all the other independent variables in the models was significant only 1 out of 14 times, about what would be expected by chance alone. We also ran a model in the mid-western site that imputed excellent health for the parent/guardian of pediatric cases and found a similar pattern of results to those reported here.

All analyses were conducted using SAS version 9.1. A 0.05 threshold of significance (using two-sided tests) was used as the standard of significance throughout – all results discussed were significant at that threshold.

RESULTS

Internal consistency reliability estimates for the four multi-item composites were: access to care (0.84), communication (0.94), coordination of care (0.65), and office staff (0.89).

Casemix-adjusted means for reports and ratings of care appear in Table 2. Absolute scores were lowest for access to care in both adult care (71 at west coast site and 77 at mid-western site) and pediatric care (71 at west coast site and 89 at mid-western site). Absolute scores were highest for shared decision making for both adult care (95 at west coast site and 94 at mid-western site) and pediatric care (97 at west coast site and 99 at mid-western site). Office staff helpfulness and courtesy were perceived more positively for adult than pediatric care (by 2 points) in the west coast site. In contrast, every one of the 7 CAHPS measures was perceived more positively for pediatric care than adult care in the mid-western site and for 4 of the CAHPS measures in the west coast site (coordination of care, shared decision making, overall rating of the doctor, would recommend the doctor to family and friends). The significant differences favoring pediatric care ranged from 2–3 points in the west coast site and 5–12 points in the mid-western site.

Table 2.

Casemix adjusted means (SD) for CAHPS reports and ratings of care.^†

CAHPS Measure	West	Coast	Mid-	West
	Adult care	Pediatric care	Adult care	Pediatric care
Access to care	71 (27)	71 (30)	77 (20)	89* (22)
Provider Communication	91 (18)	92 (20)	90 (16)	95* (17)
Coordination of Care	83 (24)	86* (26)	83 (22)	93* (24)
Shared Decision Making	95 (21)	97* (22)	94 (22)	99* (24)
Office Staff Helpful and Courteous	86* (20)	84 (22)	87 (19)	95* (21)
Overall. Rating of Doctor	90 (17)	92* (19)	88 (16)	93* (17)
Would Recommend Doctor.	91 (23)	93* (26)	90 (22)	96* (24)

^†CAHPS measures are scored on a 0–100 possible range, where 100 represent the most positive perceptions of care. Means are case-mix adjusted for race/ethnicity, age, gender, and education. West coast means are also adjusted for self-rated health.

^*Adult and child care adjusted means differ significantly from one another (p < 0.05)

DISCUSSION

Consumer perceptions of health care can be used to provide care that best fit patient’s needs as well as to promote quality improvement.^{4, 15} Traditionally, pediatric providers were considered specialists who are trained to treat medical problems specific to children; however, pediatric providers as a whole have embraced the role of primary care providers – tasked with prevention, health promotion, and providing coordination services to children and their families.¹⁶ In fact, for a majority of children, pediatric providers take on the responsibility of delivering accessible, continuous, comprehensive, family centered, and coordinated care. The pediatric provider generally follows a child longitudinally over time, knows the child’s family and their circumstances. Pediatric providers are also in a position to connect families to network of community-based services that may be beneficial to the child. Previous work has identified factors that parents value in pediatric care, including access to care, continuity, family-centeredness, and coordination.^{10, 17, 18}

Separate adult and child CAHPS surveys are administered because adults and children have different health care needs and may have distinct care-seeking patterns.^{8, 19} Comparing reports and ratings for adult and pediatric care can identify aspects of care that differ between them. We found in general significant differences favoring parent/guardian-assessed pediatric care over adult care on the CAHPS Clinician & Group 1.0 12-month Survey. The one exception was that in the west coast site, office staff helpfulness and courtesy was perceived more positively for adult than pediatric care.

Differences in reports and ratings of adult and pediatric care may be due to the following factors: 1) differences in training and care delivery styles of pediatric and adult providers; 2) differences in response tendencies due to unobservable respondent characteristics; and 3) differences inherent in reporting one’s own care and the care provided to another. The last explanation is unlikely to have had a substantial role in the observed results, since its effects would have likely been the same at both sites.

We adjusted for response tendencies by accounting for known socio-demographic characteristics (i.e., age, gender, race/ethnicity, education level).^20–21 The validity of this approach depends on assumptions that regression coefficients do not differ across adult and pediatric populations that differ substantially in the case-mix adjustors. We found that two-way interactions between the indicator of adult versus pediatric care and the other independent variables were no more significant than expected by chance. Elliott et al.¹⁴ found that spouses evaluated the care of Medicare beneficiaries similarly to what beneficiaries might have themselves, but children of beneficiaries evaluated care less positively. This suggests that proxy effects may be primarily a function of characteristics of the respondent rather than reflecting the distinction between self-rated and other-rated care. Hence, response tendencies of parents rating children’s care may be similar to that for similarly aged adults rating their own care. If so, the assumptions underlying the equivalence of case-mix adjustment for adult care and parent-assessed pediatric care would be met.

This study has several limitations. First, the data analyzed are not representative of all health plans, so that these findings are not generalizable to the universe of plans. Second, while the response rates are lower than ideal, allowing the possibility of nonresponse bias, they are typical of CAHPS clinician/group surveys and are unlikely to have differential effects that would substantially bias comparisons of adult and pediatric care. Furthermore, analyses of similar CAHPS survey data suggest that when case-mix adjustment is employed, there is no evidence that nonresponse weighting further improves estimation.^{22, 23} Another limitation was that parent/caregiver self-reported health was not collected for pediatric cases in the mid-western site. However, even if we assume the worst case for comparison of adult and pediatric care of “excellent” health for every parent/caregiver in that site, pediatric care is still perceived more positively than adult care there. Lastly, while our primary interest is in comparing parent-assessed pediatric care to patient-assessed adult care, and we have reason to believe that case-mix adjusted measurement for adult respondents in the two cases are comparable^13–14, there may be additional interest in future studies that explore the comparability of patient-assessed pediatric care for adolescents with parent-assessed care. It should be noted that even in those instances, parent-assessed care and adolescent patient-assessed care are related but different constructs that may both be of interest. Despite these limitations, our findings extend existing work and identify significant differences in reports and ratings of adult care and pediatric care in CAHPS.

These differences may indicate that the care provided by pediatric providers differs from that provided by adult providers and understanding specific differences may lead to improvement in provider-patient interactions overall. Although the specific differences between two sites may be of little importance, having two sites revealed different magnitude of pediatric versus adult care differences. This study represents an initial step that can lead to exploration of site-specific structural, setting, or other factors associated with variations in CAHPS rating between different service providers or specialty groups.

Appendix: CAHPS Composites and Ratings

Access to Care

• Got appointment for urgent care as soon as needed

• Got appointment for routine care as soon as needed

• Got answers to medical questions during regular office hours when phoned

• Got answers to medical questions after regular office hours when phoned

• Seen within 15 minutes of appointment time

Doctor Communication

• Explained things in a way that was easy to understand

• Listened carefully to you

• Gave easy to understand instructions about taking care of problems

• Showed respect for what you had to say

• Spent enough time with you

Coordination of Care

• Knew important information about your medical history

• Informed and up-to-date on care you got from other doctors

• Office followed up to give you test results

Shared Decision Making: Talked about pros and cons of treatment or health care choice

Office Staff

• Clerks and receptionists as helpful as you thought they should be

• Clerks and receptionists treat you with courtesy and respect

Global rating of the doctor (0–10 response scale)

• Would recommend doctor to family and friends (definitely yes to definitely no)