Whose Experience Is Measured?: A Pilot Study of Patient Satisfaction Demographics in Pediatric Otolaryngology

Abstract

Objectives/Hypothesis

Despite a national emphasis on patient-centered care and cultural competency, minority and low-income children continue to experience disparities in health care quality. Patient satisfaction scores are a core quality indicator. The objective of this study was to evaluate race and insurance-related disparities in parent participation with pediatric otolaryngology satisfaction surveys.

Study Design

Observational analysis of patient satisfaction survey respondents from a tertiary pediatric otolaryngology division.

Methods

Demographics of survey respondents (Press Ganey Medical Practice Survey©) between January and July 2012 were compared to a clinic comparison group using t test and chi-square analyses. Multivariate logistic regression analyses were performed to assess likelihood to complete a survey based on race or insurance status.

Results

A total of 130 survey respondents were compared to 1,251 patients in the comparison group. The mean patient age for which the parent survey was completed was 5.7 years (6.1 years for the comparison group, P =0.18); 59.2% of children were ≤5 years old. Relative to the comparison group, survey respondents were more often white (77.7% vs. 58.1%; P <0.001) and privately insured (84.6% vs. 60.8%; P <0.001). Similarly, after controlling for confounding variables, parents of children who were white (OR 1.8, 95% CI 1.13–2.78, P =0.013) or privately insured (OR 2.9, 95% CI 1.74–4.85, P <0.001) were most likely to complete a survey.

Conclusion

Methods to evaluate satisfaction did not capture the racial or socioeconomic patient distribution within this pediatric division. These findings challenge the validity of applying patient satisfaction scores, as currently measured, to indicate health care quality. Future efforts to measure and improve patient experience should be inclusive of a culturally diverse population.

Level of Evidence

2c.

INTRODUCTION

For over a decade, patient- and family-centered care has been a central theme of national efforts to improve the quality of health care and ameliorate health disparities in the United States. The Institute of Medicine’s vision of 21st century health care placed patient-centered care and equitable care as two of the six guiding aims in health care improvement and identifies patients and their families as the primary arbiters of quality care.^1,2 Patient satisfaction is one essential measure of patient-centeredness and quality care. Patient satisfaction can be divided into two broadly defined categories: components and determinants.^3,4 Satisfaction components relate to provision of care and include interpersonal mannerisms, technical quality, accessibility/convenience, finances, efficacy/outcomes, continuity, physical environment, and availability; whereas determinants relate to the patient’s characteristics and expectations.^3,4 Patient race/ethnicity and socioeconomic status (SES) are patient characteristics that can serve as determinants of patient satisfaction, and their role in understanding and addressing patient satisfaction illustrates the intersection between patient-centeredness and cultural competency. As concepts, patient-centeredness and cultural competency incorporate similar principles: Culturally competent care emphasizes diversity, equity, and the improvement of care for minorities and disadvantaged populations, and patient-centeredness more broadly focuses on quality improvement for all patients.⁵ Within pediatric surgical fields, the disparities literature focuses primarily on variation in disease prevalence and differences in outcomes across demographic patient groups, with relatively little emphasis on disparities in quality improvement and associated interventions to improve care.^6–8

In recent years, there has been increasing national emphasis on patient satisfaction, beginning with reporting of adult inpatient care and more recently extending to the outpatient setting. With the Tax Relief and Health Care Act of 2006 and the Patient Protection and Affordable Care Act (PPACA) of 2010, public reporting extended to the outpatient setting and individual providers.^9,10 However, questions remain regarding the prevalence and nature of disparities in satisfaction, available measurement tools, and associated interventions for the outpatient setting, particularly within pediatric surgical specialties. Traditionally, the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey has been used and is the national standard for evaluating patient-reported experience of care for inpatient adult care.¹¹ Beyond its significance in public reporting, survey responses are heavily weighted in current value-based purchasing models, demonstrating the now inseparable connection between patient care, satisfaction, and reimbursement.

Focusing on pediatric ambulatory surgery settings, tools tailored for children and their families to evaluate care experiences—both inpatient and outpatient—are lacking, and it is unclear how surveys on patient satisfaction will impact reimbursement for pediatric specialists. Furthermore, despite the significance of patient satisfaction scores, little is known about the racial/ethnic or socioeconomic distribution of these scores or the response rates among patients’ families in a pediatric ambulatory setting, much less in a surgical subspecialty such as pediatric otolaryngology. The goal of this pilot study was to evaluate the social demographics, specifically, race and insurance status, and associated disparities in parent participation with pediatric otolaryngology satisfaction surveys.

MATERIALS AND METHODS

Data Source

Families of children obtaining care at a tertiary pediatric otolaryngology clinic receive a standardized survey about their clinic experience. As part of a pilot study, the first months of survey collection were analyzed for this study, specifically those collected from two sites between January and July 2012. The survey vendor was Press Ganey, a for-profit agency employed by our institution to systematically administer, score, and benchmark patient experience surveys. The study utilized the Medical Practice survey as the outpatient satisfaction survey instrument, which parallels themes of the Clinician and Group–CAHPS survey as a tool to assess outpatient care. The survey includes 29 questions regarding access to care, visit processes, nursing, care provider, personal issues, and overall assessment, and is distributed to nearly all patients seen in the pediatric otolaryngology clinic either by mail or electronically—if a valid mailing or e-mail address is provided.¹³ The survey is available in English and Spanish and reads at the fifth to sixth grade level. Patient demographics such as age, gender, race, and insurance status were obtained from the electronic medical record. The study was approved by the Johns Hopkins Institutional Review Board.

Study and comparison groups were drawn from the division of pediatric otolaryngology at The Johns Hopkins Department of Otolaryngology—Head and Neck Surgery. The pediatric otolaryngology practice includes two clinic sites that together encompass diverse social and demographic environments, one within an urban/community setting and the other within a suburban setting. In 1 fiscal year, there are a total of 6,630 visits, across both sites, including new and established patients. The comparison group was acquired from a 3-month time period that included all clinic patients seen in two tertiary pediatric otolaryngology clinics in 2011, outside of the survey period. This comparison group has been used in other studies to demonstrate a representative pediatric population for this particular department, inclusive of clinical diagnoses and patient demographics.¹⁹ For this study, demographic information including age, gender, race, visit type, and insurance provider were obtained.

Data Analysis

An observational, patient-level analysis was performed. Participants were included based on patient age <18 years. Demographic variables were the primary focus. Race was coded as white versus non-white. Insurance status was classified as private (including Tricare) or public (Medical Assistance). Medical assistance has been used as a proxy for low socioeconomic status in prior studies.^12,13 For example, in Maryland children under 19 years old may qualify for Maryland Children’s Health Program (MCHP) if family income is less than 200% Federal Poverty Level or, for example, through Maryland’s Medical Assistance for Families if a family of four’s countable monthly income is a $2,131 or less.

Patient demographics were analyzed using descriptive statistics. Distributions of demographic groups were compared using t test (age) and chi-squared tests (race, insurance, gender, visit type). Multivariate logistic regression was used to calculate unadjusted and adjusted odds ratios (OR) of the likelihood for a patient’s family to complete and return the satisfaction survey based on demographic factors. Demographic variables were tested for interaction terms. A P value of <0.05 was considered significant and was adjusted for multiple comparisons. Stata/IC 10.1 for Windows (StataCorp LP, College Station, TX) was utilized for all variable coding and statistical analyses.

RESULTS

Between January 1, 2012, and July 31, 2012, a total of 130 completed mail and electronic Press Ganey Medical Practice surveys were received. A total of 1,639 surveys were distributed between the two sites between January and July 2012, which equates to an overall survey response rate of 11.5%. The majority of patients’ families, 65.4% (85/130), completed the survey online; 78.8% of those completing the survey electronically were privately insured and 75.3% were white. Between the two sites, approximately half of the surveys came from each site: 47.7% from the suburban setting and 52.3% from the urban setting. Table I demonstrates the demographics of the children whose caregivers completed satisfaction surveys and the demographics of the comparison group, a representative clinic sample that included 1,251 children. The mean age of the children whose parents completed a survey (5.7 years, range 0–17 years) was slightly younger than that of the comparison group (6.1 years, range 0–17 years) but was not significant (P =0.18) (Table I). There was a similar gender distribution across survey respondents and the comparison group (P=0.82) (Table I). White children represented the majority of survey respondents, where 101/130 or 77.7% of the surveys were completed by families of white patients, whereas non-white patients’ families represented only a minority (22.3%) of returned surveys (P<0.001) (Table I). This contrasts to the clinic sample, which is composed of 58.1% (727/1,251) white patients and 41.5% (519/1251) non-white patients (P <0.001).

TABLE I.

Patient Demographics, Survey Respondents versus Clinic Comparison Group.

	Survey Respondents N =130	Comparison Group N =1,251	P value
Mean age (SD), yr	5.7 (4.5)	6.1 (4.4)	0.179
Age 0–5, N (%)	77 (59.2)	672 (53.7)
Age 6–12, N (%)	35 (26.9)	403 (32.2)
Age 13–17, N (%)	18 (13.9)	176 (14.1)
Gender, N (%)
Male	74 (56.9)	699 (55.9)	0.819
Female	56 (43.1)	552 (44.1)
Race, N (%)
White	101 (77.7)	727 (58.1)
Non-White	29 (22.3)	519 (41.5)	<0.001
Insurance, N (%)
Private	110 (84.6)	760 (60.8)
Medical assistance	20 (15.4)	491 (39.2)	<0.001
First clinic visit, N (%) New patient	49 (37.7)	523 (41.8)
Established patient	81 (62.3)	728 (58.2)	<0.001

Differences in insurance status were also noted. The majority of families who completed the survey were privately insured; 84.6% of respondents, compared to 60.8% of the comparison group patients, were privately insured. In contrast, only 15.4% of patients completing the survey had medical assistance, compared to the 39.2% of the comparison group families who were publicly insured (P <0.001). Looking at rates of private insurance by race 93.1% (94/101) of white patients were covered by private insurance, while 55.2% of non-white patients were covered by private insurance (16/29). Overall, we found that survey respondents were more likely to be white and privately insured. Survey respondents were slightly more likely to be established patients. In comparing the percent of new versus established patients represented in the survey population, 41.8% of the comparison group was a new visit compared to 37.7% of survey responders (P <0.001).

Multivariate logistic regression was performed to determine the unadjusted and adjusted odds ratio of parents’ likelihood to complete the satisfaction survey (Table II). Parents of female children and those with older children (age 6–17 years) were not more likely to complete satisfaction surveys as compared to parents with male children and those with the youngest children, respectively. However, parents of white children were more likely to complete the survey (OR 2.51, 95% CI 1.63–3.85, P <0.001). With regard to insurance status, parents of children who were privately insured were more likely to complete the survey (OR 3.55, 95% CI 2.18–5.80, P <0.001). When adjusting for race, gender, and insurance status, the adjusted ORs were similar. Parents of children who were white were 1.77 times more likely to respond to the survey (CI 1.13–2.78, P =0.013). Parents of children who were privately insured were 2.90 times more likely to respond to the survey (CI 1.73–4.85, P <0.001). In testing for interaction between race, gender, and insurance status, the variables were not found to be codependent with no statistically significant interaction terms.

TABLE II.

Multivariate Analysis of Association of Demographics With Survey Participation.

Patient Demographics	Odds Ratio Unadjusted	Odds Ratio Adjusted	95% CI Adjusted	P Value Adjusted
Female	1.0	1.0	0.7–1.4	0.845
Patient age 6–12	0.8	0.8	0.5–1.2	0.254
Patient age 13–17	0.9	0.8	0.5–1.4	0.472
White	2.5	1.8	1.1–2.8	0.013
Private Insurance	3.6	2.9	1.7–4.9	<0.001

CI =confidence interval.

DISCUSSION

With increasing emphasis on patient satisfaction as a measure of patient-centeredness and health care quality, the number of studies evaluating satisfaction within pediatric surgical care is also increasing.^14–16 However, limited information exists on how these satisfaction surveys represent a culturally diverse population, particularly including traditionally underrepresented groups such as racial/ethnic minorities and low-income families. This pilot study demonstrates the first report of disparities in patient satisfaction survey participation in a pediatric otolaryngology outpatient clinic setting. While survey respondents were not different from the clinic comparison group in terms of patient age or gender, white and privately insured patients disproportionately represent the study population.

Results of the current study align closely with previously published data on adults who are less likely to participate in satisfaction surveys, such as CAHPS. Zaslavsky et al. and Elliott et al. explored predictors of nonresponse to the CAHPS Hospital Survey and the Medicare Managed Care version among adult patients and found racial/ethnic minorities to have higher nonresponse rates as do individuals who reside in communities with larger percentage of minorities and residents insured by medical assistance.^9,10 Zaslavsky et al. found response rates of 72.1% for whites, 53.0% for blacks, 55.6% for Hispanics, and 61.6% for other to mailed CAHPS surveys when good contact information was available–rates higher than were found in our study population.¹¹ These authors demonstrated that in communities with lower education levels, higher percent of residents on public assistance, and larger proportion of minority residents, the proportion of unreliable contact information increased.¹¹ However, even when good contact information was available, response rates were lower from these patient subgroups.^11,12

Survey nonresponse has also been associated with individuals who express higher levels of alienation and disaffiliation, which may be representative of higher rates of civic disengagement and social isolation in underserved communities.¹⁶ Moreover, higher nonresponse rates have been found among patients who had less favorable views of their care.¹⁴ This finding has important repercussions on whose satisfaction is measured with current methods, and how we identify and address potential disparities in patient satisfaction.^9,10 For pediatric populations, this methodologic challenge is complicated by historically low response rates. Potential methods of addressing the threat of nonresponse bias to the validity of the survey results may involve statistical correction, such as “risk-adjustment,” or interventions to increase response rates in targeted populations through techniques such as advanced letters, participation incentives, and patient callbacks.¹⁷ Others have demonstrated that nonresponse bias has relatively little impact on overall patient satisfaction, with relatively small differences in satisfaction scores between adult respondents and nonrespondents to an inpatient satisfaction survey.¹⁷

Opportunities for further investigation of the role of patient, family, and community characteristics in evaluating satisfaction in the ambulatory pediatric surgical setting, include comparing survey responders to the clinic comparison group based on demographics such as parental education level, sex, and age, as well as the child’s zip code and demographics associated with the zip code (i.e., racial/ethnic minorities, average income, and proportion receiving public assistance). Although this additional information would assist in better characterizing the population and in the analysis of satisfaction data, the greatest implication would come in being able to identify how our clinics may be better able to capture a more accurate representation of how all patients and families experience care, weigh available data appropriately, and begin to address potential disparities in the quality of care.

The role of race and insurance status in the assessment of quality health care must also continue to be elucidated. Studies demonstrate that factors such as social concordance between physician and patient impact patients’ perceptions of quality and satisfaction with their care, where lower rates of social concordance (i.e., when patient and provider share fewer social characteristics such as gender, race, and education level) correlate with lower rates of satisfaction.¹⁸ With evidence that race and ethnicity impact diagnosis and treatment of common pediatric otolaryngologic diseases, such as otitis media and sleep disordered breathing, it is paramount we understand how social determinants, such as gender, race, ethnicity, insurance status, and health literacy, impact patient satisfaction with care in the pediatric otolaryngology and pediatric ambulatory surgery settings.^14,19 Finally, although the majority of research on patient satisfaction stems from surveys of adult populations, which aligns with the current study’s findings, differences exist between adult and pediatric reported satisfaction within otolaryngology. One study captured lower satisfaction scores among pediatric surveys as compared to adults on almost every parameter, important reminders of the way in which the two populations differ and dedicated pediatric surveys are critical.¹⁶ These differences may extend from those between pediatric and adult care, where pediatric care is dominated by more frequent visits for routine care and more minor ailments, as provided by multiple practioners in various settings and reviewed by survey respondents who do not directly receive the care themselves.²⁰

This study introduces the impact of race and insurance status on patient satisfaction and the limitations of how satisfaction is measured within pediatric ambulatory surgical care. The study is limited by its sample size, both of the study population and the clinic comparison group. The small sample size restricts the generalizability of the results and implications. Unfortunately, this small response rate may be generally reflective of mailed survey response rates in the pediatric population. The study represents patients seen within only one institution’s pediatric otolaryngology division. There may have been seasonal differences between patients and families who sought care. Furthermore, no information on clinical health status or outcomes, which may influence likelihood to participate in the survey, was included in this analysis. The survey implemented was not specific for pediatric or surgical care; therefore, it may have appeared less relevant to parents. Likewise, given the paucity of research on the measurement of patient satisfaction within the pediatric ambulatory surgical setting, it is unknown whether these distribution methods were optimal for this environment. These methods, however, including the distribution of this particular survey, are shared by many large private and academic institutions across the country. Due to the small sample size of respondents, we were unable to analyze differences in satisfaction scores between populations, which represents an opportunity for further study. Additional satisfaction determinants should be collected and analyzed, including the gender of the caregiver completing the survey, community characteristics represented by the patient’s zip code, as well as differences in response based on survey format and language. Future studies will explore the impact of these determinants on patient satisfaction in a pediatric otolaryngology ambulatory setting.

CONCLUSION

With growing emphasis on patient satisfaction as an essential measure of health care quality and patient-centered care, patient satisfaction scores and the care experience measured by satisfaction surveys assume new significance. This study is the first to demonstrate racial and socioeconomic differences in patient satisfaction survey participation in the pediatric otolaryngology ambulatory setting. Findings from this study potentially challenge the validity of these surveys, as currently distributed, in measuring a culturally diverse care experience in pediatric surgical care.