The Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) Cultural Competence (CC) Item Set

Abstract

Background

There is a need for reliable and valid measures of cultural competence from the patient’s perspective.

Objective

This paper evaluates the reliability and validity of the Consumer Assessments of Healthcare Providers and Systems (CAHPS^®) Cultural Competence (CC) item set.

Research Design

Using 2008 survey data, we assessed the internal consistency of the CAHPS CC scales using Cronbach alphas, and examined the validity of the measures using exploratory and confirmatory factor analysis, multitrait scaling analysis, and regression analysis.

Subjects

A random stratified sample (based on race/ethnicity and language) of 991 enrollees, less than 65 years old, from two Medicaid managed care plans in California and New York.

Measures

CAHPS CC item set after excluding screener items and ratings.

Results

Confirmatory factor analysis (CFI= 0.98; TLI= 0.98; RMSEA= 0.06) provided support for a seven-factor structure: Doctor Communication-Positive Behaviors; Doctor Communication-Negative Behaviors; Doctor Communication-Health Promotion; Doctor Communication-Alternative Medicine; Shared Decision Making; Equitable Treatment; and Trust. Item--total correlations (corrected for item overlap) for the 7 scales exceeded 0.40. Exploratory factor analysis showed support for one additional factor: Access to Interpreter Services. Internal consistency reliability estimates ranged from 0.58 (Alternative Medicine) to 0.92 (Positive Behaviors), and was 0.70 or higher for four of the eight composites. All composites were positively and significantly associated with the overall doctor rating.

Conclusions

The CAHPS CC 26-item set demonstrates adequate measurement properties, and can be used as a supplemental item set to the CAHPS Clinician and Group Surveys in assessing culturally competent care from the patient’s perspective.

Among the strategies that have been advocated for reducing racial/ethnic differences in patient experiences is the provision of “culturally competent” care.^{1, 2} The National Quality Forum (NQF) (p. 2) recently defined cultural competency as the “ongoing capacity of healthcare systems, organizations, and professionals to provide for diverse patient populations high-quality care that is safe, patient and family centered, evidence based, and equitable.”³

Based on the NQF definition, patient-centered care is one of the major elements of cultural competency. The Institute of Medicine (p. 3)⁴ defined patient-centered care as “care that is respectful and responsive to individual patient preferences, needs, and values…” McWhinney⁵ described patient-centered care as being able to see through the patient’s eyes. Thus, the patient’s perspective on the care that he or she receives is an essential barometer of culturally competence care.

The Consumer Assessment of Healthcare Providers and Systems (CAHPS®) project has resulted in a set of standardized survey instruments that can be used to collect reliable information from patients about the care they have received. These evaluations provide important information about how well providers meet the needs of the people they serve.⁶ For example, the CAHPS Clinician and Group (C&G) Surveys assess patients’ experiences with health care providers and staff in doctor’s offices. The CAHPS C&G core survey contains 13 items measuring 3 domains of performance: timeliness of care, provider communication, and staff helpfulness. In addition it has one global rating for provider.⁷

CAHPS data have been used to assess racial/ethnic and language differences in patient experiences with care.^8-12 However, there are concerns that the CAHPS instrument does not fully capture domains of care of particular relevance to diverse populations, such as trust, perceived discrimination, shared decision making, and access to language services.¹ To address this gap, the CAHPS team developed an item set to assess aspects of cultural competency not adequately addressed in the existing CAHPS surveys, which could serve as a supplemental item set to the CAHPS Clinician & Group surveys. The purpose of this study is to evaluate the internal consistency reliability and validity of the CAHPS Cultural Competence (CC) item set. The psychometric analysis by survey language (English/Spanish) is reported elsewhere.¹³

Conceptual Framework

Guided by Bethell et al.’s¹⁴ conceptual model of measuring health care quality among diverse populations and a comprehensive literature review on diverse populations in the U.S., we developed a framework for obtaining the patient’s perspective on culturally competent care.¹ In this framework, health care is experienced by the patient in the context of interactions with providers within the health care system. Therefore, there are three factors that affect the quality of care for diverse populations: patient factors, provider factors, and health care system factors. The framework focuses on the areas where the three factors overlap (Figure 1). The first two domains reflect interactions between the patient and the provider: 1) Patient-provider communication; 2) Respect for patient preferences/ shared decision-making. The other four domains include patient and provider interactions, but also include interactions with other staff and the health care system overall: 3) Experiences leading to trust or distrust; 4) Experiences of discrimination; 5) Health literacy strategies; and 6) Language services. These six domains are best measured by patient assessments as opposed to organizational or provider assessments. There are other domains of quality care that are important, such as Access (ability to get timely care) or Coordination of Care (between different providers and health care settings.) However, because these domains of quality care have been extensively examined as part of patient-centered care, we chose not to include them within the scope of this study. Furthermore health literacy was not included as one of the domains of CAHPS CC item set because it was the focus of a separate project (CAHPS Health Literacy item set).¹⁵

Figure 1. Conceptual Framework of Culturally Competent Care From the Patient’s Perspective.

Source: Ngo-Metzger, Q., Telfair, J., Sorkin, D., Weidmer, B., Weech-Maldonado, R., Hurtado, M., and Hays, R.D. Cultural Competency and Quality of Care: Obtaining the Patient’s Perspective. NY: Commonwealth Fund Report; 2006.

We summarize below our literature review on five of the cultural competence domains. We focused specifically on empirical studies that included diverse populations (African Americans, Hispanics/Latinos, Asians and Pacific Islanders, or American Indians). We also only included studies conducted in the United States, as the healthcare systems and experiences in other countries may be very different and thus, not generalizable to the US.

Patient-Provider Communication

Communication in the medical interaction plays a central role in decisions about subsequent interventions, and can influence patient adherence, satisfaction with care, and health outcomes¹⁶. Some racial/ethnic groups and those of lower socioeconomic status are more likely to report poor communication with their physicians^17,18 Providers’ non-verbal and interpersonal communication behaviors have been found to be particularly important for diverse patient populations. For example, Hurtado et al.¹⁹ found that empathy and establishing rapport (as opposed to just providing health information) were more important to minority patients than to White patients. Similarly, African American, Hispanic, and Asian patients were found to rate the provider’s display of “concern, courtesy, and respect” as the most important factor in the interaction.^{20, 21} Finally, studies have found that listening and spending adequate time are especially important for Asian²² and Hispanic patients.²³

While the CAHPS C&G surveys include provider communication as a core measure, this measure focuses on the behavioral aspects rather than the content of communication. One important content area is communication about complementary and alternative medicine (CAM).¹ CAM refers to diverse practices and products that are not currently considered part of conventional medicine.²⁴ An estimated 26 percent of African Americans, 28 percent of Hispanics, 36 percent of non-Hispanic whites, and 43 percent of Asian Americans use CAM.²⁵ It is important for patients to discuss their use of CAM with their medical practitioners. Some therapies, such as herbal or vitamin therapies, may cause adverse events or interfere with medical regimens. Furthermore, knowledge of patients’ CAM practices can provide valuable insight into patients’ values, lifestyles, and health beliefs, which may, in turn; assist practitioners in providing optimum care.^{26, 27} Yet, in a national survey of U.S. adults, 70 percent of patients who used CAM reported that their providers did not discuss CAM use with them.^{24, 28}

Another content area of particular importance to diverse populations is communication about health promotion. Racial/ethnic minorities suffer disproportionately from chronic diseases, such as diabetes and heart disease.²⁹ As such communication about diet and exercise are particularly important for prevention in promoting a healthy life style among diverse patient populations. Similarly, racism and discrimination places minorities at higher risk for depression and anxiety.³⁰ Yet minorities are less likely to receive treatment for depression than Whites.³¹ Communication about stress and depression with diverse populations can help reduce disparities in care.

Respect for Patient Preferences/Shared Decision Making

The Institute of Medicine encourages providers to respect patients’ preferences and promote their active participation in clinical decision-making to the extent that patients’ feel comfortable and are willing to take part.⁴ Patients may participate in their care in a variety of ways, which includes having meaningful dialogue about their preferences, knowing all the available options, and making final decisions about treatment.

A report by the Agency for Healthcare Research and Quality (p. 122)¹⁷ noted that “Blacks, Asians, Hispanics, and low income populations are more likely to feel disenfranchised in the decision-making process.” Compared with Whites, African Americans and Asians were more likely to report under-involvement in the health care decision-making process (e.g. 22% of Whites, versus 27% of African Americans and 42% of Asians reported that they were “not as involved as they would like to have been”). Similarly, Hispanics were more likely than non-Hispanic Whites to report feeling disenfranchised (34% vs. 21%), and low-income populations were more likely than higher-income populations to report this (30% vs. 20%).¹⁸

Experiences Leading to Trust

Patient’s trust is an important element of the health care encounter. Thom et al.³² found that patients with lower levels of trust were less likely to adhere to their physician’s advice, and were more likely to report not receiving the services they requested or needed. Similarly, patients with lower levels of trust have reported lower levels of satisfaction with the patient-provider relationship.³²

Several studies have observed lower levels of patient trust among racial and ethnic minorities.^33-36 Using data from a national sample of adults, Hunt et al. (2005)³³ reported that African Americans and Latinos were less trusting and less satisfied with their physicians than Whites. Similarly, LaVeist et al.³⁶ found that African Americans were significantly more likely than Whites to report mistrust of the medical system, and those who reported greater mistrust of the medical care system were less satisfied with their care.

Experiences of Discrimination

According to the Institute of Medicine,³⁷ discrimination is differential treatment based on race, ethnicity, gender or other individual attribute. Racial bias or discrimination in the practice and delivery of healthcare may be at least partly responsible for the observed racial and ethnic health disparities.³⁸ Prior research has found that perceived discrimination is associated with various negative health outcomes. Perceived discrimination has also been found to have a negative effect on satisfaction with care.^39-41 For example, a study of the California Health Interview survey found that patient perceptions of discrimination based on race or ethnicity were associated with lower ratings of health care quality.⁴⁰ Similarly, Hausmann et al.⁴¹ found that perceived racism was associated with low patient ratings of provider warmth/respectfulness, and ease of communication. Finally, Weech-Maldonado et al. found that Medicaid beneficiaries in Florida who perceived discrimination based on race/ethnicity and insurance reported lower CAHPS scores.⁴²

Language Services

According to the 2000 census, approximately 47 million people in the U.S. speak a language other than English at home and over 21 million are Limited English Proficient (LEP)⁴³ Previous research has shown that non-English patients have worse access to care ^44-47 and report lower ratings of care than English-speaking patients.^{9, 10, 12, 48, 49}

The limited supply of bilingual providers has led health care organizations to use interpreter services to bridge language gaps. When examining the impact of language services, it is important to distinguish between professional interpreters and ad-hoc interpreters. Ad-hoc interpreters are individuals whose primary job function is something other than interpretation and includes clinic staff as well as the patient’s family members and friends.⁵⁰ On the other hand, professional interpreters receive specialized training on medical interpretation and their main function is interpretation.⁵⁰ This includes in-person interpretation, third party interpretation and remote, third party interpretation using technology (e.g. using telephone or video). Professional interpreters have been shown to reduce barriers to care among LEP patients.^{51, 52}

CAHPS CC Survey

The CAHPS CC was designed to assess five domains of cultural competence: Patient-provider communication; Respect for patient preferences/ shared decision-making; Experiences leading to trust or distrust; Experiences of discrimination; and Language services. The survey development involved five steps: 1) Evaluating existing CAHPS surveys to identify existing items that addressed the domains of interest; 2) Conducting a literature review in order to identify existing instruments or item sets that had been used in the past to collect data on cultural competency from the patient’s perspective; 3) Placing a Federal Register notice with a call for measures; 4) Reviewing and adapting existing measures in the public domain; and 5) Writing additional survey items as needed for each of the proposed cultural competency domains.

Each of these steps was conducted collaboratively by members of the CAHPS Cultural Comparability Team, a sub-team of the CAHPS Consortium that includes health researchers experienced in survey development and testing, psychometrics, and translation and cultural adaptation of survey measures. The review of the literature and the various CAHPS surveys yielded over 90 survey items related to the domains of interest for the CAHPS CC. This included survey items that could be adapted from other CAHPS Surveys, such as the CAHPS In-Center Hemodialysis Survey and the CAHPS American Indian Survey, as well as various surveys that are in the public domain (for example, the National Health Interview Survey, the California Health Interview Survey, the Child Hospitalization Communication, Quality and Safety Survey, Hablamos Juntos, the Commonwealth Fund’s Survey on Disparities in Quality of Healthcare, and the Primary Care Assessment Survey). Only one measure submission was received in response to the Federal Register Notice. The survey development team mapped each of the items that was identified through the literature and survey review to the domains of interest and adapted measures that best captured the domains of interest by rewording the items so that they would include a 12 –month reference period, the same language in referencing the provider as utilized in other CAHPS measures, and response options that were consistent with the other CAHPS measures.

The CAHPS CC item set was translated into Spanish using the Agency for Healthcare Research and Quality’s (AHRQ) Guidelines for Translating CAHPS Surveys.⁵³ First, two independent American Translators Association’s (ATA) certified translators conducted two forward translations of the survey into Spanish. Translators were instructed to aim for a 6^th grade reading level or lower. Second, a committee formed by the two translators and members of the CAHPS Cultural Comparability team reviewed the translations and reconciled the differences. The review committee had representation from different Spanish subgroups in the US, such as Mexican, Puerto Rican and South American.

Next we conducted 18 semi-structured cognitive interviews (9 in Spanish and 9 in English) with scripted probes in Los Angeles, Boston, and Chapel Hill, North Carolina. Recruitment for the cognitive testing was aimed at getting a mix of respondents in terms of age, race/ethnicity, gender, and level of education. Based on the findings of the cognitive interviews, the instrument was revised and a final 44-item set was available for field testing.

Methods

Field Test

A field test of the CAHPS CC survey was conducted with a stratified random sample of 6,000 adult (18 years and older) enrollees of two Medicaid managed care plans, one in New York (3,200) and the other in California (2,800) in 2008. We chose New York and California for the field test given the diversity of their population. The stratified sample based on race/ethnicity and preferred language was drawn using the health plans’ administrative data: 1,200 non-Hispanic White English speakers, 1,200 non-Hispanic Black English speakers, 900 Hispanic English speakers, 900 Hispanic Spanish speakers, 900 non-Hispanic Asian English speakers, and 900 non-Hispanic Asians with a preferred language other than English. Health plan enrollees 65 years and older were not included in the sample, since this population is generally dually eligible (Medicare and Medicaid) and their health care experiences may be different than those with Medicaid as their primary insurer.

The survey consisted of a 2-wave mailing with follow-up telephone interview of non-respondents. The first mailing included an English survey and a cover letter in English and Spanish. The letter directed Spanish speakers to call an 800 number to request a copy of the survey materials in Spanish. Four weeks after the initial mailing, a second survey packet was mailed to non-respondents. Telephone follow ups in English and Spanish started 2 weeks after mailing the second survey packet. A monetary incentive of $10 was offered to non-respondents after the second call attempt. A 26% response rate (n=1,380) was achieved with the multi-pronged approach. After excluding individuals that did not have a personal doctor or a doctor visit during the last 12 months, the final analytic sample consisted of 991 respondents.

Measures and Analysis Plan

The analysis focuses on the CAHPS CC 27-item set after excluding screener items and ratings. Psychometric analysis was used to assess the internal consistency reliability and validity of the CAHPS CC scales. First, exploratory factor analysis (EFA) with varimax rotation in both SAS and Mplus was used. ⁵⁴ The number of factors retained was determined by a combination of criteria: (1) the roots criterion of selecting factors with eigenvalues to be greater than 1; (2) the scree plot to examine the point at which the plot of eigenvalues begins to level off; and (3) the interpretability of factors based on the conceptual framework.⁵⁵ The language services items were factor analyzed separately, since the items only applied to a small subset of respondents, those with limited English proficiency (LEP) and who had used an interpreter (N= 76). LEP respondents were identified as those for which English was not their primary language and who spoke English not well or not at all.

Second, confirmatory factor analysis (CFA) was conducted of the CAHPS CC scales derived from the exploratory factor analysis using Mplus We used fit index levels for Root Mean Square Error or Approximation (RMSEA), Comparative Fit Index (CFI), and Tucker Lewis Index (TLI) as identified by the literature.⁵⁶ Mplus accounted for the ordinal nature of the response options.⁵⁷

Third, multitrait scaling analysis was used to assess item discrimination across scales.⁵⁸ Multitrait scaling generalizes multitrait-multimethod analytic methods from the trait level to the item level. In doing so, it allows the user to investigate convergent and divergent validity at the item level.

Fourth, we used Cronbach’s alpha to estimate scale internal consistency reliability. Cronbach alphas were estimated for the overall sample.

Finally, the convergent validity of the CAHPS CC scales was assessed. Regression analyses were conducted to examine the associations between the CAHPS CC composite scores and the overall doctor rating item, controlling for race/ethnicity, gender, age, education, self-rated health status, and language of the survey (Spanish vs. English). Separate regression models were run for each CAHPS CC composite. The CAHPS global rating (0-10) for personal doctor, where 0= worst possible doctor and 10= best possible doctor, was linearly transformed to a 0-100 possible range (i.e., multiplied by 10). The CAHPS CC composite scores were calculated by linearly transforming items to a 0-100 possible (with a higher score representing more favorable scores), and then averaging items within each composite. Negatively worded items were rescored to show a positive scale. Self-reported race/ethnicity consisted of non-Hispanic White, Hispanics, non-Hispanic Black, non-Hispanic Asian, and Other. The gender item asked whether the respondent is male or female. Age consisted of five categories: 18-24, 25-34, 35-44, 45-54, and 55-64. Education consisted of five categories: 8^th grade or less, some high school but did not graduate, high school graduate or GED, some college or 2-year degree, 4-year college graduate or higher. Self-reported health status asked respondent to rate their overall health: excellent, very good, good, fair, and poor. Finally, Spanish survey is a dummy variable that identifies whether or not the survey was completed or administered in Spanish.

Results

Descriptive statistics for the analytic sample are presented on Table 1. Hispanics were the largest racial/ethnic group (34%) in the sample, while non-Hispanic Whites and Blacks had the smallest representation with 15% each. Approximately 30% of respondents rated their health as fair or poor. The majority of the sample was female (67%) and younger than 44 years old (52%). A significant proportion of respondents (31%) had less than high school graduation or GED. Finally, 12% of respondents completed a Spanish survey.

Table 1.

Characteristics of Survey Respondents (N=991)

Variable	%
Race/Ethnicity
Hispanic	34.2
Non-Hispanic White	14.7
Non-Hispanic Black	14.9
Non-Hispanic Asian	17.5
Other	18.4
Missing	0.3
Self-rated Health
Excellent	11.0
Very Good	17.9
Good	32.5
Fair	22.9
Poor	7.0
Missing	8.8
Age
18-24	14.9
25-34	15.6
35-44	21.8
45-54	24.2
55-64	15.5
Missing	7.9
Gender
Female	67.1
Male	25.2
Missing	7.7
Education
8th grade or less	13.1
Some high school	18.3
High school graduate or GED	26.9
Some college or 2-year degree	24.3
4-year college graduate or more	8.4
Missing	9.0
Spanish Survey	11.8

A comparison of respondents and non-respondents based on gender, age, race/ethnicity, primary language, and health plan affiliation showed only a few significant differences. Respondents were more likely to be White (24% versus 20%) and older (39 versus 36 years, and less likely to be Black (18% versus 22%) compared to non-respondents. There were no significant differences between respondents and non-respondents in terms of gender, Hispanic or Asian ethnicity, preferred language, or health plan affiliation.

A seven-factor structure was obtained based on EFA (eigenvalues > 1 and scree plot) and guided by the conceptual framework: Doctor Communication-Positive Behaviors (5 items); Doctor Communication-Negative Behaviors (3 items); Doctor Communication-Health Promotion (4 items); Doctor Communication-Alternative Medicine (2 items); Shared Decision Making (2 items); Equitable Treatment (2 items); and Trust (5 items). CFA (CFI= 0.98; TLI= 0.98; RMSEA= 0.06) provided support for the seven-factor structure: Item--total correlations (corrected for item overlap) were higher than 0.40 for all the multi-item scales (Table 2). The eight domains of the CAHPS CC match closely those of the initial conceptual framework, except that Provider Communication is further divided into four different scales based on both behavioral aspects (positive and negative behaviors) and content (health promotion and alternative medicine) of communication.

Table 2.

CAHPS Cultural Competence Item-Scale Correlation, Corrected for Item Overlap

Items	Doctor Communication-Positive Behaviors	Doctor Communication-Negative Behaviors	Doctor Communication-Health Promotion	Doctor Communication-Alternative Medicine	Shared decision making	Equitable treatment	Trust
Explain things in a way that was easy to understand	0.75*	-0.47	0.37	0.12	0.29	-0.30	0.66
Listen carefully to you	0.79*	-0.53	0.38	0.12	0.32	-0.31	0.71
Spend enough time with you	0.77*	-0.49	0.38	0.11	0.36	-0.29	0.70
Show respect for what you had to say	0.73*	-0.57	0.31	0.15	0.36	-0.34	0.69
Give you easy to understand instructions	0.49*	-0.32	0.40	0.11	0.20	-0.19	0.44
Interrupt you when you were talking	-0.40	0.54*	-0.13	-0.01	-0.06	0.34	-0.36
Talk too fast when talking with you	-0.46	0.57*	-0.18	-0.07	-0.19	0.33	-0.41
Use a condescending, sarcastic, or rude tone	-0.49	0.55*	-0.18	-0.06	-0.21	0.34	-0.48
Talk about a healthy diet and healthy eating habits	0.39	-0.16	0.61*	0.18	0.23	-0.15	0.45
Talk about the exercise or physical activity	0.41	-0.19	0.62*	0.18	0.16	-0.12	0.46
Talk about things in your life that worry you or cause you stress?	0.28	-0.10	0.51*	0.24	0.19	-0.02	0.32
Ask you whether there was a period of time when you felt sad, empty or depressed	0.31	-0.14	0.47*	0.23	0.29	-0.04	0.31
Asked you if you have used these other people to help with an illness or to stay healthy (for example, acupuncturist or herbalist)	0.09	-0.04	0.19	0.40*	0.11	0.02	0.11
Asked you if you used natural herbs	0.15	-0.06	0.25	0.40*	0.16	-0.03	0.17
Talk with you about the pros and cons of each choice for your treatment	0.37	-0.12	0.28	0.11	0.51*	-0.22	0.39
Ask which choice you thought was best for you	0.29	-0.19	0.21	0.17	0.51*	-0.16	0.35
Treated unfairly at this doctor’s office because of your race or ethnicity	-0.25	0.34	-0.09	0.02	-0.15	0.49*	-0.24
Treated unfairly at this doctor’s office because of the type of health insurance	-0.33	0.38	-0.10	-0.04	-0.23	0.49*	-0.34
Feel you can tell this doctor anything	0.51	-0.29	0.41	0.10	0.21	-0.17	0.57*
Trust this doctor with your medical care	0.66	-0.50	0.42	0.12	0.30	-0.30	0.78*
Doctor always tells you the truth about your health	0.56	-0.43	0.34	0.13	0.23	-0.29	0.61*
Doctor cares as much as you do about your health	0.64	-0.44	0.45	0.16	0.41	-0.26	0.77*
Doctor really cared about you as a person	0.71	-0.47	0.44	0.16	0.40	-0.33	0.71*

Exploratory factor analysis (eigenvalue > 1) of the language services items (4 items) provided support for a one-factor structure for Access to Interpreter Services. Factor loadings were higher than 0.40 except for one item (how often did the interpreter you had most often at this doctor’s office treat you with courtesy and respect). This item was dropped from the Access to Interpreter Services resulting in a 3-item scale: 1) Did you use friends or family members as interpreters because there was no other interpreter available at this doctor’s office?; 2) How often did your visit with this doctor start late because you had to wait for an interpreter?; and 3) Was there any time when you needed an interpreter and did not get one at this doctor’s office?

Means, standard deviations, and Cronbach’s alphas for each of the multi-item scales are shown on Table 3. The lowest CAHPS CC scores were observed for Doctor Communication-Alternative Medicine (Mean= 12.1), Doctor Communication- Health Promotion (Mean= 49.1) and Access to Interpreter Services (Mean= 76). Internal consistency reliability estimates were 0.60 or higher for seven of the eight composites, and was 0.70 or higher for four of the seven composites. The two-item Doctor Communication-Alternative Medicine scale had the lowest internal consistency reliability of 0.58.

Table 3.

Descriptive Statistics and Internal Consistency Reliability Estimates (Alphas) for the CAHPS Cultural Competence Scales

CAHPS CC Composites	N	Number of Items	Mean	S.D.	Alpha
Doctor Communication-Positive Behaviors	953	5	82.9	22.5	0.92
Doctor Communication-Negative Behaviors	947	3	89.7	18.4	0.70
Doctor Communication-Health Promotion	945	4	49.1	34.9	0.76
Doctor Communication-Alternative Medicine	925	2	12.1	27.2	0.58
Shared Decision Making1	384	2	80.1	35.0	0.69
Equitable Treatment	910	2	92.5	18.3	0.64
Trust	912	5	78.2	25.5	0.89
Access to Interpreter Services2	76	3	75.9	32.7	0.66

¹Screener item: In the last 12 months, did this doctor tell you there was more than one choice for your treatment or health care?

²Applies only to respondents with limited English proficiency (LEP) and who had used an interpreter.

All CAHPS CC composites were positively and significantly associated with the overall doctor rating (Table 4). The largest unique association was observed for Doctor Communication-Positive Behaviors (B = 0.71) with a 10 point increase (p < 0.001) in this composite resulting in a 7.1 point increase in doctor rating (0-100 scale). The smallest effect was observed for Access to Interpreter Services (B =0.08) with a 10 point increase (p < 0.10) resulting in a 0.8 point increase in doctor rating. The relatively small effect of Access to Interpreter Services may be due to the small sample size for this regression, since this scale was only applicable to those with limited English proficiency (LEP) and who had used an interpreter (n=76).

Table 4.

Multiple Regression Results of CAHPS CC Composites with Overall Doctor Rating¹ (N=991)²

CAHPS CC Composites	B	SE	R2
Doctor Communication-Positive Behaviors	0.71****	0.02	0.62
Doctor Communication-Negative Behaviors	0.64****	0.03	0.36
Doctor Communication-Health Promotion	0.25****	0.02	0.22
Doctor Communication-Alternative Medicine	0.12****	0.03	0.09
Shared Decision Making	0.15****	0.02	0.15
Equitable Treatment	0.36****	0.04	0.15
Trust	0.60****	0.02	0.56
Access to Interpreter Services3	0.08*	0.05	0.20

^*p < 0.10;

^**p < 0.05;

^***p < 0.01;

^****p < 0.001

¹Overall doctor rating: Mean (83.8), SD (20.9)

²After adjusting for race/ethnicity, gender, age, education, self-reported health, and Spanish survey.

³Applies only to respondents with limited English proficiency (LEP) and who had used an interpreter (N= 76)

Conclusions

Among the strategies that have been advocated for reducing racial/ethnic differences in patient experiences with care is the provision of “culturally competent” medical care. This study provides support for the psychometric properties of the CAHPS CC item set in general, especially for the English survey. Our results suggest that the 26-item set measures eight separate domains (Doctor Communication-Positive Behaviors (Provider Communication core measure in the CAHPS C&G Survey); Doctor Communication-Negative Behaviors; Doctor Communication-Health Promotion; Doctor Communication-Alternative Medicine; Shared Decision Making; Equitable Treatment; Trust; and Access to Interpreter Services) rather than a single “cultural competence” construct (Appendix 1). Cultural competence is a multi-dimensional concept. Creating individual scales based on these items that correspond to each domain provides a rich set of items that can be used to supplement the CAHPS Health Plan and Clinician and Group survey instruments.⁵⁹

There is the possibility that the multiple factors we observe may reflect “nuisance” factors rather than substantively meaningful factors. A bi-factor analysis can be used to explore whether a multidimensional model or unidimensional (bi-factor) model best represents the measurement structure of the data. Bifactor models posit that a unidimensional factor (e.g., cultural competence) accounts for the covariance among item responses, but that one or more “nuisance” or “group” factors also exist. Group factors can result from patterns in survey questions, such as common content (e.g., repeated questions about a similar topic) or common methodology (e.g., questions with similar stems). A multidimensional factor structure consists of meaningful and separate factors. If the bifactor model fits better than the multidimensional model, one would generally choose the bifactor model over the multidimensional model. In a separate study by Carle and Weech-Maldonado,⁶⁰ we use bi-factor analysis to provide support for the multidimensional model and the use of separate CAHPS CC scales.

The lowest CAHPS CC mean scores were observed for Doctor Communication-Alternative Medicine, Doctor Communication- Health Promotion, and Access to Interpreter Services. These are domains of particular relevance to racial/ethnic minorities. Doctor communication on alternative medicine and health promotion is crucial when caring for diverse populations. Racial/ethnic minorities have greater use of alternative medicine, and tend to engage less in health promotion behaviors. Access to interpreter services is important for patient communication particularly as the population becomes more linguistically diverse. Policymakers and health plans should incentivize providers to engage in these types of behaviors.

While our results suggest that all CAHPS CC measures are positively associated with CAHPS doctor ratings, there are three scales that are most strongly associated with doctor ratings: Provider Communication- Positive Behaviors, Provider Communication-Negative Behaviors, and Trust. Health care organizations wanting to improve their CAHPS ratings can implement quality improvement activities to address CAHPS CC domains, with a focus on the behavioral aspects of communication and trust.

The study presents several limitations. First, the study was limited to a Medicaid managed care population. However, a recent study by Stern and colleagues tested the CAHPS CC with an uninsured/underinsured patient population with type 2 diabetes, and provided support for the model reported here.⁶¹ Further research is needed testing these measures with other insured populations such as Medicare and commercial. Second, due to resource limitations, the survey was only translated into Spanish. Translation into other languages as well as research examining the measurement equivalence of CAHPS CC in other languages is needed. Third, we did not have any provider identifiers. Therefore, we were not able to assess the inter-rater reliability of the measures at the provider or practice level. Future studies are needed to examine the inter-rater reliability of the CAHPS CC measures, as well as the temporal stability of the measures. Finally, the survey had a relatively low response rate. This may have resulted in potential response bias. Respondents were less likely to be Black, but there were no significant differences between respondents and non-respondents in terms of gender, Hispanic or Asian ethnicity, or preferred language. Despite these limitations, the CAHPS CC item set can serve as a tool to measure culturally competent care from the patient’s perspective.

Appendix 1

CAHPS Cultural Competency’s Domains, Survey Items, and Response Scales^*

Domain	Survey Item	Response Scale
Doctor Communication-Positive Behaviors	1. In the last 12 months, how often did this doctor explain things in a way that was easy to understand?	Never-Sometimes-Usually-Always
	2. In the last 12 months, how often did this doctor listen carefully to you?	Never-Sometimes-Usually-Always
	3. In the last 12 months, how often did this doctor spend enough time with you?	Never-Sometimes-Usually-Always
	4. In the last 12 months, how often did this doctor show respect for what you had to say?	Never-Sometimes-Usually-Always
	5. In the last 12 months, how often did this doctor give you easy to understand instructions about taking care of these health problems or concerns?1	Never-Sometimes-Usually-Always
Doctor Communication-Negative Behaviors	6. In the last 12 months, how often did this doctor interrupt you when you were talking?	Never-Sometimes-Usually-Always
	7. In the last 12 months, how often did this doctor talk too fast when talking with you?	Never-Sometimes-Usually-Always
	8. In the last 12 months, did this doctor ever use a condescending, sarcastic, or rude tone or manner with you?	Yes, Definitely
		Yes, Somewhat
		No
Doctor Communication-Health Promotion	9. In the last 12 months, did you and this doctor talk about a healthy diet and healthy eating habits?	Yes, Definitely
		Yes, Somewhat
		No
	10. In the last 12 months, did you and this doctor talk about the exercise or physical activity you get?	Yes, Definitely
		Yes, Somewhat
		No
	11. In the last 12 months, did you and this doctor talk about things in your life that worry you or cause you stress?	Yes, Definitely
		Yes, Somewhat
		No
	12. In the last 12 months, did this doctor ever ask you whether there was a period of time when you felt sad, empty or depressed?	Yes
		No
Doctor Communication-Alternative Medicine	13. In the last 12 months, has this doctor ever asked you if you have used these other people to help with an illness or to stay healthy (for example, acupuncturist or herbalist)?	Yes
		No
	14. In the last 12 months, has this doctor ever asked you if you used natural herbs?	Yes
		No
Shared Decision Making	15. In the last 12 months, did this doctor talk with you about the pros and cons of each choice for your treatment or health care?2	Yes
		No
	16. In the last 12 months, when there was more than one choice for your treatment or health care, did this doctor ask which choice you thought was best for you?	Yes
		No
Equitable treatment	17. In the last 12 months, how often have you been treated unfairly at this doctor’s office because of your race or ethnicity?	Never-Sometimes-Usually-Always
	18. In the last 12 months, how often have you been treated unfairly at this doctor’s office because of the type of health insurance you have or because you don’t have health insurance?	Never-Sometimes-Usually-Always
Trust	19. Do you feel you can tell this doctor anything, even things that you might not tell anyone else?	Yes
		No
	20. Do you trust this doctor with your medical care?	Yes
		No
	21. Do you feel this doctor always tells you the truth about your health, even if there is bad news?	Yes
		No
	22. Do you feel this doctor cares as much as you do about your health?	Yes
		No
	23. In the last 12 months, how often did you feel this doctor really cared about you as a person?	Never-Sometimes-Usually-Always
Access to Interpreter Services	24. In the last 12 months, did you use friends or family members as interpreters because there was no other interpreter available at this doctor’s office?3, 4	Yes
		No
	25. In the last 12 months, how often did your visit with this doctor start late because you had to wait for an interpreter? Do not include friends or family members.	Never-Sometimes-Usually-Always
	26. In the last 12 months, was there any time when you needed an interpreter and did not get one at this doctor’s office? Do not include friends or family members.	Yes
		No

^*For latest version, please see: Agency for Health Care Research and Quality (AHRQ). CAHPS Cultural Competence Item Set. Available at: https://www.cahps.ahrq.gov/Surveys-Guidance/Item-Sets/Cultural-Competence.aspx.

¹Screener item for item 5: In the last 12 months, did you talk with this doctor about any health problems or concerns? Yes, No

²Screener item for items 15 and 16: Choices for your treatment or health care can include choices about medicine, surgery, or other treatment. In the last 12 months, did this doctor tell you there was more than one choice for your treatment or health care? Yes, No

³Screener for items 24 and 25: An interpreter is someone who helps you talk with others who do not speak your language. Interpreters can include friends or family members, staff from the doctor’s office, or telephone interpreters. In the last 12 months, did you ever use an interpreter to help you talk with this doctor? Yes, No (Skip if Response= Yes)

⁴Screener for item 24: In the last 12 months, how often did you use a friend or family member as an interpreter when you talked with this doctor? Never- Always. (Skip if Response= Never)