Psychometric Evaluation of a Cultural Competency Assessment Instrument for Health Professionals

Abstract

Background

Few valid and reliable measures exist for health care professionals interested in determining their levels of cultural and linguistic competence.

Objective

To evaluate the measurement properties of the Cultural Competence Health Practitioner Assessment (CCHPA-129).

Methods

The CCHPA-129 is a 129-item web-based instrument, developed by the National Center for Cultural Competence (NCCC). Responses on the CCHPA -129 were examined using factor analysis; Rasch modeling; and Differential Item Functioning (DIF) across race, ethnicity, gender, and profession.

Subjects

2504 practitioners, including 1864 nurses (RN/LPN,/BSN); 341 clinicians (PA/NP); and 299 physicians (MD/DO), who completed the CCHPA-129 online between 2005 and 2008.

Results

Three factors representing domains of knowledge, adapting practice, and promoting health for culturally and linguistically diverse populations accounted for 46% of the variance. Among Knowledge factor items, 53% (23/43) fit the Rasch model, item difficulties ranged from −1.01 logits (least difficult) to +1.11 logits (most difficult), separation index (SI) 13.82, and Cronbach’s α 0.92. Forty-seven percent (21/44) Adapting Practice factor items fit the model, item difficulties −0.07 to +1.11 logits, SI 11.59, Cronbach’s α 0.88; and 58% (23/39). Promoting Health factor items fit the model, item difficulties −1.01 to +1.38 logits, SI 22.64, Cronbach’s α 0.92. Early evidence of validity was established by known groups having statistically different scores.

Conclusion

The 67-item CCHPA-67 is psychometrically sound. This shorted instrument can be used to establish associations between practitioners’ cultural and linguistic competence and health outcomes as well as to evaluate interventions to increase practitioners’ cultural and linguistic competence.

Introduction

A definitive body of evidence describes culturally and linguistically competent approaches to health care delivery as offering proven efficacy in addressing disparities in health care and health outcomes for some racial and ethnic groups or populations (1–13). This evidence has spurred accreditation agencies, quality care organizations, state professional licensing boards, and professional medical societies to view cultural and linguistic competence as core elements of health care (14–16). Cultural and linguistic competence is now considered essential for providing quality care to the increasingly diverse patient populations in the United States, territories, and tribal communities.

As the role of cultural and linguistic competence in responding to health and health care disparities expands, psychometrically sound measures of cultural and linguistic competence are needed for research, evaluation, and quality improvement. A review of the extant literature revealed that few instruments measuring cultural and linguistic competence demonstrate sound and proven psychometric properties on large sample populations (10,17–25). For example, only 13% of measures evaluating curricula assessing the cultural and linguistic competence of health professionals demonstrate some evidence of validity or reliability, thereby limiting the effectiveness of the majority of these measures (26). Kumas-Tan et al. (27) noted that most measures of provider cultural competence focused primarily on attitudes and knowledge while none specifically addressed social inequities, power differentials between provider and patient, and the skills or behaviors needed to practice effectively across diverse patient populations.

The Cultural Competence Health Practitioner Assessment (CCHPA-129), developed by the National Center for Cultural Competence (NCCC), assesses practitioners’ attitudes, knowledge, and skills defined in the literature as integral to culturally and linguistically competent care. The CCHPA-129 also assesses social inequities by inquiring about practitioners’ awareness of the influence of such factors as income, education, and neighborhood resources have on patients’ health behaviors and outcomes (28).

This study evaluated the properties of the CCHPA-129 by using factor analysis, Rasch modeling(29–34), and Differential Item Functioning (DIF) (35, 36) to analyze responses of a sample of nurses, physicians, and physician assistants.

Methods

In 2001, an expert group consisting of NCCC faculty; senior consultants known nationally for their expertise in cultural competence; and staff from the Bureau of Primary Health Care (BPHC), Health Resources and Services Administration, U.S. Department of Health and Human Services developed the initial draft of the CCHPA. This draft was critically reviewed by a 20-member panel to determine item and scale relevancy to health care practitioners’ assessment of cultural competence. The panel consisted of administrative and clinical health professionals from national organizations and academia including the National Association of Community Health Centers, the American Medical Association, the National Hispanic Medical Association, and the BPHC as well as community practices. Comments from this review process were summarized and evaluated by expert panel members for final recommendations on key content areas and modifications of the draft instrument.

A pilot version of CCHPA was developed, consisting of 138 items grouped into these content areas: Values and Belief Systems, Cultural Aspects of Epidemiology, Clinical Decision-Making, Life Cycle Events, Cross-Cultural Communication, and Empowerment/Health Management. All items were scored on four-point Likert scales with one of three-response format anchors: 1 (not at all/never) to 4 (very well/regularly/very often).

The CCHPA was reviewed by a focus group of 8 practitioners from 6 large community health centers located in, Arizona, Colorado, Georgia, Massachusetts, North Dakota and Oregon, that serve culturally and linguistically diverse patients. The focus group provided feedback on relevance of subscales, format, design, layout, item clarity, and the ease of administration. Input from the focus group further helped refine the draft instrument, which was piloted-tested in 5 of these 6 community health centers during an organizational self-assessment process conducted by the NCCC. Using this feedback, the CCHPA was reduced to 129 items and launched on the NCCC’s website in 2002.

Between January 2005 and May 2008, 8605 participants voluntarily completed the CCHPA-129 online at the NCCC website. Of those 8605 participants, 2504 identified themselves as physicians (MD/DO), clinicians including physician assistants (PA) and nurse practitioners (NP), or nurses (RN/LPN/BSN). All responses were anonymous with no identifying links to the participants This paper focuses on the responses of 2504 self-categorized clinicians, physicians, and nurses.

Psychometric analysis

A multi-step data analysis process applied exploratory factor analysis, Rasch modeling, and Differential Item Functioning (DIF) methods to develop the final version of CCHPA. A principal axes factor analysis with varimax rotation was conducted on the initial 129 items to identify the major factors for CCHPA-129. The number of factors retained was determined by examining the scree plot, large eigen values, and the number of items with loadings >.4 defining a factor.

A process for item selection and deletion from the CCHPA-129 was then performed for each factor. Items were evaluated using the following criteria: items with low factor loading (<.4), low item-total correlation (<.3), Rasch misfit (Infit and Outfit values with an outside range .7 to 1.3), and/or DIF items (DIF size >.43 logits) (37). For each factor, misfit items were removed and Rasch analysis was rerun. The process was repeated until Rasch analysis results showed that all remaining items in a factor (subscale) exhibited a good model fit.

The DIF analysis was performed again for the items in each subscale. In Rasch modeling, DIF implies that item difficulty is different for different groups and may subsequently be biased toward a certain group, which could threaten the validity of the measure and produce misleading results (35,36). In this study, DIF attributable to race/ethnicity, gender, and professional affiliations were assessed. Items with moderate to large size (DIF size >.43 logits); (37) were deleted from the final version of CCHPA-129 subscales.

Reliability of the Final CCHPA-129 Scales

Rasch person (and item) reliability statistics (38) were used to evaluate the internal consistency of the final CCHPA-129 subscales. The person reliability statistic is equivalent to traditional Cronbach’s alpha (37,38). Item reliability with no traditional equivalent depicts the level of confidence that items would have given the same respective order in another sample of participants. Person reliability statistics >.8 and item reliability statistics >.9 represent excellent internal consistency. Rasch also provides a separation index for both persons and items. A high person separation index indicates a wide range of ability scores within the sample population. A high item separation index indicates that the items cover a useful range of item difficulty appropriate for measuring persons with a wide range of the ability being assessed (31).

Validity of the Final CCHPA-129 Scales

Validity of the final CCHPA-129 subscales was examined using multiple sources of information. Initially, content validity was established through focus groups and an expert panel review of the concept, content, and item pool. Next, a principal factor analysis was performed to determine the dimensionality of the item pool. Rasch analysis was conducted on each factor retained. In Rasch modeling, a good item fit statistic and a good match between item difficulty and person ability provide evidence of construct validity (39, 40).

Fit statistics generated by Rasch analysis are used to determine the quality of items. In Rasch model expectation, individuals with lower ability obtain lower scores while those with higher ability have higher scores on any item (33, 34, 41). When all items in a measure are a good fit, there is evidence of construct validity of the measure (39, 40).

A comparison between known groups (e.g., practitioners with no employer-sponsored training on cultural competence versus those with any employer-sponsored training) was also performed as an evaluation of concurrent validity (42).

Rasch and DIF analyses, which performed using WINSTEPS 3.69 software (37), were based on the Rasch partial credit model (43). Item difficulty estimates are reported. Descriptive and inferential statistics were computed to determine if cultural competency scores differed by gender, age, race, profession, and number of years since employer-sponsored training.

Results

Of the 2504 respondents (Table 1), the majority were females (87%) ≤ 50 years old (81%) with 80% self-identified as White, 9% Black, 6% Asian, 8% Hispanic, and 5% other. Approximately 75% of the respondents were nurses, 14% clinicians (NP/PA), and 12% physicians. Across the sample, over 88% participated in employer-sponsored cultural competency training.

Table 1.

Characteristics of Practitioners

Characteristics of Practitioners	n*	%
Gender
Female	2176	86.9
Male	325	13
Age
<30	726	29.1
31–40	631	25.3
41–50	665	26.7
51–60	413	16.6
>60	58	2.3
Race*
White	1816	80
Black	213	9
Asian	145	6
Other	129	5
Ethnicity
Hispanic, Latino or Spanish origin	192	7.8
Non-Hispanic, Latino or Spanish origin	2273	92.2
Profession
Nurse (RN, LPN, BSN)	1864	74.4
Clinician (PA/NP)	341	13.7
Physician (MD/DO)	299	11.9
Employer sponsored cultural competency training
Any training	2152	87.5
No training	307	12.5

Note.

^*Due to missing data not all responses sum to the total of n=2504

Item Reduction

Missing item responses were infrequent (<1%) and imputed by mean substitution. Factor analysis supported three major groups accounting for 46% of the total variance in the data: 33.4% by Factor 1 (Knowledge), 7% by Factor 2 (Adapting Practice), and 5.2% by Factor 3 (Promoting Health). The content of these factors reinforces the ability of the CCHPA-129 to measure the targeted concepts. Knowledge factor and Adapting Practice factor reflect knowledge and skills related to delivering culturally and linguistically competent care to individual patients. Promoting Health factor assesses attitudes, intentions, knowledge, and skills required to address community health issues. Items in this factor focus on the underlying causes of social inequities contributing to health and health care disparities and advocacy on behalf of diverse patients, families, and communities to improve health care access, status, and outcomes(28).

Knowledge factor

Two misfit items were identified because of Rasch misfit statistics and measurement redundancy (e.g., Infit and Outfit values out of the range .7 to 1.3) (32) and were removed. Subsequent DIF analysis revealed items exhibiting significant DIF (>.43 logits) including 17 items with DIF for race, 4 items for gender, and 6 items for profession. An example item with large DIF that compares Blacks and Asians was “I know the impact of poverty on the health and well-being of communities I serve.” These DIF items were further reviewed by at least two content experts. Those DIF items considered biased were dropped, leaving a total of 23 items with good item-model fit and no presence of DIF (Table 2). The estimated item difficulties for retained items ranged from −1.01 logits (least difficult) to +1.11 logits (most difficult) (Table 2).

Table 2.

Estimates of Item Difficulty, Standard Error (SE), Fit Statistics and Item-Total Correlations (ITC) for Subscale 1: Knowledge of Diverse Patient Populations

Subscale 1	Abbreviated item content	Item	SE	Infit	Outfit	ITC
Item 19	Impact of family on health care	−1.01	0.04	0.97	0.97	0.7
Item 16	Impact of religion on health care	−0.57	0.04	1.12	1.12	0.66
Item 20	Knowledge of strengths/resiliencies of patients	−0.46	0.04	0.88	0.88	0.73
Item 8	Know diet patterns based on income	−0.45	0.04	0.87	0.86	0.74
Item 1	Know health beliefs of patients	−0.43	0.04	0.89	0.88	0.73
Item 35	Know limited English proficiency impacts health of communities	−0.43	0.04	1.21	1.24	0.64
Item 2	Know illness beliefs of patients	−0.39	0.04	0.92	0.9	0.72
Item 9	Know diet patterns based on education	−0.36	0.04	0.84	0.82	0.75
Item 11	Know diet patterns based on cultural preferences	−0.33	0.04	0.89	0.88	0.73
Item 30	Know unemployment impacts health of communities	−0.28	0.04	0.98	0.98	0.71
Item 3	Know wellness beliefs of patients	−0.24	0.04	0.86	0.85	0.74
Item 13	Know exercise patterns based on education	−0.04	0.04	0.84	0.83	0.75
Item 4	Know help-seeking behaviors of patients	−0.02	0.04	0.92	0.91	0.73
Item 25	Know mental health impacts health of communities	0.03	0.04	1.07	1.09	0.69
Item 10	Know diet patterns based on neighborhood resources	0.07	0.04	0.92	0.92	0.73
Item 12	Know exercise patters based on income	0.17	0.04	0.91	0.91	0.73
Item 23	Know sexual abuse/trauma impact health of communities	0.3	0.04	1.11	1.12	0.68
Item 14	Know exercise patters based on neighborhood resources	0.32	0.04	0.89	0.89	0.74
Item 32	Know homelessness impacts health of communities	0.33	0.04	1.15	1.17	0.67
Item 26	Know oral health impacts health of communities	0.73	0.04	1.17	1.16	0.66
Item 36	Know environmental factors impact health of communities	0.95	0.04	1.17	1.24	0.66
Item 6	Know patient preference for traditional healers	1.01	0.04	1.17	1.17	0.66
Item 7	Know patient preference for traditional healing practices	1.11	0.04	1.13	1.14	0.67

Adapting Practice factor

Fifteen misfit items were excluded, and the subsequent DIF analysis and content review eliminated 19 items, leaving 21 items with difficulty ranging from −.7 logits to +1.11 logits (Table 3).

Table 3.

Estimates of Item Difficulty, Standard Error (SE), Fit Statistics and Item-Total Correlations (ITC) for Subscale 2: Adapting Health Care for Diverse Patients Populations

Subscale 2	Abbreviated item content	Item	SE	Infit	Outfit	ITC
Item 107	Modify treatment/discharge plans for disabled patients	−0.7	0.04	0.83	0.81	0.64
Item 105	Modify explanation of diagnosis/results for disabled patients	−0.56	0.04	0.94	0.99	0.63
Item 106	Modify explanation of treatment options for disabled patients	−0.5	0.04	0.9	0.89	0.64
Item 108	Modify referrals and resources for disabled patient	−0.46	0.03	0.88	0.92	0.65
Item 111	Ask patients’ their likelihood of treatment adherence	−0.35	0.03	0.99	1.04	0.64
Item 110	Ask patients what may improve the illness better	−0.29	0.03	1.04	1.07	0.63
Item 46	Integrate information from family in diagnostic protocols	−0.17	0.04	1.04	1.08	0.65
Item 56	Include cultural needs in treatment/discharge plans	−0.09	0.03	0.9	0.85	0.69
Item 112	Written information appropriate for cultural/linguistic needs	−0.04	0.03	1.17	1.23	0.61
Item 96	Modify health information to meet patient’s literacy needs	−0.02	0.03	0.94	0.9	0.68
Item 109	Ask patients what they think caused their illness	0.01	0.03	1.13	1.18	0.63
Item 95	Treatment/discharge plans in patient’s preferred form	0.02	0.03	0.96	0.95	0.67
Item 45	Integrate patient’s belief of illness cause in diagnostic protocol	0.03	0.04	1.09	1.12	0.65
Item 94	Modify health education to meet needs of diverse patients	0.09	0.03	0.9	0.91	0.69
Item 92	Incorporate culturally-appropriate non-verbal communication	0.1	0.03	1.03	1.05	0.66
Item 113	Written information appropriate for literacy needs	0.25	0.03	1.1	1.17	0.65
Item 59	Include cultural preferences for referrals in treatment/discharge plans	0.28	0.03	0.96	0.93	0.7
Item 57	Include group strengths/resiliencies in treatment/discharge plans	0.38	0.03	0.94	0.92	0.71
Item 44	Integrate patient’s health beliefs & values in diagnostic protocol	0.38	0.04	1.03	1.04	0.69
Item 55	Include beliefs about healing in treatment/discharge plans	0.52	0.03	0.97	0.96	0.71
Item 118	Integrate information about cultural strengths and resiliencies	1.11	0.03	1.18	1.24	0.67

Promoting Health factor

Nine misfit items were excluded, and the DIF analysis and content review deleted 10 items exceeding the DIF criterion, leaving 23 items with difficulties ranged from −1.05 logits to +1.38 logits (Table 4).

Table 4.

Estimates of Item Difficulty, Standard Error (SE), Fit Statistics and Item-Total Correlations (ITC) for Subscale 3: Promoting the Health of Diverse Communities

Subscale 3	Abbreviated item content	Item	SE	Infit	Outfit	ITC
Item 68	Stay current on factors contributing to health disparities research	−1.05	0.03	1	0.97	0.64
Item 117	Assist patients with obtaining financial resources for services	−1	0.03	1.27	1.28	0.57
Item 120	Provide information so patients can advocate if they experience racism	−0.91	0.03	1.06	1.03	0.64
Item 118	Integrate information about cultural strengths, assets, and resiliencies	−0.84	0.03	0.82	0.8	0.71
Item 67	Keep current on race/ethnicity affecting drug metabolism	−0.62	0.03	1.18	1.19	0.6
Item 86	Consider family’s cultural beliefs during transition to adulthood	−0.43	0.03	1.06	1.06	0.64
Item 85	Consider family’s cultural beliefs during transition to adolescence	−0.36	0.03	1.1	1.09	0.64
Item 121	Advocate through professional/discipline associations	−0.28	0.03	0.9	0.91	0.69
Item 64	Stay current use of medicinal plants and minerals	−0.27	0.03	1.02	1.04	0.65
Item 84	Consider family’s cultural beliefs during middle childhood	−0.21	0.03	1.17	1.19	0.62
Item 129	Provide information to communities for health promotion	−0.18	0.03	0.9	0.92	0.69
Item 78	Deliver care to accommodate beliefs about puberty	−0.16	0.03	1.13	1.14	0.63
Item 69	Affiliated with natural networks of support	0.05	0.03	0.99	1.01	0.66
Item 125	Advocate through social justice community organizations	0.11	0.03	0.81	0.78	0.72
Item 128	Advocate through faith-based organizations	0.19	0.03	1.12	1.25	0.63
Item 75	Affiliated with social organizations	0.22	0.03	0.86	0.85	0.7
Item 122	Advocate through local and state health officials and legislators	0.25	0.03	0.83	0.8	0.71
Item 119	Offer workshops that help ESL* patients navigate the healthcare system	0.52	0.03	1.12	1.13	0.63
Item 42	Know general health status of people in the country of origin of refugees	0.73	0.03	1.23	1.28	0.5
Item 74	Affiliated with cultural advocacy organizations	0.77	0.03	0.79	0.75	0.72
Item 73	Affiliated with ethnic merchants and businesses	1.02	0.03	0.86	0.82	0.7
Item 127	Advocate through the media	1.07	0.03	0.97	0.86	0.67
Item 71	Affiliated with traditional healers	1.38	0.03	0.89	0.81	0.68

Note.

^*ESL means English as a second language

Validity and Reliability

Overall, 67 good fit and DIF free items, including 23 in Knowledge factor, 21 in Adapting Practice factor, and 23 in Promoting Health factor were retained for the CCHPA-67. Content experts reviewed all DIF items, resulting in 9 items retained in a temporary item pool for future modifications and psychometric reevaluation. Scores were computed by summing all items defining a factor.

Reliability

The Rasch person reliability coefficients were excellent for each of the subscales: .94 for Knowledge factor, .88 for Adapting Practice factor, and .92 for Promoting Health factor, indicating a high internal consistency for each subscale. The Rasch item separation indices were 13.82 for Knowledge factor, 11.59 for Adapting Practice factor, and 22.65 for Promoting Health factor. All are well above the minimum index score of 2.0 (37). Each factor demonstrated excellent internal consistency with an item reliability of .99.

Construct Validity

According to fit statistics, all items in the factors of the CCHPA-67 were fit (see Tables 2 to 4), suggesting that the factors were appropriately defined and measuring a similar construct. Moreover, since each of the CCHPA-67 factors includes no DIF items, each final subscale holds measurement invariance across different samples, which further supports the validity of the CCHPA.

The comparison between item difficulty and person ability in item-person maps shown in Figures 1a to 1c provide further evidence of the construct validity of CCHPA-67. These item-person maps display the location and distribution of both items in each factor and the practitioners’ competency on the same common logit metric. On each map, the numeric scale is located on the left, persons are charted on the middle column, and items charted on the right. Persons with higher ability and the more difficult items were charted at the top of the map.

Figure 1.

Figure 1a: Item-Person Map for Subscale 1 “Knowledge of Diverse Patient Populations”

Note: Each “#” mark represents 16 respondents, while each “.” denotes 1 to 15 respondents. Number on the right side of the map represents item numbers. “Category 1–2”: Difficulty threshold that separates the likelihood of scoring 1 from the likelihood of scoring 2 on an item; “Category 3–4”: Difficulty threshold separates the likelihood of scoring 3 from the likelihood of scoring 4 on an item. “Mean Measure”: Average item difficulty.

Figure 1b: Item-Person Map for Subscale 2 “Adopting Health Care for Diverse Patient Populations”

Note: Each “#” mark represents 13 respondents, while each “.” denotes 1 to 12 respondents. Number on the right side of the map represents item numbers. “Category 1–2”: Difficulty threshold that separates the likelihood of scoring 1 from the likelihood of scoring 2 on an item; “Category 3–4”: Difficulty threshold separates the likelihood of scoring 3 from the likelihood of scoring 4 on an item. “Mean Measure”: Average item difficulty.

Figure 1c: Item-Person Map for Subscale 3 “Promoting the Health of Diverse Communities”

Note: Each “#” mark represents 13 respondents, while each “.” denotes 1 to 12 respondents. Number on the right side of the map represents item numbers. “Category 1–2”: Difficulty threshold that separates the likelihood of scoring 1 from the likelihood of scoring 2 on an item; “Category 3–4”: Difficulty threshold separates the likelihood of scoring 3 from the likelihood of scoring 4 on an item. “Mean Measure”: Average item difficulty.

For the Knowledge factor (Figure 1a), the maximum item difficulty range was −4.5 to 4.3 logits (in contrast to the mean item difficulty level of −1.0 to 1.1) while person ability ranged −7.8 to 7.8 logits; and 5% of the sample at the higher and lower ends of ability are outside the maximum item difficulty range, suggesting items are needed to assess persons with low and high competency levels. For the Adapting Practice factor (Figure 1b), maximum item difficulty ranged −2.0 to 3.1 logits while person ability ranged −5.7 to 6.1 logits; and 18% of the sample at the higher end of ability are outside the item difficulty range, suggesting items are too easy. For the Promoting Health factor (Figure 1c) there is a good match between maximum item difficulty (ranging between −3.0 and 3.3 logits) and person ability (ranging between −5.8 and 6.1 logits), and less than 5% of the sample at the very high and lower ends of ability are outside item difficulty range, suggesting the participants were well assessed by the items.

Demographic Differences and Known Groups Validity (Table 5)

Table 5.

Statistically Significant Differences in Scores by Subscale

Subscale 1: Knowledge of Diverse Patient Populations
Group	N	Mean S1 Score	Std. Deviation	Effect Size
Race**
White	1816	63.8	12.5	0.42
Black	213	69.1	12.5
Asian	145	65	13.8	0.31
Profession**
Physicians (MD/DO)	299	65.7	14.7	0.13
Clinicians (PA/NP)	341	66.5	12.6	0.2
Nurses (RN/LPN)	1864	64	12.5
Ethnicity**
Hispanic	192	68.2	11.6	0.3
Non-Hispanic	2465	64.3	12.9
Subscale 2: Adapting Health Care for Diverse Patient Populations
Group	N	Mean S2 Score	Std. Deviation	Effect Size
Gender**
Male	325	65.1	13.9	0.21
Female	2176	67.8	12.6
Ethnicity**
Hispanic	192	70.4	11.5	0.25
Non-Hispanic	2273	67.2	12.9
Subscale 3: Promoting the Health of Diverse Communities
Group	N	Mean S3 Score	Std. Deviation	Effect Size
Age**
Under 30 years	413	57.3	15.7	0.37
31 – 40 years	58	58.6	17.3	0.46
41 – 50 years	726	53.3	15	0.11
51 – 60 years	631	51.7	14.7
Over 60 years	665	53.6	15.7	0.12
Race**
White	1816	53	15.2
Black	213	55.3	15.5	0.15
Asian	145	56.8	15.3	0.25
Profession**
Physicians (MD/DO)	299	58.5	15.3	0.42
Clinicians (PA/NP)	341	59.1	14.3	0.46
Nurses (RN/LPN)	1864	52.1	15.2
Ethnicity*
Hispanic	192	56.1	15.9	0.16
Non-Hispanic	2273	53.6	15.4

Note:

^*p ≤ .05,

^**p ≤ .01

When analyzing mean factor scores by respondent demographics for Knowledge factor, statistically significant differences in scores by race and profession were noted. When compared to Blacks (69.1), Whites (63.8) had lower scores followed by Asians (65.0) with p=.001 and effect sizes .42 and .31, respectively. Additionally, nurses (64.0) scored lower than physicians (65.7) or clinicians (66.5) with p=.001 and smaller effect sizes (.13 and .20). For Adapting Practice factor, the scores differed by gender and ethnicity with females (68.1) scoring higher than males (64.7) p=.001 and Hispanics (70.4) scoring higher than Non-Hispanic (67.2) p=.001 with small effect sizes (.21 and .25). Promoting Health factor had statistically significant differences in scores for age, race, profession, and ethnicity. Respondents age 31 to 40 years (58.6) had the highest score while those age 51 to 60 years (51.7) had the lowest p=.001 and moderate effect size (.46). Asians scored higher (56.8) than Black (55.3) and non-Hispanic Whites (53.0), p=.003 but the effect sizes were .25 and .15. Nurses scored lower than clinicians or physicians, p=.001 with a moderate effect sizes (.42 and .46). Hispanics scored higher than non-Hispanics, p=.05, but the effect size was small (.16). For all three factors, respondents with no training had statistically significantly lower scores than respondents with any training (Table 6). Knowledge factor had the smallest effect size (.19) and Promoting Health factor, the largest (.49).

Table 6.

Mean Subscale Scores by Training Status

Subscale 1: Knowledge of Diverse Patient Populations**
Employer-Sponsored Training+	N	Mean Score	Std. Deviation	Effect Size
No Training	307	62.5	12.4	0.19
Training	2152	64.9	12.8
Subscale 2: Adapting Health Care for Diverse Patient Populations**
Employer-Sponsored Training+	N	Mean Score	Std. Deviation	Effect Size
No Training	307	63.8	13.5	0.34
Training	2152	68.1	12.5
Subscale 3: Promoting the Health of Diverse Communities**
Employer-Sponsored Training+	N	Mean Score	Std. Deviation	Effect Size
No Training	307	47.3	14.7	0.49
Training	2152	54.7	15.3

^**p = .001,

⁺45 respondents did not answer the training question.

Discussion

This study evaluated the properties of CCHPA-129 using factor analysis, Rasch modeling, and DIF analysis. The final CCHPA-67 included 67 good fit and DIF-free items with excellent reliability with evidence of content and construct validity. Reliability was confirmed for each factor including Knowledge, Adapting Practice, and Promoting Health with all factors having Cronbach α’s greater than .85 and separation indices ranging from 12 to 23. The fit statistics indicated that practitioners were responding to the items as expected by the Rasch model. Additionally, the CCHPA-67 showed satisfactory variability and precision within the range of least-to-most difficult items, thus indicating wide variations in practitioners’ levels of competence with minimal redundancy among the items.

The CCHPA-67 met test content and construct validity, two validity criteria established by psychological testing (44). The test content validity is based on the collective knowledge and recommendations of national experts in cultural and linguistic competency, who engaged in an intensive iterative process, including literature review, discussions, focus groups, and pilot testing to develop the initial instrument item pool. According to the adapted Cross model (45), cultural competence requires practitioners to 1) acknowledge and respond to cultural differences, 2) reflect and understand their own culture and the culture of medicine, 3) engage in self-assessment, 4) acquire cultural knowledge and skills, and 5) adapt their practice to the cultural contexts of families and communities served. Additionally, a definition and conceptual model of linguistic competence requires organizational/personnel capacity to 1) communicate effectively, in a manner easily understood by diverse groups (e.g. persons with limited English proficiency, low or no literacy skills, and/or disabilities) and 2) address the health literacy needs of patients and their families. The three factors of the CCHPA-67: Knowledge, Adapting Practice, and Promoting Health are consistent with these conceptual frameworks. This consistency strengthens the assessment potential of the CCHPA-67, which, in turn, has implications for assessing future interventions to improve cultural and linguistic competence.

Construct validity was supported by all items in CCHPA-67 having a good-fit under the Rasch model being DIF free and hierarchically ordered. (e.g., the item difficulty order of each subscale is consistent with practical expectation). For example, in the Knowledge factor, “Know patient preference for traditional healing practices” was the most difficult item while the easiest was “Impact of family on health care.” This difference reflects that health care practitioners may understand how families impact health care but may not know patient’s preferences for traditional healing practices.

The validity was also documented by the significantly lower mean cultural competence scores for all 3 subscales between individuals with and without employer-sponsored cultural competency training. While the effects size was smaller for Knowledge and Adapting Practice than for Promoting Health factors, these associations do provide evidence of the CCHPA-67’s ability to discriminate between individuals with and without training. Also, difference in factors scores by race, gender, and profession showed sensitivity of the CCHPA-67 to group differences. Blacks and Asians had higher mean scores on the Knowledge factor than non-Hispanic Whites. This may be attributed to the experiences of Blacks and Asians both as minorities and health professionals in the U.S. There is a large body of literature citing their commitment to deliver care to underserved and diverse populations (46–48). Nurses had lower mean scores on the Promoting Health factor than physicians and clinicians. It is possible that training and direct experience providing care in community-based settings differs for nurses, clinicians, and physicians. Although these results provide some evidence that practitioners’ race and ethnicity have associations with higher levels of cultural and linguistic competence, the low to moderate effect sizes warrant future studies to substantiate this difference. Therefore, cultural and linguistic competency training/professional development should continue to focus on all practitioners independent of race and ethnicity.

Rasch model analysis provided information to improve the CCHPA-67. The item-person maps for the Knowledge and Promoting Health factors indicate good item difficulty and person ability match, suggesting that knowledge and community engagement-related cultural and linguistic competency of the majority of the participants were assessed well by items in these two factors. The item-person map for the Adapting Practice factor revealed that some items were too easy for some respondents and that more difficult questions are needed to better assess high ability. It is plausible that a number of the more difficult items in this factor were deleted because they did not met DIF criteria. Some of items that were retained after re-evaluation by content experts will be re-evaluated to fill the gaps in the Adapting Practice factor. Others items will be retain in an item pool for future use. Future work will be needed to modify and test the psychometric properties of items.

This study addresses the paucity of research using a large sample to assess cultural and linguistic competence among health care providers. However, the major limitation of this study is the self-selection of the practitioner sample, making it likely that many practitioners in the sample have a strong interest in cultural and linguistic competence. Therefore, the results could be biased by over-representing individuals with a stronger interest and more knowledge about cultural and linguistic competence. An evaluation of this measure with a more representative and racially and ethnically diverse sample of practitioners is warranted. Future validation studies should also include analyses to identify new literature to address domains of relevance for the most effective items, establish test-retest reliability, and assess sensitivity to change. as well as to test association with patient outcomes.

However, despite these limitations, the CCHPA-67 enables health care practitioners seeking a self-assessment measure of cultural and linguistic competence to identify areas for improvement in knowledge, adapting practice, and promoting health for diverse communities. This instrument also allows the development of learning plans to incorporate cultural and linguistic competence into their practices (47,49). Given that accreditation bodies for health care settings and education programs require practitioners develop the knowledge and skills needed to deliver culturally and linguistically competent care, the CCHPA-67 is a psychometrically sound instrument for identifying learning needs and guiding curriculum development.

As cultural and linguistic competence emerges as a marker for quality and equitable care, the CCHPA-67 has the potential to offer valid and reliable evidence of practitioners’ knowledge and skill sets. Ultimately, the CCHPA-67 can be used in studies seeking to establish associations between practitioners’ cultural and linguistic competence, health outcomes for culturally and linguistically diverse populations, and reduction of health and health care disparities.