Development of a Measure of Physician Engagement in Addressing Racial and Ethnic Health Care Disparities

Abstract

Objective

To develop a measure of physician engagement in addressing health care disparities.

Data Sources/Study Design

Cross-sectional survey of a national sample of physicians assessing each hypothesized component of engagement (Awareness, Reflection/Empowerment, and Action [AREA]).

Data Collection/Extraction Methods

Results examined using factorial analysis; predictive validity of final scale examined among highly engaged physicians.

Principal Findings

A nine-item scale derived from the AREA model has face validity, content validity, and applicability to a diverse group of physicians in measuring engagement. Partial correlations confirmed the mediating role of Reflection and/or Empowerment between Awareness and Action. Use of the scale among expert physicians suggests it reliably detects highly engaged physicians.

Conclusions

A nine-item survey can measure physician engagement in addressing health care disparities.

Racial and ethnic health care disparities have been well documented (Institute of Medicine Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care 2003; Agency for Healthcare Research and Quality 2005). Although some causes of disparities are outside any individual physician's hands, other causes are amenable to intervention by clinicians, and many health care leaders believe that physicians have an important role to play in efforts to eliminate health care disparities.

In 2005, physicians from more than 40 medical professional associations formed the Commission to End Health Care Disparities. A key goal of the Commission has been to increase physician engagement in addressing health care disparities, and yet it has not been clear how to track progress in this regard. At least two prior studies have examined physicians' beliefs about health disparities (Kaiser Family Foundation 2002; Lurie et al. 2005), but the conceptual models that drove these surveys were not reported, nor was any survey validation reported, and physician engagement may entail more than beliefs alone. In short, we found no research on how to measure the level of engagement of groups of physicians in addressing disparities. Indeed, there appears to be no generally accepted understanding of what professional “engagement” in addressing health disparities means.

We undertook the present study to develop a brief tool to measure physician engagement in addressing health disparities. Such a tool might help target efforts to engage physician communities, predict physician populations most amenable to certain messages, and track efforts to increase physician engagement at the local, regional, and national levels.

METHODS

Development of the Conceptual Model and Survey Items

We assembled an expert panel of 14 physicians and researchers from the Commission to End Health Care Disparities (see “Acknowledgments”) to assist in survey development; most of these experts are practicing physicians.

Next, a comprehensive, semistructured literature review was conducted to examine the notion of physician engagement generally and engagement to address health care disparities specifically. Very few relevant articles were found in a PubMed search (searching for “physician engagement in addressing health disparities” yielded zero results, for example, and searching for “physician engagement” yielded only four papers, none of which were research reports). A review of health management studies revealed an annual survey of “physician engagement” within hospitals or health plans, conducted by the Gallup organization, which focuses on domains of “confidence,”“integrity,”“pride,” and “passion.” The general framework driving the Gallup survey is the concept of “emotional attachment” to one's workplace. A similar survey by Gallup, conducted with the National Medical Association in 2003, assessed the level of minority physicians' engagement with the medical profession (report available from the authors). It assessed domains of “professional loyalty” and “emotional attachment.” Neither survey, however, focused on physicians' engagement to address a health-related issue. Finally, a review of behavioral change concepts was especially useful in our efforts. The concept of “readiness to change” or “stages of change” posits that learning can lead to a process of integration of new knowledge and contemplation about that knowledge, with stepwise movement toward action (Prochaska and DiClemente 1992; Nigg et al. 1999). This transtheoretical model of health behavior change suggests that behavioral change occurs in stepwise fashion from precontemplation, through contemplation, preparation, action, maintenance, and, in some cases, termination (Prochaska and Velicer 1997).

Based on these sources and discussions within the expert panel, we developed a conceptual model of engagement that postulates physician engagement in addressing health disparities to be a multistage process. The stages we hypothesized were (1) Awareness of the issue, (2) Reflection on the issue and one's potential role in addressing it, (3) Empowerment, or the realization that one has the capacity to make a difference, and finally (4) Action undertaken to address the issue. We named this the “AREA” conceptual model of engagement.

Survey and Scale Development

To test the AREA model, we developed survey items intended to explore each of the model's four hypothesized domains. Most response options used five-point Likert scales, ranging from “strongly agree” to “strongly disagree.” Action items were assessed with a yes/no response frame. Because physician actions to address community problems can occur at both individual and organizational levels (Gruen, Pearson, and Brennan 2004), different items referred to each of these levels. The survey was extensively pretested to ensure psychometric reliability (i.e., that the questions were understandable to a wide variety of physicians), maximize face and content validity, and to minimize the likelihood of socially desirable response bias. The survey also measured physician and practice demographics, patient case mix, and other variables that might be related to engagement.

Sample Design and Survey Administration

We administered the survey to a sample of 2,000 primary care physicians drawn from the American Medical Association Masterfile, which contains contact information for all licensed physicians in the United States. Because others have found relatively low levels of Awareness of health disparities among physicians (Taylor et al. 2006), we oversampled physicians we believed would be more likely to be aware. Half of our sample was a national random sample, the other half was drawn from physicians practicing in zip codes with high proportions (>50 percent) of minorities (National Minority Health Month Foundation 2007). Within the oversampled group, we drew equal numbers of physicians from higher and lower physician-density zip codes, by decile, to ensure that physicians from high-minority inner-city zip codes that contained an academic medical center would not make up most of this sample. To account for the sampling strategy, survey weights were created, which allow us to report results that are representative of all primary care physicians in the United States. Physicians who were retired, without a forwarding address, or in training were excluded. A survey was mailed to each physician in December 2004 with a $2 cash incentive. Two subsequent waves of surveys were sent to nonrespondents approximately 4 weeks apart. The Western Institutional Review Board (Olympia, Washington) approved the study.

Statistical Analyses

Initially, our proposed 15-item model consisting of items reflecting the domains of Awareness (three items), Reflection (three items), Empowerment (three items), and Action (six items) was tested using confirmatory factor analysis. Next, exploratory factor analysis was used to look for areas of poor fit between the data and our proposed model. A set of criteria was applied to select survey items to include in the final model: a factor loading >0.40 on only one of the factors; a large range of subject responses; and a high item-total correlation. An oblique rotation method was used to allow extracted factors to be correlated. Using these criteria, a final factor analysis model containing nine items was selected and then retested by confirmatory factor analysis. Finally, to test predictive validity, we asked the members of the Society for General Internal Medicine's Task Force on Health Disparities to complete the final nine-item survey. These primary care physicians, who are recognized as highly engaged in addressing health disparities, were not involved in designing the instrument.

RESULTS

Survey Respondents

Of the 2,000 physicians selected for the sample, seven (0.004 percent) were duplicates and 293 (14.7 percent) were otherwise ineligible. Of the 1,700 remaining physicians, 887 returned completed surveys, representing a response rate of 52.1 percent. Weighted respondent characteristics are in Table 1.

Table 1.

Respondent Characteristics (N = 887 Physicians)^*

Age, Mean (Range), Years	48 (28–84)
Male (%)	73
Race (%)
Caucasian or white	75
Asian or Pacific Islander	14
African American or black	4
Hispanic or Latino	5
Other	3
Specialty (%)
Internal medicine or general practice	29
Pediatrics	15
Obstetrics/gynecology	15
Family practice	3
Other	38
Board certified (%)	85
Primary practice site (%)†
Solo	20
Single-specialty	27
Multi-specialty	13
Community health clinic	7
Group/staff model HMO	8
Private hospital	9
Medical school or university hospital	19
State or local government hospital	6
Other	8
Size of patient panel, mean	2,600
Number of patients seen daily, mean	21

^*Column totals may be >100 due to rounding.

^†More than one may apply; frequencies weighted for sampling design.

Factor Analyses

Our first factor analysis tested the validity of our proposed four-factor (AREA) engagement model, using all 15-survey items. The goodness of fit index (GFI) was 0.92 and the adjusted goodness of fit index (AGFI) was 0.89, suggesting the model provided an adequate fit to the data.

To improve the fit, we next conducted a four-factor exploratory analysis, looking for underlying factor patterns that could lead to areas of poor fit. Using Eigenvalues >1, this analysis yielded four factors that accounted for 52 percent of the total variance. Fourteen items had a factor loading >0.40 on only one of the factors. One item loaded at >0.40 onto two factors and was assigned to the factor with higher loading. However, only two of these four factors matched our initial model subdomains exactly—one factor consisted of three items that had been designed to measure Awareness, and another consisted of five items designed to measure Action. The third factor consisted of two items designed to measure Reflection and two items to measure Empowerment, and the fourth factor consisted of one item designed to measure Reflection, one item Empowerment, and two items Action. This indicated that two of our initially hypothesized stages, Reflection and Empowerment, might not represent two distinguishable underlying theoretical constructs, and/or that some of our survey items were not reliably measuring a single construct.

Based on these results, we decided to merge Reflection and Empowerment together as a single underlying construct. We then applied a stronger factor-loading criterion (0.57) to eliminate items that did not clearly measure the intended underlying constructs. We also dropped one item that did not specifically address minorities alone (item 12, Supplementary Appendix). This resulted in our final nine-item, three-factor engagement instrument with three items (loading 0 .72, 0.80, and 0.80) designed to measure the Awareness domain, three items (loading 0.71, 0.71, 0.72) from the Reflection/Empowerment domain, and three items (loading 0.57, 0.64, 0.69) from the Action domain.

We then ran a confirmatory factor analysis on the final three-factor model. The results indicated that the model fit the survey data better than the initially hypothesized four-factor model, with fit indexes GFI (0.95), AGFI (0.91), root mean square residual (RMR) (0.05), normed fit index (NFI) (0.88), and comparative fit index (CFI) (0.89).

Scoring the Scale in the General Physician Population

A simple scoring system was derived, where each item was assigned a scale value of 0–4 (dichotomous variables were given scores of 0 or 4) such that the sum of the nine items' scores could range from 0 to 36, with higher scores representing greater levels of engagement. The mean engagement score for the general population of physicians surveyed was 18.4, while the range of scores was 2–36 and the interquartile range was 14–22. The distribution of the scores was normal (Figure 1 and Table 2).

Figure 1.

Frequency Distribution of Engagement Score of General Physicians and Experts.

Table 2.

Agreement with Items in Engagement Model among Generalist Physicians and Experts in Health Disparities

	Physicians % Agree or Strongly Agree	Experts % Agree or Strongly Agree
Awareness
1. Across the United States, minority patients generally receive lower quality care than white patients	55	100
2. Some minorities with heart disease are less likely than whites with heart disease to get specialized medical procedures and surgery	55	100
3. Whites with HIV or AIDS are more likely than some minorities with HIV or AIDS to get the newest medicines and treatments	45	97
Reflection/Empowerment
4. It is important for physicians to devote extra time to the health needs of their minority patients	51	73
5. I often think about what I can do to interact more effectively with my minority patients	50	88
6. I am in a position to make a difference in the quality of health care that minority patients receive	75	100
	% yes	% yes
Action
7. In the last month, I have spoken with colleagues about ways to address specific health care needs of minority patients	32	94
8. In the last month, I have worked with a community group to address a local health problem	15	46
9. In the last month, I have participated in a quality improvement project at my place of work to increase quality of care for minority patients	43	42

On bivariate analyses, there were small but statistically significant differences in engagement score based on physician characteristics. For example, nonwhite physicians scored slightly higher than white physicians (mean score 19.7 versus 18.0, p = .01) and women scored slightly higher than men (19.2 versus. 18.1, p = .01).

Pairwise Correlations of Scale Domains

If the AREA conceptual model of engagement is correct, we would expect higher correlations between domains, or stages of engagement, that are proximal to each other. So, for example, Awareness should be more closely correlated with Reflection and Empowerment than it is to Action. To test this, we examined Pearson's correlations between each domain after summing the original scores of all the items within each domain. The results were as follows: corr(Awareness, Reflection/Empowerment)=0.34, corr(Reflection/Empowerment, Action)=0.29, and corr(Awareness, Action)=0.13. In addition, the partial correlation between Awareness and Action, after removing the effect of Reflection/Empowerment, is only 0.03. These results are consistent with the conceptual model.

Validation in Highly Engaged Physician Sample

Finally, we sent the nine-item survey to 32 national experts in addressing health disparities, all of whom completed it. Among these respondents, all were practicing clinicians, while the mean age was 43 years, 66 percent were male, 29 percent were white, and 44 percent were African American. The mean engagement score of these national experts was 29 (compared with 18.4 in the general physician population, p<.001) the median score was 29, and the range was 20–35 (Figure 1 and Table 2).

DISCUSSION

Health care leaders have suggested that physicians should play an important role in helping to address health disparities (Institute of Medicine Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care 2003; Hargraves 2004; Agency for Healthcare Research and Quality 2005), yet there is no widely accepted understanding of what it means for physicians to be “engaged” in addressing an issue like health care disparities. Here, we report the development of a conceptual model of engagement, which we have named the AREA model, as an acronym for the stages that we hypothesized physicians would pass through as they become increasingly engaged: Awareness, Reflection/Empowerment, and Action. Recognizing that individual physician's engagement can occur at both individual and organizational levels (Gruen, Pearson, and Brennan 2004), the AREA model accommodates engagement at both levels.

Using this conceptual model, we developed a simple nine-item scale to assess physicians' engagement in addressing disparities. When applied to a national sample of physicians, this scale produced a normal distribution of results, suggesting that the scale can detect varying levels of engagement. When applied to a selected sample of physicians who are working to address health disparities through teaching, research and advocacy, the scale produced generally very high scores, suggesting that it can reliably detect a group of highly engaged physicians.

The nine-item AREA scale may help researchers answer several relevant questions. Do certain physicians (urban versus rural, minority versus non-minority) differ in their depth of engagement in addressing disparities? If so, how large are these differences and what factors mediate this association? What is the trend in physician engagement over time or across different institutions? What are the effects of various interventions to increase physician engagement? It is even possible that interventions could be designed to target-specific domains of the model, based on the results seen in a specific population. A group that is unaware of disparities might benefit from an awareness campaign or a program to gather local data to demonstrate the existence of disparities, for example, while those who feel disempowered to affect change might benefit from hearing success stories or learning about how to use quality improvement projects to alleviate disparities.

This preliminary validation study has a number of limitations. First, the scale was designed and tested as a means of assessing the level of engagement among groups of practicing physicians. It is not designed as a test of engagement at the individual level, nor is it designed for nonclinicians. Second, the statistical analyses that we conducted require assumptions, such as the assumption of multivariate normal distribution, which were not met by all of our data. Statistical validation of the scale on independent samples of physicians will be important. Third, the optimal method of scoring the nine-item scale is not known, and all scales such as the Likert scales that we used are limited in that they do not provide a real representation of the psychological or subjective distance between different ordinal responses (Lin and Tang 1995). Further refinements in item and response frame design are likely to improve the scale. Finally, it is not clear that the scale will work as well if respondents are aware of its purpose. Our respondents were aware that they were being asked about “quality of care for minority patients”; it is possible that respondents would provide socially desirable responses if they knew the survey was focused on addressing health disparities.

CONCLUSION

A nine-item survey can be used to assess the level of engagement in addressing health disparities among groups of physicians. This simple tool might help target efforts to improve physician engagement in a group or at the institution level and monitor such efforts over time.

SUPPLEMENTARY MATERIAL

The following Supplementary material for this article is available online:

Appendix A

Members of Commission to End Health Care Disparities and Members of the Data Gathering Committee under Which This Work Was Completed.

Appendix B

The AREA Scale of Physcian Engagement.

This material is available as part of the online article from: http://www.blackwell-synergy.com/doi/abs/10.1111/j.1475-6773.2007.00780.x (this link will take you to the article abstract).

Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.