Assessing Correlations of Physicians' Practice Intensity and Certainty During Residency Training

Abstract

Background

Variation in physicians' practice patterns contributes to unnecessary health care spending, yet the influences of modifiable determinants on practice patterns are not known. Identifying these mutable factors could reduce unnecessary testing and decrease variation in clinical practice.

Objective

To assess the importance of the residency program relative to physician personality traits in explaining variations in practice intensity (PI), the likelihood of ordering tests and treatments, and in the certainty of their intention to order.

Methods

We surveyed 690 interns and residents from 7 internal medicine residency programs, ranging from small community-based programs to large university residency programs. The surveys consisted of clinical vignettes designed to gauge respondents' preferences for aggressive clinical care, and questions assessing respondents' personality traits. The primary outcome was the participant-level mean response to 23 vignettes as a measure of PI. The secondary outcome was a certainty score (CS) constructed as the proportion of vignettes for which a respondent selected “definitely” versus “probably.”

Results

A total of 325 interns and residents responded to the survey (47% response rate). Measures of personality traits, subjective norms, demographics, and residency program indicators collectively explained 27.3% of PI variation. Residency program identity was the largest contributor. No personality traits were significantly independently associated with higher PI. The same collection of factors explained 17.1% of CS variation. Here, personality traits were responsible for 63.6% of the explained variation.

Conclusions

Residency program affiliations explained more of the variation in PI than demographic characteristics, personality traits, or subjective norms.

What was known and gap

Prior research has attributed variations in physicians' practice patterns to the residency program in which they trained, yet the influences of modifiable determinants of practice patterns are not known.

What is new

A survey of internal medicine residents showed the residency program to be the largest contributor to variations in resource use.

Limitations

Response rate of less than 50% creates the potential for respondent bias; survey tool lacks validity evidence.

Bottom line

Residency programs can leverage their influence over residents' resource usage patterns to improve the value of care provided by their graduates.

Introduction

A report from the Institute of Medicine (IOM) estimates that approximately 30% of health care spending is for care that is unnecessary.¹ Physician choices are behind much of this waste, but the origins of variation in physician practice patterns are complex. While a sizeable literature links this variation to geography, another IOM report suggests that practice variation is as large within regions as across them.² Since it is not easy to overcome variations explained by geography, a better understanding of the determinants that are modifiable is necessary. Those determinants might be found among physicians' training, experience, practice environment, and/or personal characteristics, and may range from more changeable to less changeable.

Training variation is an appealing factor to investigate because it is mutable, and previous research suggests that it is influential. For example, the residency program at which a physician trained is predictive of patient outcomes even after the physician graduates³; exposure to conflict of interest policies during residency has been associated with subsequent prescribing patterns⁴; and the spending intensity of the region of residency training has been associated with the intensity of physician spending after training.^5,6 Medical educators believe that graduate medical education influences the later practice patterns of its learners,^7,8 thus creating an opportunity to inculcate residents the principles of cost-consciousness and resource stewardship. Accordingly, residency programs have the potential to reduce unnecessary testing and variation in clinical practice not only during training, but also throughout a physician's career.

While these models sound promising, systematic research on the formation of physicians' practice patterns is scarce, and other factors, like personal traits, may also play a major role. We therefore conducted a multicenter study of residents at 7 internal medicine (IM) residency programs in a single metropolitan area. The goal of the study was to assess the importance of the residency program relative to individual physician characteristics (including demographics, attitudes, psychological traits, and perceived behavior control) in explaining variations in resident physicians' likelihood of ordering tests and treatments and in the certainty of their intention to order.

Methods

Participants

Seven internal medicine (IM) residency programs in the Philadelphia metropolitan area participated in the study (Crozer-Keystone Health System, Drexel University, Lankenau Medical Center, Pennsylvania Hospital, Temple University, Thomas Jefferson University, and the Hospital of the University of Pennsylvania), ranging from small community-based programs to large university residency programs. All preliminary, transitional, and categorical IM interns and residents were invited to complete a survey in March 2014. Eligible participants received an e-mail invitation, with weekly reminders for 1 month. The e-mail contained a link to the web-based survey (Qualtrics LLC). Survey participants were entered into a lottery to win 1 of 2 $500 gift cards (selected from the first 100 respondents) or 1 of 4 $300 gift cards (from the remaining respondents).

Vignettes

To determine respondents' average practice intensity, we assembled 34 clinical vignettes designed to gauge preferences for aggressive clinical care. From the Health Tracking Physician Survey,⁹ we drew 6 vignettes relevant to general medicine. We developed 28 additional clinical vignettes describing situations where ordering a test or treatment did not reflect high-value care based on the Choosing Wisely Campaign¹⁰ and a literature review.¹¹ They could be answered as “definitely yes,” “probably yes,” “probably no,” and “definitely no.” A total of 7 medical education, instrument design, and high-value care experts reviewed these vignettes prior to piloting them with 23 IM faculty. The extent of concentration across responses was assessed using the Simpson index,¹² calculated as the sum of the square of each response level's share of respondents. With 4 response categories, this index ranges between 0.25 (equal distribution of answers across response levels) and 1 (all answers in a single response category). We excluded 11 vignettes with a Simpson index value greater than 0.60. The remaining 23 vignettes (4 from the Health Tracking Physician Survey and 7 designed by the authors) spanned diagnostic testing (n = 13), request for consultation (n = 2), and treatment (n = 8), and were predominately outpatient based (n = 13).

Attitudes and Psychological Traits

We used several scales with evidence of reliability and validity in other contexts to assess physician attitudes, psychological traits, and perceived behavior control that may influence physicians' intention to order a test or treatment.^13–21 The Risk Aversion Scale measures a physician's attitude toward risk-taking in 6 questions rated on a 6-point scale from “strongly agree” to “strongly disagree.”^18,19 A higher score suggests risk aversion and has previously been associated with higher resource utilization.¹⁹ Two domains from the revised Physicians' Reaction to Uncertainty Scales measure stress related to uncertainty (5 questions) and related to concern about bad outcomes (3 questions), also rated on a 6-point scale from “strongly agree” to “strongly disagree.”^15,16 A higher score is indicative of stress related to uncertainty or fear of bad outcomes, and has also been linked with higher resource utilization.²² The Big Five Inventory²¹ measures personality traits (openness, conscientiousness, extraversion, agreeableness, and neuroticism) with 10 questions using a 5-point scale ranging from “disagree strongly” to “agree strongly.” The Core Self-Evaluation scale measures self-perceived ability (self-efficacy and self-esteem), neuroticism, and perception of locus of control with 12 questions using a 5-point scale that ranges from “disagree strongly” to “agree strongly.”¹⁷

The theories of reasoned action and planned behavior²⁰ posit that many behaviors can be predicted by a person's intentions,^20,23,24 which are influenced by expected attitudes and perception of related subjective norms. On this basis, we created 11 additional questions to assess subjective norms (“Attending physicians at my institution pay attention to the number of tests ordered,” with answer choices of “yes,” “no,” or “I don't know”); behavioral intentions (“I only order tests that will change patient management”); and self-perceived knowledge related to high-value care (“I know where to find how much my patients will be billed for tests”) on a 5-point scale ranging from “strongly disagree” to “strongly agree.” We also collected basic sociodemographic data.

Main Outcomes

The primary outcome was the participant-level mean response to the 23 vignettes as a measure of practice intensity (PI). The secondary outcome was a certainty score (CS) constructed as the proportion of vignettes for which a respondent selected “definitely” versus “probably.”

The protocol was approved by the Institutional Review Boards at all sites.

Data Analysis

Cronbach α statistic was calculated for the PI and CS outcomes, as was the PI-CS Pearson correlation coefficient. Linear regression models predicting each outcome as a function of 4 blocks of explanatory variables (personality traits, perceptions of subjective norms, demographics, and residency program indicators) were estimated. A Shapley-Owen decomposition^25–27 of the R² value from each model was conducted to assess how much of the total explained variation was attributable to each block of variables and each of the specific personality dimensions. All analyses were performed using Stata version 13.1 (StataCorp LP). P < .05 was considered statistically significant.

Results

Of the 690 residents surveyed, 325 (47%) responded. Their characteristics are shown in table 1. Median survey completion time was 17.7 minutes (interquartile range [IQR] = 13.5–26.7).

TABLE 1.

Demographics of Survey Respondents (N = 325)

The mean PI was 2.52 with a SD of 0.31, where a higher score (maximum possible score of 4) indicates a more intense practice style (median 2.48, IQR = 0.39). The mean CS was 0.36 with a SD of 0.20 (median 0.35, IQR = 0.30), where a higher score implies that the respondent was more certain in his or her answer. Cronbach α was 0.68 for PI and 0.78 for CS. The PI-CS Pearson correlation coefficient was −0.045 (P = .42). Scores for each of the determinants of behavioral intentions in our study are shown in table 2.

TABLE 2.

Determinants of Behavioral Intention

The 4 blocks of explanatory variables collectively explained 27.3% of the variation in the PI. Table 3 reports the association of each of the personality elements with PI along with its decomposed contribution to the total predictable variation. Considering 100% of the explained variation, personality traits overall contributed 10.1% of the explained variation, with that effect dominated by the neuroticism scale (5.7%), where respondents with a higher score for neuroticism had significantly more aggressive practice patterns after controlling for the other factors. Subjective norms of the institutional culture contributed 9.7% of the explained variation. Demographic characteristics contributed 33.5%. Women and international medical graduates had significantly more aggressive practice styles, and residents were less aggressive than interns. Residency programs accounted for the remaining 46.7% of the explained variation.

The explanatory variables collectively explained 17.1% of variation in the CS. Considering 100% of the explained variation, personality traits contributed 63.6% (table 3), led by core self-evaluation (27.4%). Subjective norms of the institutional culture contributed 11.9% of the explained variation, while demographics contributed 16.1%. Women were, on average, less certain about their answers. The CS scores did not significantly differ between interns and residents. Residency programs accounted for the remaining 8.4% of the explained variation.

TABLE 3.

Percentage of Total R² Attributable to Determinants of Behavioral Intentions

Discussion

We were able to create a measure of practice intensity with substantial discrimination across individuals. While further work is required to test the association of this measure (derived from responses to hypothetical vignettes) to actual practice intensity, the measure has some evidence in support of construct validity. We also developed a measure of physicians' confidence in practice choices that lends further insight into practice variation. Notably, the correlation between practice intensity and certainty was negligible, suggesting each captures a distinct aspect of clinical decision making.

Second, while measurable personality traits (particularly neuroticism) contribute to our understanding of variation in practice intensity, experience and sex contribute more, and the residency program contributes the most. The contribution of the residency program suggests that practice intensity is principally created by the socialization that occurs within training. Previous work reveals that the academic orientation of a physician's medical school predicts future resource utilization, with academic environments fostering more intensive use of diagnostic services,²⁸ while other work suggests that physicians who trained at academic medical schools and/or residency programs, in general, order fewer tests except when confronted with uncertainty.²⁹ Similarly, the spending pattern of the geographic region where a residency program is located has been associated with the spending pattern of its graduates, regardless of their later practice site.⁵ The residency programs in this study were all located within the same metropolitan area; however, further underscoring that within-region variation is substantial and important.³⁰ Future investigation should focus on identifying specific differences between residency programs that drive this variation in an effort to influence physician practice patterns as they develop.

Third, while practice intensity is most associated with a training program, certainty is most associated with personality. To the extent that personality characteristics are less mutable, strategies to reduce practice intensity might need to bypass personality characteristics. Further exploration is needed to understand how to modulate individuals' practice intensity at the systems level.

This study has limitations. First, physician practice style was measured using hypothetical clinical situations. Vignettes have been used successfully in other settings to assess physicians' quality of care,^31,32 and they help isolate individual clinician impact from what, in the real world, are often team decisions. Second, the portions of the survey that measure subjective norms did not undergo testing for validity. Third, our model explained only 27.3% of the total variation in the mean physician-level practice intensity score, suggesting the presence of other factors that drive practice. Fourth, our sample includes only 7 residency programs in 1 metropolitan area, limiting generalizability. Fifth, the response rate of 47%, while high for a survey of resident physicians, leaves open the possibility that nonresponders differed systematically from responders.

Conclusion

Internal medicine trainees' residency program affiliations explained more of the variation in their practice patterns than demographic characteristics, personality traits, or subjective norms. The combination of demographics and the residency program identity explained a large proportion of observed variation, suggesting residency programs play an important role in influencing physician practice patterns during their formation. Residency programs can leverage this influence to improve the value derived from physicians' practice patterns after training is complete.