Personality Testing May Improve Resident Selection in Anesthesiology Programs

Abstract

Background

Current methods of selecting future residents for anesthesiology training programs do not adequately distinguish those who will succeed from the pool of seemingly well-qualified applicants. Some residents, despite high exam scores, may struggle in the OR in stressful situations.

Aims

This study examined whether specific neuropsychological and personality measures can distinguish high competency residents from low competency residents to aid in resident selection.

Methods

25 residents enrolled in an anesthesiology program at a major academic institution were identified for participation. 13 were evaluated identified as “high competency” residents and 12 as “low competency ” by the department's clinical competency committee. Groups were evaluated on measures of fine motor dexterity, executive functioning, processing speed, attention, and personality using IPIP-NEO.

Results

There were no significant differences between groups on measures of fine-motor dexterity, executive functioning, processing speed, or attention. High competency residents scored significantly higher than low competency residents on measures of cooperation, self-efficacy, and adventurousness, and lower on measures of neuroticism, anxiety, anger, and vulnerability.

Conclusion

Although measures of fine-motor dexterity, executive functioning, processing speed, and attention do not appear to distinguish between high- and low competency residents in anesthesiology, specific personality characteristics may be associated with future success in an anesthesiology training program.

Introduction

The selection of residents for an anesthesiology training program is a complicated, expensive, and time-consuming process. It traditionally involves a preliminary review of the United States Medical Licensing Examination (USMLE) scores, medical school performance, participation in research projects, volunteer work, Dean's letters, and letters of recommendation, followed by on-site interviews. Although this process can become burdensome, it is very important to both the long-term quality of anesthesia providers being trained and the amount of effort involved in their training. Residents who struggle with training requirements create extra work and expenses for their programs. They may also be more likely to contribute to critical incidents during clinical care or to cause other problems within the training program.

However, the complex system of resident selection does not provide enough information about how resident applicants will perform in the high stress anesthesiology training environment (Metro et al., 2005), or why applicants have chosen anesthesiology as their intended specialty (Wass et al., 2003). Previous research has demonstrated that applicants who match with a specific anesthesiology residency program do not differ from their nonmatching peers on test scores, medical school grade point average, or class rank (Baker et al., 1993). In addition, standardized test results and undergraduate grade point average are not related to performance during residency (Warrick & Crumrine, 1986). Rather, nontechnical skills (Fletcher et al., 2002) such as conscientiousness, confidence, and composure, which are generally not assessed during the resident selection process, may be the best predictors of a resident's future clinical performance (Rhoton et al., 1991). Lack of these skills may also predict ineffective behaviors in response to a critical incident (Altmaier et al., 1997).

Given these findings, recent studies have focused on identifying cognitive and personality factors that might contribute to the prediction of a resident's success in an anesthesiology training program. Gough and colleagues (Gough et al., 1991) found that scores on certain subscales of the California Psychological Inventory (CPI) and the Strong Interest Inventory correlated with performance two years later. Later research by the same group confirmed the utility of the CPI for this purpose (McDonald et al., 1994). In addition, Reich et al (Reich et al., 1999) demonstrated that a divided attention task, visual target detection, and certain subscales of the CPI were associated with poor clinical performance among residents.

Although it has demonstrated concurrent and predictive validity for anesthesia resident selection, the CPI is somewhat burdensome to administer. It contains 462 items and is not available in the public domain. This study was conducted to see if a battery of neuropsychological and personality tests, that would be quick and easy to administer and score, would be cost-effective and assist in resident selection in an anesthesiology training program.

Methods

Participants

Residents who were identified by the residency clinical competency committee as exhibiting either high levels of competency or low levels of competency were invited to participate in the research. Participants were 26 residents enrolled in an anesthesiology training program at a major academic institution who were selected out of more than 80 residents in the program. The rest of the residents were performing on an average level. Participants were not informed of their group category or the reason for their inclusion in the study. The residents ranged in age from 27 to 49 years of age (M = 33.7 years, SD = 5.1), and were currently in their first (n = 3), second (n = 9), third (n = 6), or fourth (n = 7) year of residency training. Thirteen of the participants were identified by the department's clinical competency committee as “high competency” residents, while the other 13 were evaluated as “low competency” in areas of training. Evaluation of residents included scores in five main domains: patient care skills, medical knowledge, professionalism, interpersonal skills, and communication skills. The scores were based on a scale of 1-4, with 1 being the highest score and 4 the lowest. Those residents who scored 3 or higher in 4 out of 6 domains were placed in the “low competency” category; residents scoring 2 or lower in 4 out of 6 domains were placed in the “high competency” category. In addition, results of mock oral examinations and in-service examinations were taken into consideration.

Measures

Fine Motor Dexterity

Motor skills were assessed using the Finger Tapping test (Reitan & Wolfson, 1985) and the Grooved Pegboard test (Reitan & Davison, 1974).

Executive Functioning

Ability to rotate between two tasks simultaneously was assessed using the Trail-Making Test (Reitan & Wolfson, 1985).

Processing Speed

Speed of processing was assessed using the Symbol Search and Coding subtests of the Wechsler Adult Intelligence Scales, Third Edition (WAIS-III) (Wechsler, 1977). The Symbol Search subtest of the WAIS-III is used to measure processing speed via visual processing efficiency.

Attention

Sustained attention was assessed using the Conner's Continuous Performance Test, Second Edition (CPT-II) (Conners, 2000), a computerized task.

Personality

Scores on the Five-Factor Model of Personality (i.e., extraversion, agreeableness, conscientiousness, neuroticism, openness to experience) and underlying personality facets were assessed using the International Personality Item Pool Representation of the NEO PI-R (IPIP-NEO) (Goldberg et al., 2006). The IPIP-NEO is a 300-item personality questionnaire that was designed to assess the five broad domains and 30 subdomains of personality. It is administered online and yields percentile scores for each facet of personality measured. The IPIP-NEO can be completed in approximately 20-30 minutes.

Procedure

All procedures were approved by the University of Florida Institutional Review Board, and informed consent was obtained. All 25 residents were informed in detail about this study and agreed to participate voluntarily. Residents remained blinded to their group assignment and the study hypotheses. Each resident was instructed to complete the online version of the IPIP-NEO independently and to print their results anonymously. Next, residents in each group were administered a battery of neuropsychological tests by a trained psychometrician who was not affiliated with residency training in anesthesiology, and was blinded to the category of the participants (i.e., “high competency” or “low competency”). Neuropsychological testing lasted approximately 30 minutes.

Data Analysis

Scores for each test were standardized (e.g., z-scores, T-scores, standard scores, or percentile scores) before analysis. Scatterplots were then created for each variable to assess for outliers. Data from one participant in the low competency group were dropped due to a pattern of outlier responses indicative of socially-desirable responding on the IPIP-NEO. Next, independent sample t-tests were computed for each study variable to assess for group differences among the high competency and low competency groups. Given the small sample and the exploratory nature of the study, the significance level was set at p < .10 to minimize the likelihood of Type II error.

Results

Neuropsychological testing indicated no significant group differences related to fine motor coordination, executive functioning, or processing speed. Similarly, CPT data indicated no significant differences between groups in sustained attention.

Analysis of the personality scores yielded several significant results consistent with our hypotheses (Table 1). Specifically, individuals in the high competency group scored significantly higher than individuals in the low competency group on three subscales assessing positive characteristics, and they scored lower than the low competency group on four subscales assessing negative characteristics.

Table 1. Scores for High Competency (n = 13) and Low Competency (n = 12) Groups on Relevant Study Variables.

MEASURE	Low Competency Group		High Competency Group		Sig
	Mean	(SD)	Mean	(SD)
IPIP-NEO Cooperation	52.44	(23.46)	75.36	(18.26)	†
IPIP-NEO Self-Efficacy	46.78	(28.29)	73.00	(21.98)	†
IPIP-NEO Adventurousness	33.38	(19.26)	53.00	(21.62)	*
IPIP-NEO Neuroticism	40.25	(17.43)	19.63	(17.39)	†
IPIP-NEO Anxiety	55.22	(19.95)	24.64	(25.68)	‡
IPIP-NEO Anger	55.00	(28.09)	21.45	(18.54)	‡
IPIP-NEO Vulnerability	48.89	(18.72)	24.73	(12.72)	‡

^*Note: p < .10;

^†Note: p < .05;

^‡Note: p < .01

High competency residents scored higher on cooperation (t = 2.46, p = .02), indicating a higher level of compliance and lower likelihood of intimidating others to get their way. Given that the scoring instructions for the IPIP-NEO describe scores as falling into “Low” (i.e., bottom 30%), “Average” (i.e., middle 40%), or “High” (i.e., top 30%) scores, all high competency residents scored average or above on this subscale (i.e., all scored greater than 40^th percentile). They also scored higher on self- efficacy (t = 2.34, p = .03), with all except one of the high competency residents scoring above the 50^th percentile. These results indicate that the high achievers generally display greater confidence in their ability to accomplish tasks. On the adventurousness subscale, the high competency group again displayed a higher mean score than the low competency group (t = 1.94, p < .08), indicating a greater eagerness to try new things. In fact, all low competency residents scored at or below the 55^th percentile on the adventurousness subscale.

Individuals in the high competency group scored significantly lower than their low-competency counterparts on the neuroticism subscale (t = 2.51, p = .02), indicating less emotional reactivity among the high competency group. All high competency residents scored in the average range or lower on this subscale (i.e., less than 60^th percentile). They also displayed lower mean scores for anxiety (t = 2.92, p = .009), indicating less tension and anxiety among high competency residents. All except one of the high competency residents scored in the average range or below (i.e., below the 60^th percentile). Anger scores were also lower for the high competency group (t = 3.21, p < .005), indicating less likelihood of becoming upset during an untoward incident. All high competency residents scored at or below average on this subscale (i.e., below the 60^th percentile). The high competency residents scored lower on the subscale assessing vulnerability (t = 3.43, p < .003), indicating a lower likelihood of feeling panicked, confused, or helpless when under pressure or stress. In addition, all high competency residents scored at or below average (i.e., less than 60^th percentile) on this subscale.

Finally, although the group differences were not significant, it is noteworthy that the residents in the high competency group all scored above the 60^th percentile on cautiousness (i.e., tendency to think through possibilities before acting) and above the 50^th percentile on conscientiousness (i.e., tendency to set goals and pursue them with determination). In addition, the high competency group scored below the 50^th percentile on self-consciousness (i.e., concerns about rejection; feeling awkward/uncomfortable around others).

Discussion

Current selection procedures for anesthesiology residency programs are generally quite subjective and may provide suboptimal outcomes. In virtually every training program, there are residents who fall behind and require extra time and attention from faculty (Slogoff et al., 1994). It is unclear whether these residents will eventually achieve the necessary level of competence as an anesthesia provider. Though programs often extend the training period for these residents, the long-term results of the remediation process are unknown. Instead, these individuals, who are generally very talented physicians with strong credentials, may actually perform better and be more adept in a specialty other than anesthesiology. Unfortunately, though the interview may be helpful in assessing interpersonal concerns among applicants, there is currently no standard personality evaluation utilized in the selection process.

Consistent with prior research (Gough et al., 1991; McDonald et al., 1994; Reich et al., 1999), the results of the present study support the hypothesis that personality factors, such as confidence, conscientiousness, adaptive social skills, and mental well-being, are associated with success in clinical anesthesia. We used the International Personality Item Pool (IPIP-NEO) (Goldberg et al., 2006) to assess personality characteristics. The use of the IPIP-NEO has advantages over the use of other measures in that it is free and available in the public domain. In addition, this measure can be administered online and scored immediately by the computer. The IPIP-NEO is also shorter than many other popular personality measures (e.g., California Personality Inventory, Minnesota Multiphasic Personality Inventory, etc.), and can be self-administered in approximately 20-30 minutes. As with all self-report measures, measurement error, misunderstandings, carelessness, and mischievous responses can invalidate the IPIP-NEO report. However, test results should never be used for decision-making in isolation of other available data.

Performance on measures of fine motor dexterity, executive functioning, processing speed, and sustained attention did not discriminate between high competency and low competency residents in the present study. This suggests that including measures of these qualities during the application process would not incrementally improve the selection of anesthesiology residents.

Although the results of the current study are encouraging, some important limitations should be noted. For example, the sample was small, and the ability to generalize the results remains unclear. Similarly, the present study reflects a preliminary attempt to identify factors that may distinguish high competency residents from low competency residents in an anesthesiology training program. As a result, future research is needed to cross-validate the findings. In addition, research is needed to determine the predictive ability of these measures by testing all residents prior to beginning their residency and correlating their results with future measures of performance throughout their training.

Despite these limitations, the results of this study (Table 2) support the use of personality testing as an adjunct to other methods of resident selection. Applicants scoring high on IPIP-NEO measures of cooperation, self-efficacy, achievement-striving, cautiousness, and conscientiousness likely have characteristics that will help them succeed in an anesthesiology residency program. On the other hand, those scoring high on measures of neuroticism, anxiety, anger, and vulnerability may struggle in such a program.

Table 2. Attributes Associated with High or Low Achievement Among Residents.

Personality Attribute	Group Category Associated with High Scores
Cooperation: Self-reports high levels of compliance and low likelihood of intimidating others to get their way.	High Competency
Self-Efficacy: Self-reports confidence in their ability to accomplish things.	High Competency
Adventurousness: Self-reports an eagerness to try new things.	High Competency
Cautiousness: Self-reports a tendency to think through possibilities before acting.	High Competency
Conscientiousness: Self-reports a tendency to set goals and pursue them with determination.	High Competency
Neuroticism: Self-reports strong emotional reactivity in response to situations.	Low Competency
Anxiety: Self-reports experiencing significant tension and anxiety.	Low Competency
Anger. Self-reports a tendency to becoming upset when things do not go their way.	Low Competency
Vulnerability: Self-reports a tendency to feel panicked, confused, or helpless when under pressure or stress	Low Competency

Practice Points.

• Current selection procedures for anesthesiology residency programs are generally subjective. No standard personality evaluation is presently utilized.

• Personality factors, such as confidence, conscientiousness, adaptive social skills, and mental well-being may be the best predictors of future clinical performance and success in clinical anesthesia.

• The results of this study support personality testing as an adjunct to other methods of resident selection to distinguish high competency from low competency residents.