Assessing Applicant Personality in Residency Selection

Most people say that it is the intellect which makes a great scientist. They are wrong: it is character.

—Einstein

A web-based review of multiple residency websites for US categorical pathology residency training programs shows that most are National Residency Matching Program (NRMP) based and require an Electronic Residency Application Service (ERAS) application, curriculum vitae, 3 letters of recommendation, a Dean’s letter, Educational Commission for Foreign Medical Graduates (ECFMG) status, United States Medical Licensing Examination (USMLE) scores, a transcript, and personal statement. The webpage generally contains a statement that “we do not discriminate based on age, color, disability status, national origin, race, religion, or sex.” From an outside applicant’s view, most programs appear generic with much mystery surrounding the creation of the programmatic rank list. Programs recognize that the ERAS package of application materials are part of a larger assessment process. Programs are beginning to look more closely at personality and best-fit, based on specific attributes relative to the specialty. General surgery residencies in the United States, for instance, have suffered from high attrition rates (14%-32%). ¹ Using personality test results of successful surgical faculty compared to program applicants may help to fine-tune the rank list into a more useful predictor of specialty satisfaction and training program completion for surgery. ¹ This is one example of the use of personality data in rank list creation. Pathology, as a specialty, has not yet determined the exact personality traits that lead to career success, but it can be useful to know which traits will help determine success for a particular program or institution. Medical schools have long administered personality tests to help determine specialty fit, but the specialty of Pathology has suffered from negative stereotypes, sometimes suggesting neurotic introverts are the best fit. ^2,3

As a program director, I believe the paper by Prystowsky et al ⁴ is an important addition to the literature of Pathology residency recruitment. The challenges described with program rank order list creation are universal, including the sentiments, such as “the ability to communicate with patients and peers is critical for success” and “we believe that all of the candidates whom we (select) to interview are academically acceptable.” But, how do we define and enumerate criteria for cultural, personality, institutional, and specialty features? And how do we gauge that in a limited, restrained, and highly managed algorithmic system? How can we improve the small piece of the process that we have influence over for our own programs? Without clear guidance in terms of culture, identity, and program brand, “fit” risks being interpreted as “similarity to the status quo” and therefore may actually threaten diversity. ⁵

The authors are bestowed with a strong sense of mission-identity by their institutional namesakes, Montefiore and Einstein. Montefiore demands a strong sense of “social justice” and “to care for the underserved” as institutional priorities. Einstein wrote “the important thing is to never stop questioning.” ⁶ These values are current and relevant. The second, possibly easier to assess in applicants. How do we gauge empathy in an interview? The program’s location in the borough of the Bronx, New York City, lends itself well to cultural diversity. An applicant who applies here may be showing they value diversity simply through the act of applying. The literature supports that diversity begets diversity. ⁷

The authors attempt to quantify, qualify, search social media sites, color code, analyze, and rank the candidates in a way that considers the full array of ERAS-provided data as well as the personality assessment inventories and other attributes important to this program, such as attributes of “Highly Reliable Teamwork.” The ERAS data were mined by the program director and associate program directors. Applicants were given scores for their academic performance, letter of references, Dean’s letter, and USMLE scores. Program cultural fit, as best assessed by these data, was a part of this preinterview composite score. The authors offer comment in the discussion section on the devalued Dean’s letter, such that it is not as relevant for ranking as it used to be. A few criteria are also offered as automatic exclusion criteria, such as “dismissed from training elsewhere.” The interview process generates a second ranking into 4 groups. The composite preinterview score is then compared to the interview rank, where personality traits were assessed more closely. The interview evaluation utilizes 4 color groups from blue and green (best groups) to red (worst group). Within each of the 4 interview groups, the final rank order is determined. The red group is scrutinized carefully and likely excluded from ranking, while the two best categories are likely included. Outcome criteria are also statistically considered, for eventual job outcomes. It is interesting to see that, while green and blue category candidates typically were now in faculty positions at leading academic and medical research centers, candidates in the yellow and red candidates did reasonably well; a higher percentage of private practice graduates were from the yellow category. This is not a statement that one outcome is better than the other.

The description of the importance of personality assessment as a reinforcement of the preinterview data is something that most program directors understand well, and the authors provide examples of two hypothetical candidates. Both were within the 10% to 15% of applicants selected for an interview, based on ERAS review. On interview day, one is humble and cooperative and the other is pushy and aggressive. This exemplifies the importance of the interview day for full personality discovery. Hopefully this will not be lost in the virtual realm, an area that will still need to be analyzed. These authors report an emphasis on implicit bias training as preparation of faculty for the virtual interviews. This is likely a good idea for any interview faculty, but not particularity more necessary virtually than for in-person interviews.

The list of character personality traits utilized (resilience, conscientiousness, good interpersonal skills, etc) would likely be important to most programs. Although the list of personality traits is very comprehensive and well selected for this program, the difficulty may be in how the faculty ascertain the candidate’s personality through questions, in a time-limited interview. The examples given of interview questions, such as “What arts do you enjoy? Give an example of a memorable pathology case; and What extracurricular activities do you enjoy?” are fairly open ended. The use of structured interview questions with grading rubrics may add some evaluative depth to these interview sessions. The use of a structured question can help the interviewer eliminate bias from the process and more fairly assess for a particular trait. ⁸ In the end, the statistics prove that these faculty agree well with themselves, pre- and post-interview, albeit admitting the lower categories (preinterview composite score and interview color category) agreed very well, endorsing that “we may like something, but are surer of what we do not care for.”

The new front of residency interviews is the holistic interview, defined by the American Association of Medical Colleges (AAMC), and being adopted in medical school selection as well as residency, as “a flexible, individualized way of assessing an applicant’s abilities, by which balanced consideration is given to experience, attributes, competencies, and academic or scholarly metrics, and when considered in combination, how the individual might contribute value to the institution’s mission.” ⁹ While part of holistic recruitment is contained within the ERAS-provided data, that is, the screening step, other parts, attributes such as personality, attitude, team player, integrity, and so on, can be assessed from the interview, but require a fair and balanced systematic approach. A holistic approach requires that programs link selection to their mission and goals, as well as acknowledge that diversity and inclusion are essential to excellence, translated to some type of ranking numeric. We need to examine the culture of our programs and the personalities and root causes of past trainee failures, as well as develop methods of numeric weighting of nonnumeric data. Of course, we can only recruit for diversity if the pool of applicants is diverse. Although Black/African American and Hispanic/Latino physicians make up 13% and 17% of the US population (respectively), they each make up 4% of US physician workforce. Holistic recruitment for residency is, for US graduates, downstream to medical school recruitment. ¹⁰ In pathology, we have overlapping problems with low US graduate recruitment to our specialty.

This article is an advance in the field of recruitment for pathology programs and an example of how a program might begin such a process. The authors have succeeded in dividing the process into preinterview review, interview, and ranking. The ranking is unified and mission driven, specific for their institution and transparent to scrutiny. I wonder if the preinterview ranking is not numerically burdensome, simply to end up where we would have more effortlessly, that is, knowing what we don’t want. Perhaps structured questions could improve this process. Perhaps we are implicitly biased toward extroverts despite implicit bias training, while introverts quietly and successfully serve large communities in private settings rather than academic setting. Perhaps in the end, it is best that programs are only a part of the process, downstream to applicants. If we do not examine this process, we cannot hope to improve it. Most importantly, we must understand that physicians need to have values that enable them to have compassion for the diverse populations they serve. Quantifying those values in the candidates we interview is the challenge ahead.