Abstract
Background
Although postinterview communication (PIC) guidelines exist, adherence is voluntary. There are no studies of PIC practices in critical care medicine (CCM) and pulmonary and critical care medicine (PCCM) fellowship recruitment.
Research Question
What is the frequency, format, goals, and content of PIC between CCM/PCCM applicants and program directors? What is the impact of PIC on applicant and program rank order lists (ROLs)?
Study Design and Methods
CCM/PCCM applicants and program directors were separately surveyed after the 2022-2023 National Resident Matching Program Specialty Match. Surveys included multiple-choice, Likert-scale, and two free text questions. Thematic content analysis of free text responses was performed.
Results
One-third of eligible participants responded (applicants: n = 373 [34%]; program directors: n = 86 [32%]). Applicant respondents applied to CCM (19%), PCCM (69%), or both (12%). Program directors represented CCM (17%), PCCM (57%), or both (26%) programs. Applicant (66%) and program director (49%) respondents reported initiating PIC. PIC did not impact ROL decision for most applicants (73%) or program directors (83%), though 21% of applicants and 17% of program directors moved programs or applicants up on their ROL in response to PIC. One-quarter (23%) of applicants strongly agreed or agreed that PIC was helpful in creating their ROL, 27% strongly disagreed or disagreed, and 29% were neutral. PIC challenges identified by both groups included time; lack of uniformity; peer pressure; misleading language; and uncertainty about motives, rules, and response protocols.
Interpretation
PIC is common among CCM/PCCM applicants and program directors. About 50% of applicants and 20% of program directors share ranking intentions via PIC. Although PIC did not impact ROL for most applicants and program directors, a minority of applicants and program directors moved programs up on their ROL after receiving PIC from the other party. Applicants have mixed perspectives on PIC value. Applicants and program directors alike desire clear guidance on PIC to minimize ambiguous and misleading communication.
Key Words: fellowship, medical education, postinterview communications, recruitment
Graphical Abstract
FOR EDITORIAL COMMENT, SEE PAGE 1035
Take-home Points.
Study Question: What is the frequency, format, goals, content, and impact on rank order lists (ROLs) of postinterview communication (PIC) between critical care medicine/pulmonary and critical care medicine applicants and program directors?
Results: PIC is common and did not impact ROL decisions for most applicants or program directors. PIC challenges identified by both groups included time, lack of uniformity, peer pressure, misleading language, and uncertainty about motives, rules, and how to respond.
Interpretation: Both applicants and program directors desire clearer guidance on PIC to minimize ambiguous and misleading communication.
In 2019, the Alliance for Academic Internal Medicine (AAIM) published updated guidelines for interview and postinterview communication (PIC) for graduate medical education recruitment in internal medicine (IM).1 Published in response to reports of problematic communication during graduate medical education recruitment,2, 3, 4, 5, 6, 7, 8, 9 the guidelines recommend that programs consider the benefits of uniform communication, review best practices on eliminating coercion, and reflect on applicant perspectives regarding PIC.1 Programs are advised to state their PIC policy and describe procedures for program-initiated PIC and expectations for applicants’ response. The guidelines also recommend adherence to the National Residency Matching Program (NRMP) Match agreement, NRMP Codes of Conduct, and other AAIM policies and standards created to protect applicants’ right to privacy and noncoercion.1,10,11 Guideline adherence is voluntary, potentially leading to variability in PIC practices between applicants and training programs.
PIC occurs between applicants and training programs within critical care medicine (CCM) and pulmonary and critical care medicine (PCCM), yet little is known about the content or its impact. Prior publications on PIC have focused primarily on practices among non-IM specialties,5,6,8,9,12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22 or have assessed PIC across all residency specialties, including IM.4,23,24 In addition, most of these studies included either applicants or program directors; few surveyed both simultaneously.16,21 PIC effects on rank order list (ROL) are varied, with some studies noting minimal impact and others reporting ROL changes in response to PIC.5, 6, 7, 8,12,14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24 Two publications on IM fellowship recruitment practices focused on interview-day conduct and NRMP violations, though applicants in both studies reported inconsistent expectations, deceptive statements, and inappropriate questions during PIC.2,3
To our knowledge, our study is the first to examine PIC practices exclusively in CCM/PCCM fellowship recruitment and to survey both CCM/PCCM applicants and program directors simultaneously. Our objectives were as follows: (1) to characterize the frequency, format, and content of PIC between CCM/PCCM applicants and program directors; (2) to identify the goals of PIC for applicants and program directors; and (3) to assess the impact of PIC on ROL decisions for both applicants and programs.
Study Design and Methods
We used a modified Delphi method to create separate cross-sectional surveys for CCM/PCCM fellowship applicants and program directors. The survey questions aimed to characterize frequency, format, content, goals, and impact of PIC from the applicant and program directors perspectives. Additional questions were asked about applicant demographics including self-identified sex, race/ethnicity, residency training program, stage of training when applying for fellowship, number of interviews offered and attended, and whether they matched to a fellowship program. The surveys were pilot tested with fellows from cardiology (n = 3), hematology/oncology (n = 2), and gastroenterology (n = 1), and program directors from other IM fellowship programs (cardiology: n = 1; hematology/oncology: n = 2). We intentionally recruited fellows and program directors in similarly competitive subspecialties as CCM/PCCM.25 Survey questions were revised in response to participant feedback. The final applicant and program director surveys contained 43 and 33 questions, respectively, using multiple-choice, Likert-scale, and free text formats (e-Appendices 1, 2), and took about 5 min to complete.
Surveys were disseminated separately to CCM/PCCM fellowship applicants and program directors using REDCap.26 The survey links were sent on December 1, 2022, just after release of the 2022 NRMP Medical Specialties Match results; the survey remained open through January 20, 2023. Three follow-up emails were sent at 2-week intervals, excepting the year-end holiday week. Participation was voluntary and without compensation.
Applicants were identified by pooling their Electronic Residency Application Service system email addresses from the authors’ six institutions (intentionally selected for diversity in program size and geography) by program type (CCM, PCCM, both). The pooled list included 2,898 names; after removing duplicates, the final sample included 1,105 unique applicants. We anticipated that these applicants would represent approximately 80% of NRMP registrants based on a similar prior survey and the percentage of applicants applying to both fellowships at our institutions.27 Applicants received an email invitation with a unique, de-identified survey link to track participation and send reminders to nonresponders. CCM/PCCM fellowship applicants who did not receive interview invitations were excluded from survey completion and subsequent analysis. Program directors were recruited via the Association of Pulmonary and Critical Care Medicine Program Directors, an organization representing 98% of CCM and PCCM fellowship programs in the United States. The Association of Pulmonary and Critical Care Medicine Program Directors emailed 269 CCM/PCCM program directors with an invitation and link to the anonymous program director survey, with follow-up emails at the same interval as applicants.
We used STATA version 17.0 (StataCorp LLC) to summarize the frequency and proportion of responses for each question. Four applicants did not receive interview invitations and were excluded from analyses. Fourteen percent of the eligible applicant surveys (n = 51) had responses to only the questions about programs and interview offers. To account for potential reporting biases, we included these surveys in analyses and reported missing data for each question. We also conducted sensitivity analyses to examine potential response bias based on early vs late responders, defined as completion during the first or second half of the survey period.
We used χ2 tests to examine differences in the frequency of (1) initiating PIC or (2) changing rank order based on PIC and by applicants’ sex, race, and whether they matched. For reporting purposes, we collapsed five-point Likert scales on agreement to agree, neutral, and disagree. We could not directly compare applicant and program director responses because the questions needed different framing: applicants responded only for themselves, whereas program directors answered for many applicants and program faculty. Two authors (M. M. J. and B. Ç.) independently performed inductive analysis of the two free text questions regarding PIC challenges and recommendations and grouped them into themes that were then reviewed and agreed on by all authors.28,29 We used the Checklist for Reporting of Survey Studies to organize presentation of results.30 The University of Washington institutional review board deemed the study (STUDY00015881) exempt from research oversight.
Results
Applicant and Program Director Demographics
About one-third of eligible participants responded to the survey (applicants: 34% [373 of 1,105]; program directors: 32% [86 of 269]). Applicants reported applying to PCCM (69%, n = 256), CCM (19%, n = 70), or both (12%, n = 46) programs. Program directors represented PCCM (57%, n = 49), CCM (17%, n = 15), or both (26%, n = 22) programs. Among the applicants, 61% were current residents and 58% attended university-affiliated residency programs. One-half (51%) of the applicants reported receiving ≥ 11 interview invitations, and 45% reported attending ≥ 11 interviews. Three-fourths of applicants (n = 281) reported that they matched into fellowship in this application cycle. Among program directors, 86% reported working at university-affiliated programs and sending a mean of 47.1 interview invitations for 5.1 match positions. Thirty-seven percent of program directors reported that they had a program policy on PIC, and among these, 90% (n = 29, 34% of all program directors) reported that their policy was reviewed in-person during the interview day. Table 1 summarizes applicant and program director demographics; there were no demographic differences between early (n = 219) and late (n = 154) responders; however, among the 322 applicants who answered the match question, more early responders matched (early: 79% [n = 173]; late: 70% [n = 108]; P = .009).
Table 1.
Applicant and Program Characteristics
| Applicants (n = 373) |
Program Directors (n = 86) |
|||
|---|---|---|---|---|
| Characteristic | No. (%) | Characteristic | Value | |
| Fellowship program applications | Type of program(s) | |||
| CCM | 70 (19) | CCM | 15 (17) | |
| PCCM | 256 (69) | PCCM | 49 (57) | |
| Both CCM and PCCM | 46 (12) | Both CCM and PCCM | 22 (26) | |
| Not answered | 1 (0.3) | |||
| Residency program base (check all that apply) | Fellowship program base (check all that apply) | |||
| Community | 186 (50) | Community | 12 (14) | |
| University | 217 (58) | University | 75 (87) | |
| Other/military | 3 (1) | Other/military | 0 (0) | |
| Interviews | Offered | Attended | Program size | |
| 1-5 | 79 (21) | 85 (23) | No. of match positions | 5.1 (1-21) |
| 6-10 | 107 (29) | 122 (33) | No. of applicant interviews | 47.1 (3-100) |
| 11-15 | 97 (26) | 93 (25) | PIC policy | |
| 16-20 | 35 (9) | 48 (13) | Yes | 32 (37) |
| ≥ 21 | 55 (15) | 25 (7) | No | 53 (62) |
| Not answered | 0 (0) | 0 (0) | Not sure | 1 (1) |
| Sex identity | How policy is shared (check all that apply) | |||
| Man | 182 (49) | Not shared | 3 (3) | |
| Woman | 132 (35) | Posted online | 2 (2) | |
| Prefer not to answer | 8 (2) | Reviewed in person during interview | 29 (34) | |
| Not answered | 51 (14) | Shared via email | 3 (3) | |
| Self-identified race/ethnicity (check all that apply) | ||||
| White | 136 (36) | |||
| Asian | 99 (27) | |||
| Black or African American | 21 (6) | |||
| Hispanic/Latin(x) | 22 (6) | |||
| > 1 race | 16 (4) | |||
| Other | 14 (4) | |||
| Prefer not to answer | 14 (4) | |||
| Not answered | 51 (14) | |||
| Status at time of application | ||||
| During residency | 228 (61) | |||
| During chief residency | 71 (19) | |||
| After residency | 73 (20) | |||
| Not answered | 1 (< 1) | |||
| CCM/PCCM fellowship match | ||||
| Matched | 281 (75) | |||
| Did not match | 41 (11) | |||
| Not answered | 51 (14) | |||
Values are No. (%) or mean (range). CCM = critical care medicine; PCCM = pulmonary and critical care medicine; PIC = postinterview communication.
Experiences of Applicants
Two-thirds (66%) of applicants initiated PIC to an average of 4.4 programs. Reasons for contacting programs included conveying gratitude (49%), informing a program that they would be ranked first (30%) or highly (22%), and asking additional questions (20%). Twenty-three percent engaged in PIC with their top program to convey interest in the hopes of matching there, and 10% reported initiating PIC with additional programs in case they did not match at their top choice. A minority (8%) reported engaging in PIC about ranking decisions after receiving advice that this would optimize their chance of matching. One-third (36%) reported that a mentor or residency program directors contacted programs on their behalf.
Nearly one-half (49%) of applicants reported receiving program-initiated PIC about fit with their program (25%); offers to meet with additional faculty or fellows (19%); messaging regarding being ranked highly (16%) or ranked to match (10%); and offers of grant funding, job offers, and scholarships (2%). Applicants reported being contacted by program directors (36%), assistant/associate program directors (8%), faculty interviewers (9%), fellows at the program (6%), division chiefs (3%), and others (8%). Table 2 summarizes applicant experiences with PIC.
Table 2.
Applicant Experiences With PIC
| Applicant-Initiated PIC |
Program-Initiated PIC |
|||
|---|---|---|---|---|
| Characteristic | Value | Characteristic | Value | |
| Applicants who sent PIC | 245 (65) | Applicants who received PIC | 184 (49) | |
| Messages sent | ||||
| Any type of PIC | 4.4 (1-10) | Programs who sent PIC | 2.6 (1-10) | |
| Highly ranked message | 1.1 (0-10) | |||
| No. 1 program rank message | 0.7 (0-2) | Type of messages from fellowship programs (check all that apply) | ||
| Offers to meet with additional faculty or fellows | 72 (19) | |||
| Motives (check all that apply) | Offers of support (eg, grant funding, scholarships, postfellowship job offers) | 7 (2) | ||
| Ask follow-up questions about the program | 76 (50) | Ranked to match; response is not expected/required | 43 (11) | |
| Convey gratitude for the opportunity to interview | 184 (49) | Ranked to match or would match if wanted | 36 (10) | |
| Inform I would be ranking them highly | 82 (22) | Ranked highly | 61 (16) | |
| Inform I would rank them No. 1 | 113 (30) | Fit in well in their program | 95 (25) | |
| Other | 5 (1) | |||
| Motives for sending ranking-intent PIC (check all that apply) | Rank Highly | Rank No. 1 | ||
| I was advised to send this form of correspondence to optimize my chances of matching | 29 (8) | 27 (7) | ||
| To convey interest in the hopes I would match there | 85 (23) | 71 (19) | ||
| To convey interest in the hopes I would be ranked highly there (in the event I did not match to my No. 1 choice) | 38 (10) | N/A | ||
| Other | 4 (1) | 0 (0) | ||
| Residency PD/other mentor contacted program on my behalf | ||||
| Yes | 135 (36) | |||
| No | 162 (43) | |||
| Not answered | 76 (20) | |||
| Programs contacted by mentor | 3 (1-8) | |||
| Programs receiving outreach from mentors | ||||
| Top ranked programs only | 103 (28) | |||
| Multiple programs | 30 (8) | |||
| All programs | 2 (1) | |||
Values are No. (%) or mean (range). N/A = not applicable; PD = program director; PIC = postinterview communication.
Experiences of Program Directors
Among program directors, 49% reported initiating PIC with applicants to express gratitude (23%), offer additional meetings with faculty/fellows (23%), and share information on the likelihood of matching at their institution (20%). Communication was initiated by program directors (37%), assistant/associate program directors (5%), faculty interviewers (5%), and their division chief (3%). Applicants were contacted via email (38%), telephone (12%), and the Electronic Residency Application Service system (3%). Among those who initiated PIC, 12% only contacted applicants ranked to match, another 12% contacted applicants in a likely position to match, and another 12% contacted all applicants. Reasons offered for sending PIC included competing for highly desirable applicants (28%), replying to applicant-initiated PIC (19%), coordinating complex matches (12%), recognition of peer programs engaging in PIC (10%), disseminating program updates (7%), and developing a relationship with applicants (7%).
Almost all the program directors (91%) reported receiving applicant-initiated PIC, with expressions of gratitude for the interview (40%), requests for additional program information (31%) or meetings with additional faculty (30%), and statements about ranking the program highly (77%) or as their top choice (57%). Program director experiences with PIC are summarized in Table 3.
Table 3.
Program Director Experiences With PIC
| Applicant-Initiated PIC |
Program-Initiated PIC |
||
|---|---|---|---|
| Characteristic | No. (%) | Characteristic | No. (%) |
| Applicants who sent PIC | 78 (91) | Programs that sent PIC | 42 (49) |
| What % of applicants contacted your program? | How applicants were contacted (check all that apply) | ||
| 1-20 | 34 (40) | 33 (38) | |
| 21-40 | 32 (37) | Electronic Residency Application Service | 3 (3) |
| 41-60 | 12 (14) | In-person visit | 1 (1) |
| 81-100 | 0 (0) | Telephone | 10 (12) |
| Not answered | 0 (0) | Video conference | 1 (1) |
| What information was conveyed? (check all that apply) | Other | 1 (1) | |
| Request for additional information about program | 27 (31) | Which applicants were contacted? | |
| Gratitude for opportunity to interview | 34 (40) | All applicants | 10 (12) |
| Statement about ranking your program highly | 66 (77) | Applicants underrepresented in medicine | 2 (2) |
| Statement about ranking your program No. 1 | 49 (57) | Only applicants that are in a position to match based on how far you usually go down your list | 10 (12) |
| Request for additional meetings with faculty/fellows | 26 (30) | Only applicants that are ranked to match | 10 (12) |
| Other | 10 (12) | ||
| Types of messages sent (check all that apply) | |||
| Offering additional meetings with faculty | 13 (15) | ||
| Offering additional meetings with fellows | 7 (8) | ||
| Gratitude for applicant interviewing at program | 20 (23) | ||
| Sharing additional program information | 8 (9) | ||
| Statement on likelihood of matching at your institution with applicant response actively encouraged | 0 (0) | ||
| Statement on likelihood of matching at your institution with applicant response actively discouraged | 8 (9) | ||
| Statement on likelihood of matching at your institution with no mention of applicant response | 9 (10) | ||
| Other | 11 (13) | ||
| Motives for sending PIC (check all that apply) | |||
| Peer programs engage in PIC | 9 (10) | ||
| To compete for highly desirable applicant(s) | 24 (28) | ||
| To coordinate complex matches (eg, couples match with other specialty) | 10 (12) | ||
| To develop a relationship with applicant(s) | 6 (7) | ||
| To disseminate program updates | 6 (7) | ||
| To respond to applicant PIC | 16 (19) | ||
| Other | 7 (8) | ||
PIC = postinterview communication.
Impact on Final ROLs
Among the 184 applicants who received program-initiated PIC, 45 (24%) changed their ROL and 134 (73%) reported no impact on their ROL by program-initiated PIC. Among the 78 program directors who received applicant-initiated PIC, 13 (17%) changed their ROL and 65 (83%) reported no impact (Fig 1A). Of applicants who received program-initiated PIC, 38 (21%) moved the program up and five (3%) moved the program down on their final ROL. Among program directors who received applicant-initiated PIC, 13 (17%) moved the applicant up and no program directors moved applicants down on their ROL (Fig 1B).
Figure 1.
A, B, Postinterview communication (PIC) impact on final rank order lists (ROLs). A, Impact of PIC on ROL creation. B, Change in ROL due to PIC.
Utility and Challenges
Applicants were divided about the helpfulness of PIC in creating their final ROL: 23% agreed, 27% disagreed, 29% were neutral, and 21% did not answer. There were no differences in these distributions by whether the applicants matched (P = .99) or were early or late responders (P = .81). The most beneficial aspects of program-initiated PIC for applicants included the likelihood of matching with explicit instructions not to reply (42%) and information about follow-up meetings with current fellows and/or faculty with similar clinical or research interests.
Thematic content analysis of the two free text questions regarding PIC challenges and recommendations for future PIC processes revealed multiple shared themes among both applicants and program directors. Most applicants (88%) and program directors (94%) provided at least one comment. Although 77 applicants (21%) and six program directors (7%) reported no challenges, other comments from both groups expressed uncertainty around PIC rules, response protocols, motives, and impact. Applicant 210 shared, “Some programs seemed like they expected thank you notes—even providing a specific email address and mailing address to send thank you letters to, while others explicitly stated no post-interview communication, and others did not comment. Felt different from residency where pretty much every program explicitly said no postinterview communication.” Both groups recommended uniformity and standardization to address this challenge. Program director 15 wrote, “All programs should utilize the same expectations for applicant communication, the match process is supposed to create a more equitable approach to fellowship recruitment.” Applicants and program directors also recommended sharing the program’s PIC policy with applicants to help address some of these concerns.
Misleading language was another challenge identified by both groups. Applicant 72, for example, shared how they “received a strong letter of interest from a program and even a gift basket to my house; ranked the program #1 but did not match there.” Program director 73 similarly wrote how “applicants were not always honest in their postinterview communication with several writing ‘I am ranking you #1’ and then the fellow not matching at our program despite being in a rank-to-match slot.” While both groups advocated for honesty as a solution, they also acknowledged the difficulties in avoiding deceptive or confusing language. Applicant 153 noted their main challenge was “formulating a diplomatic response that wasn’t misleading about how I intended to rank the program.” Program director 2 identified “trying to be uniform in process for all applicants,” as their primary challenge, while program director 24 wrote about “ensuring that there was no confusing or misleading language or language that would pressure the applicant to respond.”
Both groups also identified peer pressure and lack of uniformity with the current PIC process. They worried that lack of PIC participation might give an undue advantage to applicants and programs with more active PIC practices. For example, applicant 182 wrote, “It’s hard to hear that many people have different experiences and send ‘love letters’ to program directors or that certain applicants receive post interview communication as this creates pressure to be doing these things.” Program director 62 shared similar sentiments: “I believe other programs send more postinterview communication than we do, and I understand for applicants it can be alluring to rank a program that you know has you ranked to match.”
Additionally, both groups listed time burden, particularly that tracking and responding to emails made it “hard to keep up,” and created additional pressure to tailor every letter to the individual or program. Applicant 176 shared, “The additional burden of responding to program-initiated postinterview communication often added stress to the interview process. I felt an obligation to send timely, thoughtful, and politic responses to all communications as they seemed evaluative, no matter how they were worded.” Both groups recommended considering impact to mitigate this challenge. “Applicants have a lot of factors they are weighing,” wrote program director 1, “and don’t need us calling and applying pressure or providing what we feel as innocuous communication, but they may feel is pressure.”
Comments from applicants and program directors also revealed mixed perspectives on allowing PIC, ranging from allowing all types, only for one’s top program/ranked to match applicants, and allowing PIC but clarifying that applicants need not respond, to prohibiting all PIC. Fifteen applicants recommended that programs discourage thank you notes, and four program directors suggested using a generic response for all applicants. Table 4 includes the summary of common challenges with additional illustrative quotations. e-Figure 1 contains common recommendations from both applicants and program directors.
Table 4.
Common Challenges With Postinterview Communication
| Challenges | Applicants | Program Directors |
|---|---|---|
| Time | “I felt a little obligated to communicate back with them and attend the ‘town hall’ meetings and virtual tours they were offering, even if I didn’t really have time or want to go.”(Applicant 56) “Writing personalized thank you emails for so many programs was sometimes a challenge while interviewing while working.” (Applicant 168) |
“I am most frustrated by applicants who have many people contact me on their behalf. Responding to those emails and calls can take a lot of time.” (Program director 21) “There are so many emails during interview season that it is hard to keep up and didn’t want an applicant to feel slighted if they did not get a response.” (Program director 78) |
| Uniformity | “Mixed messages and inconsistency from different places. Most of the places I interviewed at asked for no post-interview communication. However, my residency program encouraged it despite the message from the places I interviewed at.” (Applicant 103) “Uncertainty as to significance of followup emails or lack thereof given some programs seem to do it as a rule and some don’t.” (Applicant 248) |
“Ensuring consistency in faculty with policy on not responding/engaging in communication except very specific questions that should go through the coordinator.” (Program director 51) “Trying to be fair to all applicants…” (Program director 77) |
| Peer pressure | “The rules and guidelines for postinterview communication are unclear, however I did it because I assumed other applicants were doing it and I didn’t want to be at a disadvantage.” (Applicant 110) “Worrying about the expectation that I would have to communicate if I wanted a program to think I was interested because I assumed everyone else was likely communicating.” (Applicant 276) |
“Knowing that peer programs are engaging in lots of postinterview communication.” (Program director 2) “Challenging to know what other programs are doing. We followed the recommendation to not communicate but worry that put us at a disadvantage if others did.” (Program director 71) |
| Impact | “I understand why there are recommendations about postinterview communication across programs. It is a small field and you worry about programs holding a grudge down the road when reviewing grants, papers, and faculty positions, especially when a program calls the night before the rank lists are due when you have not ranked them 1.” (Applicant 73) “I was very surprised by program directors calling me and telling me I was ranked to match. I had not prepared for this conversation and didn’t know what to say. It was difficult to not let these communications influence my decision.” (Applicant 97) |
“Do not want to apply pressure in any way either, just hoping that OUR interest may affect their rank list.” (Program director 28) “Do not want to place applicants in awkward position.” (Program director 85) |
| Misleading language | “Vague, sense of false hope given. I actually ranked one of the programs higher because of the postinterview communication and thought maybe I would match there but I didn’t match there.” (Applicant 87) “Got response back from my #1 ranked program that made it seem like I was ranked to match, but then did not match there. Would have preferred not to receive any communication.” (Applicant 240) |
“Conveying that they are high on our list without giving too much hope if they are not ranked to match.” (Program director 4) “Also, trying to find the right tone of positivity without sending a signal of how we intend to rank them.” (Program director 68) |
| Uncertainty about motives | “Certain communications were easy to interpret in different ways. For example, I was told I was ranked in the top 10 and ‘very highly and in a position that traditionally has a high likelihood of matching’ by different programs. I interpret this to mean that I am not ranked to match and took it personally whereas others interpret this to mean they will match with that program.” (Applicant 101) “Statements from program directors that you will be ranked ‘at the top’ are not helpful and anxiety-inducing. Am I ranked to match? Top half? It’s very vague and I’m certain some people would change their rank list based on such a communication.” (Applicant 154) |
“Also though I tell folks not to email us including there is no need to tell us if they are ranking us first when I hear nothing of course I worry when I don’t hear things from people we really like (even though I have told them not to).” (Program director 21) “Have had multiple experiences where applicants have told us they were ‘highly interested’ and then despite being ranked highly or to match, ended up matching elsewhere.” (Program director 39) |
| Uncertainty about rules | “I did not know if it was an appropriate thing to do. There is a grey area on if this is allowed or acceptable.” (Applicant 110) “This whole system is designed in a way to make it difficult to know what’s even appropriate which is a very strange approach to a professional type of work agreement.” (Applicant 196) |
“Unclear on what rules say about reaching out to applicants.” (Program director 6) “No clear guidance as to what is or isn’t appropriate.” (Program director 8) |
| Uncertainty about how to respond | “I also did not like the fact that a program reached out to me to say I was ranked to match. Although it was very kind on a personal level, it made me unsure as how to respond and concerned that if I did not also confirm that I was ranking them anything but #1 that they would rank me lower instead.” (Applicant 146) “When I was told ‘I would be great fit for X program,’ I had to answer in a polite, appreciative but neutral way. It was challenging finding those words without lying or ruining my chances of matching there. That program was not in my top 3.” (Applicant 320) |
“How to respond to fellows you ranked, but not at the very top of your list.” (Program director 7) “It’s uncomfortable to decide how/whether to respond to candidates who are unlikely to match.” (Program director 87) |
Discussion
Our study highlights that PIC is common in CCM and PCCM and initiated by both applicants and program directors. Although both groups expressed similar motives for participating in PIC, 25% of applicants reported unclear expectations and pressure to participate. PIC content included expressions of gratitude, follow-up questions, and offers for additional meetings, and just over one-half (52%) of applicants and 20% of program directors shared ranking decisions. Common themes regarding PIC included time burden; lack of uniformity; perceived peer pressure; misleading language; and uncertainty regarding rules, motives, and how to respond to PIC.
This study also reinforces the mixed perspectives and preferences regarding PIC among both applicants and program directors. Some studies have revealed that PIC is valuable in making ROL decisions for residency applicants,15,23 while others noted that PIC can be stressful and suggested that program-initiated PIC can be unprofessional and even unethical.2,23 One study of residency applicants found that program-initiated PIC caused undue pressure and led to negative perceptions of some programs, with respondents overwhelmingly recommending that PIC be eliminated or that programs set clear expectations for this communication.31 In our study, there was an equal distribution of applicants who found PIC helpful, unhelpful, or neutral; free text responses about challenges and recommendations were also mixed. Many applicants reported favorable experiences with PIC because of opportunities to connect with future mentors and transparency about ranking decisions, whereas others reported either neutral statements or strong apprehension about PIC.
Our findings demonstrate similarly divided perspectives among program directors. In a prior study of IM residency program directors, most reported participating in PIC to demonstrate “good manners,” even though few program directors indicated that PIC was helpful; they concluded that more definitive guidance for PIC was needed.7 Meanwhile, Farber et al17 concluded that PIC was common between urology applicants and residency program directors but not highly valued; they raised concerns about the risk of exchanging misleading information. Although some of the program directors respondents saw value in PIC, the majority highlighted the importance of uniformity, transparency about a PIC policy, and using clear language to minimize bias or coercion.
Our study also illustrates that although PIC did not impact the final ROL for most applicants and programs, a small minority of applicants and program directors reported altering their ROL based on PIC from the other party. These results contribute to findings of prior studies that program-initiated PIC may impact applicants’ ROL,4,6,13, 14, 15,18,20,23 whereas applicant-initiated PIC appears to have less impact on their position on a program’s ROL.2,9,23,31 These findings raise the following question: if there is minimal or no benefit for applicants to send PIC to programs, why does that perceived expectation persist among applicants? If program-initiated PIC can impact applicant ROL, further guidance on PIC language is needed to avoid misleading comments. Multiple previous studies have suggested that both applicants and PDs would benefit from more definitive guidance from the NRMP or other organizations.7,8,12,19,20,22,24,32 Our findings demonstrate that variance and ambiguity remain in the current practice of PIC. The common themes reported by both applicants and program directors suggest that there is inadequate information provided about the PIC process despite the current AAIM guidelines. Our findings also identify an opportunity to establish more explicit guidance regarding preferred content should an applicant or program choose to participate in PIC.
The primary limitation of our study is the low response rates in both cohorts; however, this is comparable with response rates from a similar survey of residency applicants.4 Although selection bias remains possible, the applicant participants appear to be representative of all 2022 to 2023 CCM/PCCM applicants by sex and race/ethnicity (e-Fig 2).33,34 Although we found no differences between early and late responders, we acknowledge a potential nonresponder bias from applicants who felt misled by PIC, although the mixed opinions regarding PIC suggest otherwise.
Despite these limitations, our findings have implications regarding future practices in CCM/PCCM fellowship recruitment and demonstrate utility in keeping some form of communication between applicants and programs. Next steps may include recommendations for greater transparency regarding rules and expectations, greater uniformity across programs, encouraging creation and dissemination of program policies, and creating best practices for PIC. Additionally, the AAIM guidelines could be revised to include clearer instructions, allowing for more consistent adherence and decreased variability in PIC practices. It is critical that program-initiated PIC avoid coercion while applicants are assessing future training opportunities and preparing to join the CCM/PCCM community.
Interpretation
PIC is common for both applicants and program directors in CCM and PCCM. About 50% of applicants and 20% of program directors share ranking intentions via PIC. PIC did not impact the ROL for most applicants or program directors who reported receiving PIC; however, a minority of applicants and program directors moved programs up on their ROL based on PIC from the other party. Applicants have mixed perspectives on the value of PIC, with nearly equal numbers finding it helpful, unhelpful, or neutral in ROL creation. Our findings do not allow us to draw conclusions about whether PIC practices should continue; however, both applicants and program directors identified a need for best practices and clearer guidance on PIC to minimize ambiguous and misleading expectations.
Funding/Support
Use of REDCap was supported by award number UL1 TR002319 from the National Institutes of Health National Center for Advancing Translational Sciences.
Financial/Nonfinancial Disclosures
None declared.
Acknowledgments
Author contributions: M. M. J. is guarantor of the paper, taking responsibility for the integrity of the work as a whole from inception to publication. M. M. J., H. S., K. M. B., and B. Ç. contributed to concept and design. All authors contributed to acquisition, analysis, or interpretation of the data. H. S. contributed to statistical analysis. M. M. J. and B. Ç. contributed to the drafting of the manuscript. All authors contributed to revising the manuscript critically and approval of the final version.
Role of sponsors: The sponsor had no role in the design of the study, the collection and analysis of the data, or the preparation of the manuscript.
Additional information: The e-Appendixes and e-Figures are available online under "Supplementary Data."
Footnotes
K. M. Burkart and B. Çoruh contributed equally to this manuscript.
Part of this article was presented at the American Thoracic Society International Conference, May 19-24, 2023, Washington, DC.
Supplementary Data
e-Figure 1.
e-Figure 2.
References
- 1.Alweis R.L., Williams C.M., Luther V.P., et al. AAIM guidelines for interview and post-interview communication for graduate medical education recruitment. Am J Med. 2019;132(9):1106–1111. doi: 10.1016/j.amjmed.2019.05.023. [DOI] [PubMed] [Google Scholar]
- 2.Williams C.M., Alweis R.L., O'Connor A.B., et al. Inappropriate communication during internal medicine fellowship recruitment: a mixed-methods analysis. Am J Med. 2019;132(6):770–775. doi: 10.1016/j.amjmed.2019.02.013. [DOI] [PubMed] [Google Scholar]
- 3.Cornett P.A., Williams C., Alweis R.L., et al. Problematic communications during 2016 fellowship recruitment in internal medicine. J Community Hosp Intern Med Perspect. 2017;7(5):277–281. doi: 10.1080/20009666.2017.1381546. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Berriochoa C., Reddy C.A., Dorsey S., et al. The residency match: interview experiences, postinterview communication, and associated distress. J Grad Med Educ. 2018;10(4):403–408. doi: 10.4300/JGME-D-17-01020.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Frishman G.N., Matteson K.A., Bienstock J.L., et al. Postinterview communication with residency applicants: a call for clarity! Am J Obstet Gynecol. 2014;211(4):344–350.e1. doi: 10.1016/j.ajog.2014.07.034. [DOI] [PubMed] [Google Scholar]
- 6.Holliday E.B., Thomas C.R., Jr., Kusano A.S. Integrity of the National Resident Matching Program for radiation oncology: national survey of applicant experiences. Int J Radiat Oncol Biol Phys. 2015;92(3):525–531. doi: 10.1016/j.ijrobp.2015.02.032. [DOI] [PubMed] [Google Scholar]
- 7.Chacko K.M., Reddy S., Kisielewski M., Call S., Willett L.L., Chaudhry S. Postinterview communications: two surveys of internal medicine residency program directors before and after guideline implementation. Acad Med. 2018;93(9):1367–1373. doi: 10.1097/ACM.0000000000002261. [DOI] [PubMed] [Google Scholar]
- 8.Brooks J.T., Reidler J.S., Jain A., LaPorte D.M., Sterling R.S. Post-interview communication during application to orthopaedic surgery residency programs. J Bone Joint Surg Am. 2016;98(19):e84. doi: 10.2106/JBJS.15.01364. [DOI] [PubMed] [Google Scholar]
- 9.Sbicca J.A., Gorell E.S., Peng D.H., Lane A.T. A follow-up survey of the integrity of the dermatology National Resident Matching Program. J Am Acad Dermatol. 2012;67(3):429–435. doi: 10.1016/j.jaad.2011.09.035. [DOI] [PubMed] [Google Scholar]
- 10.National Resident Matching Program Match code of conduct for programs. https://www.nrmp.org/wp-content/uploads/2022/08/NRMP-Match-Code-of-Conduct_Programs_Final.pdf
- 11.Alliance of Academic Internal Medicine Interview and post-interview communication guidelines for GME. Alliance of Academic Internal Medicine website. https://www.im.org/resources/ume-gme-program-resources/post-interview-guidelines/post-interview-guidelines-new
- 12.Berriochoa C., Ward M.C., Weller M.A., et al. Applicant interview experiences and postinterview communication of the 2016 radiation oncology match cycle. Int J Radiat Oncol Biol Phys. 2016;96(3):514–520. doi: 10.1016/j.ijrobp.2016.08.009. [DOI] [PubMed] [Google Scholar]
- 13.Tom M.C., Berriochoa C., Reddy C.A., Tendulkar R.D. Trends in radiation oncology residency applicant interview experiences and post-interview communication. Int J Radiat Oncol Biol Phys. 2019;103(4):818–822. doi: 10.1016/j.ijrobp.2018.11.042. [DOI] [PubMed] [Google Scholar]
- 14.Fereydooni A., Ramirez J.L., Morrow K.L., et al. Interview experience, post-interview communication, and gender-based differences in the integrated vascular surgery residency match. J Vasc Surg. 2022;75(1):316–322.e2. doi: 10.1016/j.jvs.2021.05.060. [DOI] [PubMed] [Google Scholar]
- 15.Camp C.L., Sousa P.L., Hanssen A.D., et al. Orthopedic surgery applicants: what they want in an interview and how they are influenced by post-interview contact. J Surg Educ. 2016;73(4):709–714. doi: 10.1016/j.jsurg.2016.03.009. [DOI] [PubMed] [Google Scholar]
- 16.Jewell C., David T., Kraut A., Hess J., Westergaard M., Schnapp B.H. Post-interview thank-you communications influence both applicant and residency program rank lists in emergency medicine. West J Emerg Med. 2019;21(1):96–101. doi: 10.5811/westjem.2019.10.44031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Farber N.J., Neylan C.J., Srivastava A., et al. The urology match process and the limited value of post-interview communication for program directors. Urology. 2019;128:23–30. doi: 10.1016/j.urology.2019.01.042. [DOI] [PubMed] [Google Scholar]
- 18.Anderson K.D., Jacobs D.M. General surgery program directors’ perceptions of the match. Curr Surg. 2000;57(5):460–465. doi: 10.1016/s0149-7944(00)00309-3. [DOI] [PubMed] [Google Scholar]
- 19.Farber N.J., Neylan C.J., Kaplan A., Singer E.A., Elsamra S.E. The urology match and postinterview communication. Urology. 2018;122:44–51. doi: 10.1016/j.urology.2018.03.057. [DOI] [PubMed] [Google Scholar]
- 20.Opel D., Shugerman R., McPhillips H., Swanson W.S., Archibald S., Diekema D. Professionalism and the match: a pediatric residency program's postinterview no-call policy and its impact on applicants. Pediatrics. 2007;120(4):e826–e831. doi: 10.1542/peds.2006-3189. [DOI] [PubMed] [Google Scholar]
- 21.Sbicca J.A., Gorell E.S., Kanzler M.H., Lane A.T. The integrity of the dermatology National Resident Matching Program: results of a national study. J Am Acad Dermatol. 2010;63(4):594–601. doi: 10.1016/j.jaad.2009.11.009. [DOI] [PubMed] [Google Scholar]
- 22.Yarris L.M., Deiorio N.M., Gaines S.S. Emergency medicine residency applicants’ perceptions about being contacted after interview day. West J Emerg Med. 2010;120(4):e826–e831. [PMC free article] [PubMed] [Google Scholar]
- 23.Jena A.B., Arora V.M., Hauer K.E., et al. The prevalence and nature of postinterview communications between residency programs and applicants during the match. Acad Med. 2012;87(10):1434–1442. doi: 10.1097/ACM.0b013e31826772a6. [DOI] [PubMed] [Google Scholar]
- 24.Nagarkar P.A., Janis J.E. Fixing the match: a survey of resident behaviors. Plast Reconstr Surg. 2013;132(3):711–719. doi: 10.1097/PRS.0b013e31829ad2bb. [DOI] [PubMed] [Google Scholar]
- 25.National Resident Matching Program, Results and Data Specialties Matching Service, 2023 appointment year. https://www.nrmp.org/wp-content/uploads/2023/04/2023-SMS-Results-and-Data-Book.pdf
- 26.Harris P.A., Taylor R., Minor B.L., et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95 doi: 10.1016/j.jbi.2019.103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Allam J.S., Burkart K.M., Çoruh B., et al. The virtual interview experience: perspectives of pulmonary and critical care fellowship applicants. ATS Sch. 2022;3(1):76–86. doi: 10.34197/ats-scholar.2021-0076OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Neuendorf K.A. SAGE Publications Inc; 2017. The Content Analysis Guidebook. [Google Scholar]
- 29.Hsieh H.F., Shannon S.E. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–1288. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
- 30.Sharma A., Minh Duc N.T., Luu Lam Thang T., et al. A Consensus-Based Checklist for Reporting of Survey Studies (CROSS) J Gen Intern Med. 2021;36(10):3179–3187. doi: 10.1007/s11606-021-06737-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Swan E.C., Baudendistel T.E. Relationship between postinterview correspondence from residency program applicants and subsequent applicant match outcomes. J Grad Med Educ. 2014;6(3):478–483. doi: 10.4300/JGME-D-13-00329.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Grimm L.J., Avery C.S., Maxfield C.M. Residency postinterview communications: More harm than good? J Grad Med Educ. 2016;8(1):7–9. doi: 10.4300/JGME-D-15-00062.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Electronic Residency Application Service Pulmonary disease and critical care medicine (internal medicine) https://www.aamc.org/media/41121/download
- 34.Electronic Residency Application Service Critical care medicine (internal medicine) https://www.aamc.org/media/40931/download
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.