Financial Implications of the Emergency Medicine Interview Process

Abstract

Background

Emergency medicine (EM) residency interviews are an important, yet costly process for programs and applicants. The total economic burden of the EM interviewing process is previously unstudied. Graduate medical education funding and student finances are both fragile shifting sources, which appear to fund most of these economic expenditures.

Objectives

The total economic impact of the EM interview season is unknown. This study sought to calculate total dollars spent by EM residency programs and senior medical students (M4) during interview season. Potential solutions for reducing this burden will be outlined.

Methods

Institutional review board–approved, piloted e‐mail surveys were sent to accredited (Accreditation Council for Graduate Medical Education [ACGME] and American Osteopathic Association [AOA]) EM program directors (PDs) and M4 student members of EMRA. PDs were queried after the 2014–2015 interview season. PDs questions included demographics, estimated faculty, and resident and administrative time used, along with dollars spent during the 2014–2015 interview season. M4 questions included demographics and dollars spent during the 2015–2016 season. Results were reported using descriptive statistics. Financial data for EM programs were calculated with academic EM faculty, resident, and administrative assistant salaries along with reported hours used during the interview season.

Results

A total of 82 of 223 EM PDs completed the survey, reporting an mean annual cost of $210,649.04 per program to review, screen, and interview applicants based on time spent by faculty, resident, and administrative assistants. A total of 84.6% of EM program costs were due to faculty hours. A total of 180 of 1,425 EM‐bound M4 students completed the survey, reporting a mean annual estimate of US$5,065.44 per student to apply and interview. Seventy‐two percent of estimated costs were due to airfare and lodging. Loans and credit cards were the top two methods of payments of these interview costs by students. Extrapolating the cost of EM personnel with hours spent, the economic burden of an interview season for EM programs is approximately US$46,974,735.92. M4 students spent US$19,724,823.40 for application fees and interview‐related expenses.

Conclusions

Emergency medicine residency programs and applicants appear to spend over US$66 million per cycle on the interview process. EM residency programs may save resources by reducing faculty hours associated with the interview process and leveraging administrative and resident resources. Creation of regional or national fixed interview locations may also be appropriate. Applicants may reduce travel costs by participating in video interviews, reducing program applications, and attending regionalized interview days. A full conversation among all specialties and organized medicine needs to take place to reform the systems in place to reduce the economic burden on students and residency programs.

The National Residency Match Program (NRMP) or The Match is a competitive and stressful process for fourth‐year medical students intending to further their training and enter clinical practice after graduation. Of many components in this process, interviews represent one of the most sensitive evaluative factors for both applicants and residency programs.1 In regard to their rank list, applicants cite residency interviews and their personal interactions with other residents among their most important considerations in formulating their rank lists.2

Residency interviews are also very expensive for applicants and for residency programs. The average US. medical school student graduates with nearly US$180,000 in medical school debt, while the mean total 4‐year medical school cost of attendance is over US$240,000.3 Compounding these costs, EM residency positions have become increasingly more competitive to obtain with the number of applications per student rising more than 46% in the past 6 years.4 More students with more applications results in increasing application fees along with the necessary finances to fund interview expenses. Costs to student applicants have been studied previously,5, 6, 7 but a review of the entire economy of an interview cycle including costs by residency programs has not.

Increasing debt burden of medical students serves as a primary impediment to national policy efforts to improve physician workforce diversity and mitigate shortages in primary care and certain geographic areas. Additionally, applicants state that this significantly impacts major life choices such as fellowship decisions, starting families, or major purchases such as a home.8 Students have voiced concerns that additional interview debt impacts their choices in specialties, number of programs applied to, and geographic decisions and is “the glass ceiling … set up to make economic minorities fail.”9

Because of the growth of interview season, residency programs divert numerous hours and resources away from their primary missions of education, research, and clinical care. A lengthy, time‐intensive process of evaluating hundreds of students for each potential PGY‐1 residency position has developed. The Centers for Medicare and Medicaid Services (CMS), along with state and local health systems, have traditionally funded graduate medical education (GME) training, but at unsustainable levels. As a result, residency program budgets continue to tighten as vital governmental funds are threatened on the budgetary chopping block year after year.10, 11

In this study, we will attempt to describe the economic scope of the EM residency interview process. Through retrospective time and expense reports along with financial databases, an approximate estimate of the financial impact carried by both EM student applicants and residency programs will define this economy. As an innovative specialty, EM should propose meaningful changes that may make this process more efficient and less burdensome for all stakeholders.

Methods

Study Design

This is a retrospective human subject study using e‐mail–based surveys that were created from SurveyGizmo LLC. Study approval by the lead author's institutional review board was obtained. The survey questions were piloted among 40 Philadelphia‐area senior EM‐bound medical students along with four separate EM program directors (PDs). After feedback from pilot participants, questions were revised and distributed.

Study Setting

Two populations were identified to be studied. The PD group consisted of 223 EM PDs obtained from the Council of Emergency Medicine Residency Directors (CORD) listserv. The student group were 1425 fourth‐year medical students (M4) applying for EM residency who were members of the Emergency Medicine Resident Association (EMRA). Final surveys were sent to PDs in August 2015 and to M4 students in February 2016. Because of the confidentiality of the EMRA database, direct use of student e‐mail addresses was not available. Thus the survey link was given to the EMRA research leadership to release to their members. Reminders to both the CORD listserv and the EMRA research leadership were sent on the first, second, and third weeks after release of the surveys to promote participation. No financial or tangible incentive was offered to either PD or M4 to participate in the surveys.

Study Protocol

Questions sent to PDs (Data Supplement S1, available as supporting information in the online version of this paper) were about the most recent NRMP interview season (2014–2015) as of August 2015. These included demographics of their programs, numbers of applications received and reviewed, and number of applicants interviewed. These data could be readily retrieved from sources such as the Electronic Residency Application Service (ERAS). Retrospective time spent by EM faculty, residents, and administrative assistants in hours spent in two separate processes were recorded.

The selection process was the time and personnel estimates of all the activities (phone calls, e‐mails, data acquisition from ERAS, and other activities) that led to selecting candidates for an EM interview. The conduction process consisted of retrospective estimates of time and personnel required to conduct and report on the actual interview candidates. This would include the physical events of an interview session along with ranking/evaluation processing following an interview. Additional information regarding dollars spent by EM programs on venues, housing, food/beverages, and transportation on interview days were also obtained. These “social event” costs were direct costs, even though they were retrospectively collected.

Questions in the M4 student survey (Data Supplement S2, available as supporting information in the online version of this paper) occurred at the conclusion of the 2015–2016 interview season in February 2016. This group of students was also queried in August of 2015 before the interview season began to collect data on predictions of financial and educational costs of the interview season. The August 2015 data will be the basis of a future publication. February 2016 was chosen so that students could report their actual rather than predicted costs since it was at the conclusion of the interview season. M4 questions included demographics, school type, numbers of EM programs applied to, interviews attended, and interviews declined by the student. Students were asked to report the total amounts they spent on transportation, lodging, clothing, and miscellaneous items needed for interviews. Reasons for declining interviews and method of payment for interview costs were ranked, with multiple‐option answers.

Measurements and Data Analysis

Application fees for ERAS are tiered and predetermined by the AAMC.12 The dependent variable in these fees is the total number of applications. The formula is the following:

ERAS fee formula (in USD): $99.00 + ($12.00 per application for 11–20 applications) + ($16.00 per application for 21–30 applications) + ($26.00 per application for 31 or more applications).

Student costs for board examinations such as Step 2 CK/CS and COMLEX along with the costs of “audition” away rotations in EM were not used in this study, but have been considered in previous studies.

Results from the PDs survey were then combined with an hourly pay rate for each of four groups, EM PD, EM faculty, EM residents, and administrative assistants.13, 14, 15 Direct hourly pay rates from a recent academic salary survey are available for PDs and EM faculty. Using the 50th percentile of respondents (including total compensation), $150.00/hour was used for PDs and $168.00/hour was used for other EM faculty. A lower rate for PDs is most likely for protected academic time and possibly administrative stipends. Hourly rates for residents and administrative assistants are more problematic and required some assumptions. Using Accreditation Council for Graduate Medical Education (ACGME) criteria, residents cannot work more than 80 hours per week clinically but not more than 72 hours in total duties.16 Typically, EM residents may receive approximately 3 weeks of vacation per year; thus their hourly rate would be:

$$\begin{array}{cc}& \text{USD}\mathrm{\$}54,000/49\text{weeks divided by}72\text{hours/week}\hfill \\ & =\text{USD}\mathrm{\$}15.30/\text{hour}.\hfill \end{array}$$

Administrative assistants are usually full‐time workers at 40 hours per week, but subject to state and local holidays and vacation time. Assuming vacation and paid holidays of approximately 3 weeks, their hourly rate would be:

$$\begin{array}{cc}& \text{USD}\mathrm{\$}36,500/49\text{weeks divided by}40\text{hours/week}\hfill \\ & =\text{USD}\mathrm{\$}18.60/\text{hour}.\hfill \end{array}$$

Note that resident and administrative assistant total compensation packages (bonuses, overtime, benefits, etc.) were not used; however, this would be expected to raise these hourly rates. All rates are summarized in Table 1.

Table 1.

Hourly Pay Rate for EM Subgroups (USD)

Subgroup	USD/hour
EM program director	150.00
EM faculty	168.00
EM resident	15.30
Administrative staff	18.60

With hourly pay rates for each position, key calculations can be made based on the results of the estimated time that PDs reported necessary to perform two main processes during the interview season. These are resources needed to select candidates for interviews and to conduct the interviews (including preinterview events, evaluation, and ranking). Selection personnel included PDs, EM faculty, and administrative staff groups, while residents were considered as conduction personnel.

The formula to calculate the dollars spent on selecting candidates were based on the following:

$$\begin{array}{cc}& \text{Mean number of people needed in a group}\hfill \\ & \text{(e.g., EM faculty)}\times \text{mean hours worked per person}\hfill \\ & \times \text{hourly rate of that group}(\text{Table}1).\hfill \end{array}$$

All three groups used to select candidates were totaled and divided by percentage of dollars per group.

The formula to calculate dollars spent on conduction of interviews were based on the following:

$$\begin{array}{cc}& \text{Mean number of people needed in a group}\hfill \\ & \text{(e.g., EM residents)}\times \text{number of interview days}/\hfill \\ & \text{sessions}\times \text{mean number of hours worked per}\hfill \\ & \text{person}\times \text{hourly rate of that group.}\hfill \end{array}$$

The conduction dollars were again totaled and divided by the percentage of dollars per group. Social costs were defined as the direct expenditures on activities such as preinterview dinners, food and beverage for interview sessions, and lodging and ground transportation for candidates on each interview session/day. Descriptive statistics were used for responses from the two study groups.

Results

A total of 82 of 223 (36.7% response rate) EM PDs and 180 of 1425 (12.6% response rate) EM candidate M4 students fully completed the surveys. Three incomplete PDs surveys and two M4 surveys were not used in subsequent formula calculations. PDs responses regarding the workload of the interview season (Table 2) reveal a wide range of responses to the number of applications received and applicants interviewed.

Table 2.

Work Load for Interview Season (Total Numbers)

Total	Mean	SEM	SD	25th Percentile	75th Percentile
PGY‐1 positions	11.67	0.46	3.96	9	14
Applications received	783.91	35.4	304.53	650	998.75
Applicants interviewed	139.02	4.56	39.18	115	163.75
Interview days/sessions	13.46	0.56	4.80	10	16

This workload created the reported mean number of interview sessions, which is a key variable that will be used to calculate conduction costs along with direct social costs. The amount of time in hours taken by the three different subgroups in selecting a student for interviews (including application review, communications, organization, and interview invitation processes) is reflected in Table 3. A wide range was present across all subgroups. No program had EM residents performing preinterview selection activities for student applicants.

Table 3.

Selection Activity (in Hours)

Total Hours	Mean	SEM	SD	25th Percentile	75th Percentile
Program director	101.32	13.48	115.93	40	117
Other EM faculty	149.05	4.73	250.51	46.56	194
Administrative staff	159.24	2.50	73.98	60	200

EM residents only participated in the actual conduction of interviewing activities (Table 4). Their work time, along with the previous subgroups, included social events prior to and during the interview day/session, managing the interviews themselves, and postinterview communications and ranking. EM faculty and PD time were combined in the conduction of the interviews to reflect additional effort by EM faculty. Similar to selection activities of the subgroups, conduction time of interviews had a wide range of responses.

Table 4.

Conduction Activities (in Hours)

Total Hours	Mean	SEM	SD	25th Percentile	75th Percentile
Faculty	820.9	1.25	139.61	365	876
Resident	810	1.16	194.53	324	972
Administrative staff	204.57	0.55	21.90	124.74	237.6

Using the formulas previously described, multiplying means of reported hours and the approximate hourly wage (Tables 1, 3, and 4) for each subgroup, calculations of dollars utilized are possible. Figures 1 and 2 indicate the amount of dollars spent for both selection and conduction of interviews for the entire interview season.

Figure 1.

Mean dollars spent in selecting students for interviews (USD).

Figure 2.

Mean dollars spent in conducting interviews (USD).

In addition to the subgroups' hours spent, PDs reported $13,338.00 on average for the direct costs of social expenses throughout the interview season. This included preinterview dinners and food/beverage during the day along with transportation and lodging expenses as seen in Figure 3.

Figure 3.

Social expenses for season (USD).

The total mean expenses experienced by an EM program (selecting for and conducting interviews, social expenses) thus calculate to approximately US$210,649.04 per year, per program: selection expenses = US$43,201.84; conduction expenses = US$154,109.20; social expenses = US$13,338.00; and total per program = US$210,649.04.

A total of 84.6% of these costs are due to the combination of PD and EM faculty salary time spent in selecting and conducting interviews. Extending to 223 ACGME and American Osteopathic Association (AOA) EM programs nationwide, the annual economic effort by all EM programs in one interview season is approximately US$46,974,735.92.

Student data included 180 of 1,425 (12.6% response rate) EM bound M4 students who completed their survey in February 2016 at the conclusion of the traditional EM interview season. Groups resulted included U. S. allopathic students comprised 69% of the respondents (124/180), U. S. osteopathic students 26% (47/180), and international medical school students 5% (9/180). Further results are divided among these groups (Figure 4).

Figure 4.

Respondent student schools.

Financial costs to students are dependent on the workload created by number of applications and interviews attended. This workload is divided into the three groups in Table 5.

Table 5.

Applicant Workload of Interview Season

Total Numbers of	Mean	SEM	SD	25th Percentile	75th Percentile
Applications submitted
US allopathic	46.16	2.14	23.88	31	52
Osteopathic	50.17	3.69	25.33	38	64
International	107.56	10.85	32.54	90	108
All students	50.28	2.08	27.94	32	60.25
Interviews attended
US allopathic	13.27	0.33	3.64	11	16
Osteopathic	10.79	0.68	4.63	7	14
International	7.67	1.57	4.72	4	9
All students	12.34	0.32	4.24	10	15
Interviews rejected
US allopathic	8.22	0.58	5.88	4	12
Osteopathic	4.68	0.60	3.33	2	6
International	1.25	0.25	0.5	1	1.25
All students	7.21	0.48	5.6	3	10

Key findings show that U. S. osteopathic students applied to approximately 8.6% more programs on average over U. S. allopathic students, while international students applied to 133% more programs. Osteopathic students attended fewer interviews and international students even less. All students rejected an mean of 7.21 interview invitations. Students rated programs being undesirable, conflicts with interview dates, and interviewing costs as the top three reasons to reject interviews (Table 6).

Table 6.

Reasons for Declining Interviews (in Order)

Rank	Reasons	Total Respondents
1	Decided that a program was not desirable	109
2	Interview date with another program was a conflict	92
3	Costs to travel too much	88
4	Unable to get time off from a clinical elective	30
5	Other	21

Costs that students experienced were divided into air and ground transportation, lodging, clothing, and miscellaneous expenses (Table 7). ERAS application fees that each group experienced were divided by group and as an aggregate (Table 8).

Table 7.

Subgroup Expense Categories (in USD)

	Mean	SEM	SD	25th Percentile	75th Percentile
US allopathic
Air travel	1,868.55	139.82	1,556.96	600	2600
Ground travel	575.81	38.64	430.23	300	800
Lodging	1,016.13	77.65	863.62	475	1400
Interview clothing	396.77	45.07	501.86	100	500
Miscellaneous	440.32	31.59	351.75	200	500
Total	4,297.58
Osteopathic
Air travel	1723.4	259.33	1,777.91	550	2400
Ground travel	544.68	74.61	527.06	300	550
Lodging	968.09	97.05	665.32	500	1200
Interview clothing	389.36	58.60	401.75	100	550
Miscellaneous	395.74	48.09	329.66	200	500
Total	4,021.27
International
Air travel	1,255.56	299.12	897.37	600	1500
Ground travel	322.22	52.12	156.35	200	400
Lodging	644.44	161.68	485.05	200	1000
Interview clothing	322.22	101.07	303.22	100	400
Miscellaneous	433.33	119.02	357.07	200	500
Total	2,977.77
All students
Air travel	1800	118.64	1,591.77	600	2500
Ground travel	555	33.21	445.62	300	625
Lodging	985	59.78	802.06	475	1300
Interview clothing	391.11	34.87	467.88	100	500
Miscellaneous	428.33	25.71	345	200	500
Total	4,159.44

Table 8.

ERAS Fees in USD (Number of Applications × ERAS Formula)

US allopathic	795
Osteopathic	899
International	2,355
Aggregate	906

ERAS = Electronic Residency Application Service.

Students reported that 67% of the interview costs were from airfare and lodging. Each category of expenditures was very similar for each student sub group. U. S. osteopathic students paid similar amounts in ERAS fees to U. S. allopathic students, while international students expected to pay 196% more in ERAS fees. An aggregate ERAS fee total was based on the averages and count of all subgroups, which resulted in US$906.00 in fees.

Combining aggregate ERAS fee costs along with all student mean interview expenses results in a total of US$5,065.44 per student across the entire interview season. With 3,894 students in the 2015 application season,17 the calculated interview season economic costs by EM student applicants would be US$19,724,823.40.

Students ranked how they paid for these expenses according to the methods found in Table 9. Students reported student loans and credit cards to be the two most likely methods to pay for these expenses.

Table 9.

Method of Payment for Total Interview Costs (in Order)

Rank	Method of Payment	Total Respondents
1	Student loan	126
2	Credit card	111
3	Family support	78
4	Cash	64
5	Other loan	10

Discussion

Emergency medicine residency programs and applicants together appear to spend over $66 million per cycle on the EM application and interview process. While EM is a very competitive specialty, application of these calculations to all specialties would reveal large amounts of GME funding spent to place appropriate candidates in open PGY‐1 positions. Using our total estimate of economic expenditures of EM programs in 2015 (US$46,974,735.92), with 1,821 PGY‐1 positions available, it appears the cost to fill each position to be approximately US$25,796. While EM‐bound students bear a significant burden in this calculated economy; however, when considering all students applying to all specialties, more concerns arise. Using the ERAS fee formula, 2015 ERAS fees alone for all student applicants (with no interview costs) were over US$72 million.18 These ERAS fees represent approximately 40% of the AAMC 2015 operating revenue.19 Several areas of opportunity exist in the EM and all academic specialties to reduce this and other interview related costs.

For student applicants, receiving appropriate advice regarding their level of competitiveness would be the first logical step. Deans, PDs, and EM career advisors all contribute to numerous and extensive resources on advising EM bound students. Specifically, they guide students in regard to the appropriate number, type, and variety of EM programs to which they may apply.20, 21, 22, 23 In this study, approximately seven interviews per applicant were rejected which is a direct reflection of the overapplication strategy that students continue to use in EM. Reducing the total numbers of applications would lead to lower ERAS fees for the students with an additional benefit of reducing faculty hours needed to review applications. The largest interviewing costs to students in this and other studies22 are airfare and lodging expenses, which can be mitigated by a variety of methods. In large‐population cities, “clustering” interviews to decrease travel time are starting to develop. The expansion of this urban cluster concept to multistate strategies where programs join together to have a common interview dates may help as well. Special interview rates or passes with national airlines and hotel chains through agreements by EM organizations and even the AAMC should be pursued. Grassroots efforts by medical schools and residency programs hosting students during interview season has begun and should be encouraged. Adoption of screening candidates by recorded video or conducting interviews by streaming video by EM programs are another attractive alternative in interviewing.24, 25 The AAMC Standardized Video Interview Research Study is in its data collection phase and is slated to end in December 2017. Results of that study will be used to evaluate the viability of a standardized video interview for use in residency selection in the future. Beyond screening tools, video conferencing may especially be helpful by allowing multiple faculty to interview with candidates at one time considering difficult academic faculty schedules. Whether students will accept changes in interview strategies is unclear. Traditional interviews which incur higher student and faculty costs have had attempts at modification with mixed results.26, 27

Our reported interview related student costs approaches 10% of an average EM PGY‐1 salary. With the choices of high‐interest student loan and credit card payments to handle these interview costs, we may be unduly influencing students to turn toward career choices where debt reduction is paramount, rather than abilities and desire. Dedicated low‐interest interview loans or grants may be strategies used to reduce this total debt burden while making appropriate career decisions.

For EM programs, the largest cost center is the number of faculty hours required for both PD and additional EM faculty. Considering an EM program interviews only 18% of applicants on average, improving screening methods and student advising are important. While controversial, making students aware of minimally acceptable USMLE and COMLEX scores, minimum numbers of SLOEs and other criteria on individual EM program websites may help students determine their competitiveness and desire for a given program. Outsourcing or delegating processes in preparation for and conducting interviews to lower‐cost entities such professional human resource departments or well‐trained EM administrative personnel may be a start. Smaller pools of more competitive candidates, along with a regionalized method of interviews, may make these faculty hours more efficient in choosing the appropriate candidate.

Future prospective studies following a cross‐section of programs and applicants longitudinally may yield more precise estimates of the total costs associated with the residency interview process. In addition, it would be beneficial to also understand the total educational cost of this process for students (i.e., missing clerkship time and choosing less challenging electives to attend more interviews). Is this increasingly time‐ and resource‐consuming process potentially impeding the education of our physicians in training?

We are hopeful that the true impact of this and future studies into the cost of the residency interview process will lie in the use of this data to inform meaningful conversation among EM leaders and those in other specialties, in concert with organized medicine, to achieve consensus strategies to reduce costs and increase efficiency for all stakeholders.

Limitations

This study was limited in that we surveyed a small segment of the EM residency applicant and EM PD pools over only one application cycle. Our response rates were low due to indirect access to e‐mails via a listserv and through a third party (EMRA database). Our student response rate was similar to a previous AAMC student study questionnaire. While our response rates were low, our focus is to start a meaningful conversation about the staggering amount of money being spent in this interview economy. PDs were tasked with reporting numbers from their most recent interview cycle, and it is possible that a significant portion of their numbers were memory‐based estimates and not actual figures. Faculty and PD pay rates are direct costs just as the social expenses reported. These pay rates should be considered when factoring opportunity cost, which represent significant money not able to be earned. The value of faculty to conduct and develop curriculum programs for EM residents, to pursue grants, for device innovation, and even for filling in clinical time slots is being lost to time spent filling residency positions. Protected academic time is calculated into core ACGME faculty positions; however, with the rising number of applicants who overapply to EM residencies, this academic time is being eroded.

In addition, our study is limited by possible volunteerism bias and recall bias. It is possible that a disproportionate number of applicants who experience a larger financial burden were more likely to participate, thus potentially overestimating the true overall costs incurred by applicants. It should also be noted that we did not include estimated costs associated with “audition” electives in EM completed outside of applicants' home institutions. These month‐long‐away electives can often play a significant role in the evaluation process for both programs and applicants and are universally expensive. We did not calculate the costs of examinations necessary to graduate from medical school (USMLE and COMLEX).

Conclusion

Large amounts of funds are required to evaluate a growing pool of competitive candidates for emergency medicine residency programs. Further study will need to be undertaken to see if these numbers are appropriate across all specialties. The total economic efforts to prepare for and conduct interviews are budgetary items not typically considered by GME programs and warrant review for efficiencies and alternative sources. Students should take advantage of all advising resources available to them along with making sound economic choices in the interview process.

Supporting information

Data Supplement S1. Program director questions.

Data Supplement S2. Student questions.

AEM Education and Training 2017;1:60–69.