Still want to be a doctor? Medical student dropout in the era of COVID-19

Abstract

This research examines the intention of undergraduate medical students to withdraw from the medical profession and pursue a career in a different field upon graduation during COVID-19. We leverage the first and most comprehensive nationwide survey for medical education in China, which covered 98,668 enrolled undergraduate students from 90 out of 181 Chinese medical schools in 2020. We focus on these students’ self-reported intention to leave the healthcare industry (the “dropout intention”) before and after the outbreak of the epidemic. We also designed a randomized experiment to test whether and to what extent medical students dropout intention responded to an information nudge that highlighted the prosociality of health professionals in the fight against the virus. Results from a difference-in-differences model and a student fixed effect model suggest that after the onset of COVID-19, the proportion of Chinese undergraduate medical students with a dropout intention declined from 13.7% to 6.8%. Furthermore, the nudge information reduced the intent-to-drop-out probability by 0.8 additional percentage points for students in their early college years. There was large heterogeneity underneath the treatment effect. Specifically, we find that prior dropout intention and exposures to COVID-19-related information tended to mitigate the nudge effects. Data on students’ actual dropout outcomes support our findings.

1. Introduction

During the COVID-19 pandemic, 59 million healthcare professionals worked around the clock to deliver quality care to over 271 million confirmed cases worldwide (as of Dec 16, 2021). Although many countries have recognized the critical role of healthcare workers during regular periods and in the event of public health emergencies, there exists a major and persistent shortage in the supply of health workers around the world. According to the World Health Organization (WHO), the gap between supply and demand in the labor market of healthcare industries has reached 17.4 million in 2013, a number predicted to remain at about 18 million by 2030 (WHO, 2016). Many countries facing severe shortages of health workers are economically underdeveloped. However, the Association of American Medical Colleges (AAMC) estimated that the U.S. would see a shortage of up to nearly 122,000 physicians by 2032 (Dall et al., 2019). This gap cannot be easily closed because of a high attrition rate in the medical field. O’Neill et al. (2011) reviewed 625 studies of factors associated with dropping out of medical school and estimated an average of 9.1% attrition rate with a range between 2.7% and 20.1%.1 Besides dropping out from medical schools, many countries saw a high proportion of students who choose to leave the medical profession upon graduation. This paper focuses on this dropout phenomenon in China. Based on our surveys across China, about 14% of new medical school graduates have deviated from their projected career path, threatening to enlarge the labor market disequilibrium in the health profession. Given that medical schools are highly selective and attract a relatively mature group of students (Arcidiacono and Nicholson, 2005), the high dropout rate (in the form of either withdrawing from a medical education program or leaving the medical field after graduation) incurs a significant loss of human capital in medicine (Johnson, Hutchins, 1966, Lien, Kosik, Fan, Huang, Zhao, Chang, Wang, Chen, 2016, Maher, Hynes, Sweeney, Khashan, O’Rourke, Doran, Harris, O’Flynn, 2013, Yeung, 2018).

Like students in other higher education programs, a medical student may choose to deviate from the planned career path due to a wide array of demographic, socioeconomic, psychological, and educational factors (Arulampalam, Naylor, Smith, 2007, Bound, Lovenheim, Turner, 2010, Kruzicevic, Barisic, Banozic, Esteban, Sapunar, Puljak, 2012, Arcidiacono, Aucejo, Maurel, Ransom, 2016, Nawa, Numasawa, Nakagawa, Sunaga, Fujiwara, Tanaka, Kinoshita, 2020). Academic preparation, family/financial stress, social isolation, and uncertain career plans are among the top triggers for dropout (O’Neill et al., 2011). Additionally, students in a medical program may face a number of unique disincentives and behavioral risks, including burnout, workplace violence, and occupational hazards that are particularly prominent at the advanced stages of study (Anderson, 2002, Dyrbye, Thomas, Huntington, Lawson, Novotny, Sloan, Shanafelt, 2006, Samadzadeh, Aghamohammadi, 2018). However, these risks usually do not lead to dropping out at the cost of forfeiting all earned credits. Similarly, very few medical students re-declared a major in their third to fifth years of study since many medical schools, especially those that operate without affiliation with a comprehensive university, have limited flexibility for students to switch to other academic programs. Besides, the cost of transition from medical education to other academic programs is much higher than from others. Due to these reasons, dropping out from medicine occurs more often at the stage of graduation, incurring a substantial loss of the professional skills and intellectual assets accrued from the medical training.

This research investigates (1) whether and to what extent medical students’ willingness of deviating from a planned career in the healthcare profession upon graduation (i.e., the “dropout intention”) could be affected by public health emergencies like COVID-19, and (2) whether a low-cost behavioral intervention could promote students’ willingness to stay in the medical profession. We collaborated with the National Center for Health Professions Education Development of China (NCHPED) that conducted the first nationwide survey on health professional education and development. The online survey was administered by 90 out of 181 randomly selected medical schools in China between February and March 2020. About 44% ($N=98,668$) of undergraduate students majoring in clinical medicine in these institutions participated in the study.2 The sampled students have demographic characteristics highly comparable with the administrative data covering the full population of medical students in 2019. We asked each student to recall his or her intent to drop out from medicine upon graduation before the epidemic.3 We then randomly assigned students into two groups and provided the treatment group (49,549 out of 98,668) with a gentle behavioral intervention. The intervention took the form of an information nudge which displayed an image with doctors in protective gear admitting a patient during the COVID-19. Besides the image, we included a short paragraph with encouraging words highlighting the heroic performance of Chinese doctors at the frontline of COVID-19. The text-image combined information aims at stimulating the prosocial identity (i.e., the self-conceptualization of helping and benefiting others as discussed in Grant et al., 2009) of medical students in the treatment group. Those in the control group did not have access to this information and thus were shown one screen less before proceeding. Students in both groups were then asked whether their intent to drop out still held after the outbreak of the epidemic.

Normatively, medical students’ responsiveness to public health events and any information nudges could come from changes in their intrinsic and extrinsic incentives. Intrinsic incentives relate to job mission as a motivation mechanism (Besley and Ghatak, 2005) and identity or self-image as components of preferences (Akerlof, Kranton, 2005, Bénabou, Tirole, 2011). Extrinsic incentives, on the other hand, relate to external factors such as compensation, career opportunities, and benefit packages that drive a behavioral outcome. Previous studies have shown that both kinds of incentives may play a role in motivating health workers, although sometimes with limited effectiveness (Sherry, Bauhoff, Mohanan, 2017, Ashraf, Bandiera, Davenport, Lee, 2020).

We expect the COVID-19 epidemic to activate both incentives. For instance, government institutions and social media have extensively recognized the altruistic act of healthcare workers, thus promoting medical students prosocial identity (Soled, Goel, Barry, Erfani, Joseph, Kochis, Uppal, Velasquez, Vora, Scott, 2020, Byrnes, Civantos, Go, McWilliams, Rajasekaran, 2020). The reported shortage of healthcare professionals could also inspire, attract, and retain more talented individuals who anticipate growing career benefits and opportunities in the healthcare profession. However, the potentially stronger incentives to study medicine may be mitigated by the added risks of workplace exposure to COVID-19 during the epidemic (Ripp, Peccoralo, Charney, 2020, Bo, Chen, Song, Zhou, 2020, Lancet, 2020). Moreover, external monetary compensation in the form of hazard pay may work against the desired direction and reduce medical students’ intrinsic motivation (the “crowding-out” effect) (Benabou, Tirole, 2003, Bénabou, Tirole, 2006, Ariely, Gneezy, Loewenstein, Mazar, 2009, Judson, Volpp, Detsky, 2015). Therefore, the COVID-19 epidemic may create a bi-directional impact on students’ incentives of working in the medical profession, with the joint effect depending upon their subjective evaluation of prosocial identity, potential career development opportunities, as well as personal safety threats during the epidemic.

Using a difference-in-differences model and a student fixed effects model, we found encouraging evidence that medical student dropout rate had substantially reduced by about 6.9 percentage points or 50 percent (i.e., from 13.7% to 6.8%) after the outbreak of the epidemic, indicating that public health emergencies can induce behavioral responses and promote medical students’ incentives to stay in the medical profession. Our results also revealed that a short-term, costless behavioral intervention could alter long-term career plans, but the impact concealed great heterogeneity. Those in early grades (i.e., the first 2–3 years of the undergraduate program, receiving general and basic medical education) and not intending to drop out before the epidemic were most strongly incentivized by the nudge. In contrast, students in an advanced phase of the undergraduate degree program and those with high propensities of withdrawing from the medical field were less likely to respond to any external information. We explored whether the observed heterogeneity was a joint outcome of differences in students’ prior attitude toward the medical career and various levels of prior exposure to COVID-19-related information. By analyzing the reading time on the nudge information, we found support to both sources and documented evidence of information avoidance either intentionally or unintentionally among the students who resisted to change. These results are consistent with the growing literature in nudge (Bhargava, Loewenstein, 2015, Bénabou, Tirole, 2016, Damgaard, Nielsen, 2018) and show the potential of using gentle intervention tools to help improve students’ career commitment. However, they also stress the importance to understand the behavioral mechanisms in order to maximize the nudge effectiveness and avoid backfire.

We conducted robustness checks to examine potential biases arising from retrospectively reported pre-COVID-19 dropout intention. First, we leveraged two annual surveys on medical school graduates in China in 2019 and 2020, respectively, to offer a pre-post comparison in dropout intention with the cutoff time being the outbreak of the epidemic. A coarsened exact matching method was used to account for the pseudo-panel nature of the survey. Estimation using the supplementary data uncovers a slightly smaller but significant effect of COVID-19 on reducing medical students’ dropout intention. Consistent with our experimental findings, students with stronger intrinsic incentives in studying medicine and a long-term career commitment were particularly motivated by the COVID-19 epidemic to stay in the medical profession. Second, we substituted each student’s reported pre-epidemic dropout intention with a predicted value based on traditional econometric and machine learning methods. This substitution helped to alleviate threats from potential misreports. Our results were robust across these dimensions. We also showed that our results were unlikely to be driven by the potential bias of selection into the survey.

The paper proceeds as follows. Section 2 introduces the medical education in China, describes the sample recruitment process, and summarizes our data. Section 3 provides the econometric framework. Section 4 presents the empirical findings, including both model-free evidence and regression results. Section 5 discusses and provides evidence to the underlying behavioral mechanism at work for our empirical findings. Section 6 contains robustness checks and external validity. Section 7 concludes with policy implications of our findings.

2. Background and data

2.1. Medical education in China

Unlike many developed countries, China offers medical education starting from undergraduate programs in a comprehensive university or a specialized medical college. Although the courses offered in each year of study are somehow similar across institutions, the program duration can vary substantially depending on the terminal degree earned. An eight-year program in clinical medicine leads to a Doctor of Medicine degree; a 5+3-year program in clinical medicine leads to a Master of Medicine degree; a five-year program in clinical medicine leads to a Bachelor of Medicine degree. There is also a five-year affirmative action program for rural students. Similar to the American pre-medical track, Chinese undergraduate medical students spend the first two to three years in large classes for general and basic medical education in preparation for advanced studies. The next three or two years involve specialized clinical medical training or clerkship. After the fifth year, students enrolled in an eight-year program or a 5+3-year program automatically transition into the post-graduate section of the programs without going through the application and admission process. Those enrolled in a five-year program face the choice of entering the labor market immediately or continuing to pursue a more advanced degree.

By the year 2020, China has 181 (14.54%) academic institutions that offer an undergraduate degree in clinical medicine (Peng et al., 2020). Over half of them are housed under comprehensive universities. The geographical distribution of these institutions is uneven across the country, with roughly 40% locating in the more developed Eastern region. Besides clinical medicine, many of these institutions also offer degrees in dental medicine, Chinese medicine, pharmacy, among others. However, a substantial shortage of health workers in various specialties remains despite a considerable expansion of undergraduate enrollment in Chinese colleges in the past decades. Specifically, the average number of undergraduate clinical medicine programs per 10 million people is 1.3 in China, significantly below the world average of 3.9 (Yu et al., 2021).

2.2. The national survey and pre-COVID-19 dropout intention

Approved by the Peking University Institutional Review Board, we partnered with the NCHPED to conduct the first nationwide survey of current medical students in the network of 181 higher institutions that offered an undergraduate clinical medicine degree in China. The survey was a part of NCHPED’s online education study during COVID-19, and 90 randomly selected medical schools in stand-alone medical colleges or comprehensive universities were invited to participate. The sampling process followed stratification by geographic location, institutional type, and program quality. An official survey invitation from each participating medical school was sent out to all the undergraduate students majoring in clinical medicine in these schools ($N=225,329$) between February 21, and March 14, 2020.4 The survey received responses from 98,668 students (response rate = 43.8%) residing in 34 provinces in the mainland, Hong Kong, Macau and Taiwan of China, and overseas.5 Participation was completely voluntary, and responses were anonymous. A disclosure statement and a consent protocol were presented to students prior to their participation. To increase the response rate, we provided lottery-based monetary rewards that could be obtained immediately after survey submission.6 We used the most popular online survey platform in China - WJX - to monitor and store survey results. Each survey response was unique to each participant’s IP address, but no other personal information was collected or identified. Figure s1 shows the survey cover page in both Mandarin and English.

The survey contained two sections with separate modules (See Fig. 1 ).7 The first section focused on collecting data on undergraduate medical students academic background, demographic characteristics, and pre-epidemic career plans. The second section included the behavioral nudge and focused on students’ learning experience and career decisions after the outbreak of COVID-19. We collected a rich set of student demographic features, including gender, residential location, indicators for doctor parent and home internet access, program type, and academic ranking. Column (1) of Table 1 summarizes these variables in the full sample. About 61% undergraduate students from all types of degree-offering institutions are female. About 10% students have a doctor parent, and over 64% have known teachers or schoolmates fighting at the frontline of COVID-19. This indicates that the surveyed students represent a group of population with relatively high information exposure to COVID-19. However, only 63.5% of them reported that they had paid attention to COVID-19 intentionally in February or March. This moderate level of media attention may be explained by a similar percentage of students with home internet access (63%). Furthermore, our sample contains about 30% rural students and over 40% suburban students.

Fig. 1.

Survey flow. Notes: This figure shows the modules of the two survey sections. The randomization at the beginning of Section II results in 49,115 control students and 49,553 treated students.

Table 1.

Randomization balance test.

	(1)	(2)	(3)	(4)
	All	Control	Treatment	C-T difference
Intent-to-drop-out (pre-COVID-19)	0.137	0.137	0.136	$-$0.001
	[0.343]	[0.344]	[0.343]	(0.003)
Female	0.611	0.611	0.611	0.001
	[0.487]	[0.488]	[0.487]	(0.004)
Parent is a doctor	0.102	0.103	0.102	$-$0.001
	[0.303]	[0.304]	[0.302]	(0.002)
Family location, vs. rural areas
Urban	0.287	0.287	0.288	0.002
	[0.453]	[0.452]	[0.453]	(0.003)
Suburban	0.411	0.412	0.410	$-$0.002
	[0.492]	[0.492]	[0.492]	(0.003)
Pre-clinical period	0.581	0.582	0.580	$-$0.005
	[0.493]	[0.493]	[0.494]	(0.006)
Enrolled program, vs. 5-year program
Rural 5-year program	0.089	0.090	0.089	$-$0.004
	[0.285]	[0.286]	[0.284]	(0.002)
5+3-year program	0.066	0.066	0.066	0.001
	[0.249]	[0.249]	[0.249]	(0.003)
8-year program	0.016	0.016	0.017	0.000
	[0.127]	[0.126]	[0.128]	(0.001)
GPA, vs. ranked top 25%
25–50%	0.359	0.359	0.358	$-$0.001
	[0.480]	[0.480]	[0.479]	(0.003)
50–75%	0.212	0.213	0.210	$-$0.003
	[0.409]	[0.410]	[0.408]	(0.003)
Bottom 25%	0.063	0.063	0.064	0.001
	[0.244]	[0.243]	[0.244]	(0.002)
Undermatched in college admissions	0.378	0.378	0.377	$-$0.001
	[0.485]	[0.485]	[0.485]	(0.003)
Home access to internet	0.630	0.630	0.629	$-$0.001
	[0.483]	[0.483]	[0.483]	(0.003)
Having paid attention to COVID-19	0.635	0.635	0.634	0.000
	[0.482]	[0.481]	[0.482]	(0.003)
Teachers/Schoolmants at the frontline	0.646	0.642	0.651	0.008**
	[0.478]	[0.479]	[0.477]	(0.003)
Students	98,668	49,115	49,553
Schools	90	90	90

Notes: Column 1 shows the mean and standard deviation for each variable in the full sample. Columns 2 and 3 show the same statistics for the control and treatment groups, respectively. Column 4 reports the estimated treatment-control difference for each variable using eq. 4 without controlling for covariates ($y_{ij}=\alpha +\gamma Nudg{e}_{ij}+{ϵ}_{ij}$). A joint F test, performed by regressing the treatment indicator on all the covariates in this table, yields a statistic of 0.868 with a $p$-value of 0.607, suggesting that the treatment group assignment was not based upon any observed student attributes. Undermatched in college admissions is a self-reported indicator evaluating whether a student perceived his or her college entrance exam score to be above the others with the same admission outcome in the same cohort. Having paid attention to COVID-19 is a self-reported indicator for voluntary attention to COVID-19 related information. Teachers/Schoolmates at the frontline indicates whether students have teachers or schoolmates in their close social network who have served at the frontline in Wuhan, the epicenter of the epidemic in China. Robust standard errors are clustered at the college-grade level. *** $p<0.01$, ** $p<0.05$, * $p<0.1$.

The observed demographic features of the sampled students are largely comparable with the ones reported by the population of all medical students in the administrative data from the Higher Education Evaluation Center of the Ministry of Education of China in 2018. Although some of these statistics are unavailable to the public, we were allowed to release the following information for comparison purposes: (1) the type of medical schools “ there were 38% of undergraduate medical students in comprehensive universities in the administrative data, and the share in our survey was 41%; (2) there were 39%, 36%, and 25% of undergraduate students in medical schools in the eastern, central, and western regions in the administrative data, while the corresponding shares in our survey were 38%, 42%, and 20%. According to the survey implementation reports, a main source of the variation in participation in the survey came from college-level factors, such as how colleges collected the survey and engaged their students.

At the end of the first section, students were asked to recall their pre-COVID-19 dropout intention after graduation. As pointed out earlier, “dropout” in this paper means discontinuing a medical career upon graduation instead of quitting an educational program before earning a degree. We coded the reported dropout intention as an indicator that equals 1 or 0, with 1 indicating a student being willing to withdraw from the medical profession after graduation.

A potential selection bias may arise from unobserved demographic attributes that drive students’ willingness of completing the survey. For instance, a student may have anticipated the presence of dropout-related questions in the survey and intentionally avoid them by choosing not to participate. This type of selection issue will likely lead to biased estimates on the pre- and post-epidemic dropout intentions and thus limit the external validity of our results. However, in our invitation, students were uninformed about any questions related to their intention to withdraw from the medical field upon graduation. In fact, the survey covered a wide range of questions, most of which pertained to students’ on-campus and online learning experience. Furthermore, we compared the reported dropout intention in our sample with the real dropout rate from another independent survey conducted by NCHPED in 2019, which included a national sample of new graduates from medical schools. Our survey shows that 13.7% of medical students intended to drop out prior to the epidemic, which is not statistically different from the 13.04% dropout rate reported by the 2019 sample. Therefore, we concluded that the survey participation was unlikely to be correlated with students’ dropout intentions. We further adopted re-sampling strategies to address any remaining external validity concerns in Section 6.3.

2.3. The impact of COVID-19 and the nudge experiment

The second section of the survey assessed how the COVID-19 epidemic has reshaped medical students’ career commitment and investigated the role of a gentle information nudge in shifting their career decisions. We asked students a series of questions regarding their coursework and experience in college, self-assessment of remote learning quality, and exposures to COVID-19-related information. Students were then randomly assigned into a treatment group and a control group.8 Treated students, representing about half of the survey participants, received a text-image combined information highlighting the role of over 10,000 health professionals fighting against COVID-19 in China (as shown in Figure s2). Students in the control group received no nudge information and thus were shown one screen less than those in the treated group. The second section concluded with a question asking students’ post-epidemic dropout intention after graduation, with or without reading the nudge information. In other words, students in the treatment group and the control group answered the same number of questions, including the two questions regarding their pre-COVID-19 and post-COVID-19 dropout intentions. The only difference was that those in the treatment group were presented with the nudge information before answering the post-COVID-19 dropout question. While the epidemic had not ended at the time of the survey, we used the term “post-COVID-19” to differentiate the period after the outbreak from the period before the first case was reported.

The nudge information was designed to highlight the prosociality of medical workers by describing how they contributed to the fight against the virus. We used motivational quotes that frequently appeared in social media to emphasize the courage of heroic doctors that served at the frontline and to summarize the social benefits of medical professionals’ contributions during the epidemic.

In particular, we highlighted the remarks made by Zhong Nanshan, the doctor who led China through COVID-19 and SARS and was regarded as the “Public Face of China’s War against Coronavirus,” that “[like Li Wenliang] most Chinese doctors are heroes [in the fight against COVID-19]!”9 The text message presents both Zhong and Li as heroic role models to young medical students, while the image aims at stressing the prosociality of the model doctors by showing them wearing masks and personal protective equipment at work in an emergency. In our pilot study, the text-image combined information took no more than 10 s for a college student to read.

According to the survey tracking record, students spent an average of 7 min and 56 s (median=6 min 47 s, 5/95 percentile=4 min 20 s /14 min 11 s) on both sections of the survey. Those in the treatment group spent 3.6 s ($p=0.006$) longer than those in the control group, suggesting an average reading time of 3.6 s on the nudge information. The reading time was 6.4 s ($p=0.045$) among students who did not intend to drop out of medicine before COVID-19 but only 3.2 s ($p=0.020$) among students who had a prior dropout intention. We discuss the implication of the reading time difference using the theory of information avoidance in Section 5.1.

Table 1 columns (2) and (3) summarize the observed student characteristics across the two experimental groups, including 49,115 control students in column (2) and 49,553 treated students in column (3). Column (4) of Table 1 shows that the randomization process generates no systematic difference in the observed covariates between the treatment and the control groups, except for the indicator of having teachers or schoolmates at the frontline against COVID-19. In addition, we conduct an F test by regressing all covariates on the treatment dummy. The F statistic is 0.868 ($p$-value = 0.607), suggesting that the treatment group assignment was orthogonal to the observed attributes.

3. Econometric methods

3.1. Factors affecting medical students’ pre-epidemic dropout intention

We first estimated the effects of various student- and college-level contributing factors to medical students’ dropout intention $y_{ij}$ before the epidemic using a linear probability model.

$$y_{ij}=\alpha +\mathit{\Psi }{\mathit{X}}_{\mathit{ij}}+\mathbf{\Theta }{\mathit{Z}}_{\mathit{j}}+{\epsilon }_{ij}.$$ (1)

${\mathit{X}}_{\mathit{ij}}$ is a vector of student-level covariates displayed in Table 1, which include dummies for female students, students with at least one doctor parent, subjective evaluation of being undermatched in college admission10 , indicators of urbanicity, program type11 , and college GPA ranking quartiles. ${\mathit{Z}}_{\mathit{j}}$ captures college-level covariates such as an elite college indicator, region of campus, main program type, and highest degree offered. ${\epsilon }_{ijt}$ is robust standard errors clustered at the college-grade level to account for possible correlation of unobserved factors (e.g., teachers quality) within cohorts.

Table 2 displays the estimation results using different collections of covariates. Columns (1)–(3) and (4) include student- and college-specific covariates, respectively. Column (5) consists of both sets of control variables, while column (6) replaces college characteristics by a vector of college fixed effects to further reduce the impact of unobserved college-specific factors that shift student dropout intentions in each medical program. We found that female, urban, and low-achieving students and those in five-year undergraduate programs were more likely to terminate a medical career. In contrast, students with a doctor-parent were less likely to drop out. Students who believed they were undermatched in college admissions also have higher dropout propensities, suggesting that early motivation and belief could have a long-lasting effect on students’ subsequent behaviors (Dillon and Smith, 2020). However, college-level factors were not strong predictors of student dropout behaviors. Types of college, geographic location, and program offerings only explained about 1% of the outcome variations. When college fixed effects were accounted for, we found the model fit to be slightly improved, and the estimated impact of each student attribute was consistent with those in columns (1) to (3).

Table 2.

Predicting medical student pre-COVID dropout intention .

	(1)	(2)	(3)	(4)	(5)	(6)
	Outcome: Intent-to-dropout before COVID-19
INDIVIDUAL COVARIATES
Female	0.015***	0.015***	0.021***		0.020***	0.017***
	(0.003)	(0.003)	(0.003)		(0.003)	(0.003)
Parent is a doctor	$-$0.016***	$-$0.016***	$-$0.015***		$-$0.014***	$-$0.012***
	(0.004)	(0.004)	(0.004)		(0.004)	(0.004)
Family location, vs. Rural areas
Urban	0.012***	0.012***	0.011***		0.007	0.010***
	(0.004)	(0.004)	(0.004)		(0.004)	(0.004)
Suburban	$-$0.001	$-$0.001	$-$0.002		$-$0.003	$-$0.001
	(0.003)	(0.003)	(0.003)		(0.004)	(0.003)
Pre-clinical period		$-$0.011**	$-$0.003		$-$0.004	$-$0.005
		(0.005)	(0.005)		(0.005)	(0.004)
Enrolled program, vs. 5-year program
Rural 5-year program			$-$0.037***		$-$0.034***	$-$0.034***
			(0.006)		(0.006)	(0.007)
5+3-year program			$-$0.030***		$-$0.053***	$-$0.049***
			(0.007)		(0.008)	(0.007)
8-year program			$-$0.008		$-$0.026*	$-$0.040***
			(0.012)		(0.014)	(0.014)
Undermatched in college admissions			0.015***		0.015***	0.015***
			(0.003)		(0.003)	(0.002)
GPA, vs. ranked top 25%
25–50%			0.049***		0.047***	0.047***
			(0.003)		(0.003)	(0.003)
50–75%			0.115***		0.112***	0.114***
			(0.006)		(0.006)	(0.006)
Bottom 25%			0.044***		0.047***	0.048***
			(0.003)		(0.003)	(0.003)
COLLEGE COVARIATES
Elite college				0.010	0.018*
				(0.011)	(0.010)
College location, vs. East region
Central				$-$0.000	$-$0.000
				(0.010)	(0.009)
West				0.011	0.009
				(0.009)	(0.009)
University-based (vs. independent) medical schools				$-$0.004	$-$0.001
				(0.007)	(0.006)
Highest program offered, vs. 5+3-year program
Undergraduate program				$-$0.023*	$-$0.031**
				(0.012)	(0.013)
Doctoral program				$-$0.012	$-$0.017
				(0.014)	(0.016)
Constant	0.126***	0.132***	0.090***	0.151***	0.112***	0.095***
	(0.004)	(0.004)	(0.005)	(0.009)	(0.011)	(0.005)
College Fixed Effects	No	No	No	No	No	Yes
Observations	98,668	98,668	98,668	98,668	98,668	98,668
R-squared	0.001	0.001	0.013	0.001	0.015	0.026

Notes: This table reports OLS estimates of the correlation between pre-COVID-19 intent-to-drop-out and the individual-/college-level attributes summarized in Table 1. Home access to internet, attention to COVID-19, teachers/schoolmates at the frontline are post-COVID-19 measures and should not affect students’ pre-COVID-19 intentions. Elite college is an indicator for the selective colleges identified by the Ministry of Education. Robust standard errors are clustered at the college-grade level. *** $p<0.01$, ** $p<0.05$, * $p<0.1$.

3.2. Impact of COVID-19 and nudge: A difference-in-differences model

Our baseline assessment on the impact of COVID-19 and nudge information relied on the assumption that aside from COVID-19, there existed no change in the external factors contributing to the dropout intention during the extended winter break in the academic year of 2020 (see Arcidiacono, Hotz, Maurel, Romano, 2020, Wiswall, Zafar, 2018, Giustinelli, Shapiro, 2019, Aucejo, French, Araya, Zafar, 2020 for applications of the same approach). Under this assumption, pre-epidemic career plan served as a valid counterfactual, and the pre-post change could be interpreted as the subjective treatment effect of COVID-19 on dropout intentions.

Our baseline model was built upon a difference-in-differences framework with two layers of interventions: the COVID-19 epidemic and the behavioral nudge. Let $i$ index student, $j$ index college, and $t$ index time period (before or after the COVID-19 outbreak). We modeled the dichotomous outcome of intent-to-drop-out as a linear function of COVID-19 period $COVI{D}_t$ (1=post-epidemic) and an indicator of the treatment group $Nudg{e}_{ij}$.

$$y_{ijt}=\alpha +\beta Nudg{e}_{ij}+\gamma COVI{D}_t+\varphi COVI{D}_t*Nudg{e}_{ij}+{\epsilon }_{ijt}$$ (2)

The coefficient $\gamma$ captures how the COVID-19 epidemic has influenced the career commitment for students in the control group, and $\beta$ measures the differences in dropout intentions between the nudge treatment group and the control group before the epidemic. An insignificant $\beta$ indicates no systematic difference in the pre-COVID-19 dropout intention across the treated and the control groups, i.e., the two samples are balanced due to the randomized experimental design. $\varphi$ shows the differential changes in dropout intention between the treatment and the control groups, resulting from the nudge intervention. Notice that Eq. (2) contains no individual characteristics as they are presumably unchanged within a short time window.

To address the concern regarding the potential bias arising from unobserved individual characteristics, we also estimated a slightly different version of Eq. (2) by adding individual fixed effects (${\delta }_{ij}$) and dropping $\beta Nudg{e}_{ij}$ to avoid multicollinearity:

$$y_{ijt}=\alpha +\gamma COVI{D}_t+\varphi COVI{D}_t*Nudg{e}_{ij}+{\delta }_{ij}+{\epsilon }_{ijt}$$ (3)

We tested the consistency in coefficients $\gamma$ and $\varphi$ from Eqs. 2 and 3 in Section 4.2.

3.3. Heterogeneity in the nudge effect

In addition to the baseline difference-in-differences framework, we also followed the standard randomized controlled trial estimation approach and employed a linear model to estimate the impact of the behavioral nudge. To increase model fit, we controlled for observed student characteristics (${\mathit{X}}_{\mathit{ij}}$) and college fixed effects (${\pi }_j$).

$$y_{ij}=\alpha +\beta Nudg{e}_{ij}+\mathit{\psi }{\mathit{X}}_{\mathit{ij}}+{\pi }_j+{ϵ}_{ij}$$ (4)

The coefficient of interest, $\beta$, captures the treatment effect of the information nudge on students’ dropout intention. We assumed that $\beta$ can be different for students who were willing to continue in the medical profession and those who wanted to drop out before the epidemic. Thus, we separately identified the value of $\beta$ for these subgroups of students and omit the subscript $t$ in the specification.

Existing research in the behavioral economics literature suggests that information-based nudge can heavily depend on the target population and the way that the message is framed. Based on this notion, we examined how the nudge effect varies by student characteristics using the following linear interaction model.

$$y_{ij}=\alpha +\beta Nudg{e}_{ij}+\mathit{\psi }{\mathit{X}}_{\mathit{ij}}+\theta Nudg{e}_{ij}*{\mathit{X}}_{\mathit{ij}}+{ϵ}_{ij}$$ (5)

Equation. (5) incorporates an interactive term between $Nudg{e}_{ij}$ and student characteristics ${\mathit{X}}_{\mathit{ij}}$ to capture a more or less strong nudge effect when certain characteristics are present. All of the models were estimated using Ordinary Least Squares (OLS) regressions. Standard errors were clustered at the college-grade level; results were consistent when using robust standard errors. The randomization was at the student level, so that we included no additional control for randomization blocks.

4. Results

4.1. Model-free evidence on the impact of COVID-19 and the nudge intervention

Figure 2 provides model-free evidence on the impacts of COVID-19 and the nudge intervention, respectively. Triangles and circles connected by dashed lines represent the pre-COVID-19 intent-to-drop-out rate, while those connected by solid lines represent the post-COVID-19 intent-to-drop-out rate. In general, students in early study phases were less prone to dropping out than those at advanced degree stages, although they had more years remaining till graduation. For both treated and control groups, the COVID-19 has substantially decreased the dropout rate by almost 50%, revealing medical students’ stronger willingness to continue in the healthcare profession post the onset of the epidemic. However, the treatment effect of the information nudge shows some degree of heterogeneity. Students at the general education and basic medical education stages were gently triggered to persist in the medical field, while those who have started their clinical medicine training and clerkship saw minimal changes in dropout rate after reading the nudge information. We explore the underlying mechanisms of this behavioral adjustment in Section 5.

Fig. 2.

Impacts of COVID-19 on medical student dropout intention. Notes: This figure plots the average intention-to-drop-out rates before and after COVID-19 by randomized groups and study phases. Dropout intention before COVID-19 was asked retrospectively in the survey. Dropout intention after COVID-19 was asked at the end of the survey after the treatment group received the nudge information. We classified students in early study phases as those in their first 2–3 years of the undergraduate program and those receiving general and basic medical education. Students in late study phases were typically in their next 2–3 years, receiving clinical education and clerkship. The sample size is 98,668..

4.2. Regression results on the impacts of COVID-19 and the nudge intervention

Table 3 reports estimates from the difference-in-differences model and the individual fixed effects model that are specified by Eqs. (2) and (3). The two models yield highly consistent results. For undergraduate medical students in the control group, COVID-19 statistically significantly reduced the intent-to-drop-out rate by about 50 percent, reflecting an absolute decline from 13.7% to 6.8% (6.9 percentage points decrease, $p<0.001$). Among these students, those in early study phases were more responsive to COVID-19. In particular, the epidemic-induced decline in dropout rate for general education and basic medical education students reached 56 percent (7.4 percentage points, $p<0.001$), outweighing a 42 percent (6.1 percentage points, $p<0.001$) reduced dropout rate for students in clinical medical education and clerkship. Besides the COVID-19 effect, students in early study phases were further motivated by the information nudge. Specifically, those at the general education and basic medical education stages experienced an additional reduction in the dropout intention by 0.8 percentage point ($p<0.05$). The same effect is not significant for those who have entered the advanced study phase at the time of survey.

Table 3.

Impacts of COVID-19 and the nudge intervention on medical students’ intent-to-dropout.

	(1)	(2)	(3)	(4)		(5)	(6)	(7)	(8)
	DID model (Eq. (2))					Student FE model (Eq. (3))
Sample	All	Early	Late	(2)-(3)		All	Early	Late	(7)-(8)
		study phases	study phases				study phases	study phases
Post COVID-19	$-$0.069***	$-$0.074***	$-$0.061***	$-$0.013**		$-$0.069***	$-$0.074***	$-$0.061***	$-$0.013
	(0.004)	(0.005)	(0.004)	(0.006)		(0.005)	(0.002)	(0.003)	(0.008)
Nudge	$-$0.001	$-$0.000	$-$0.001	0.000
	(0.003)	(0.004)	(0.004)	(0.002)
Nudge $\times$ Post-COVID	$-$0.004	$-$0.008**	0.001	$-$0.009*		$-$0.004	$-$0.008**	0.001	$-$0.009
	(0.003)	(0.003)	(0.004)	(0.005)		(0.004)	(0.003)	(0.004)	(0.007)
Constant	0.137***	0.132***	0.144***			0.137***	0.132***	0.143***
	(0.004)	(0.005)	(0.004)			(0.003)	(0.001)	(0.001)
Individual FE	No	No	No			Yes	Yes	Yes
Observations	197,336	114,694	82,642			197,336	114,694	82,642
R-squared	0.014	0.018	0.009			0.631	0.629	0.632

Notes: This table presents estimates of the impacts of COVID-19 and the nudge intervention on medical students’ intent-to-drop-out using a difference-in-differences model (columns 1–3) and a individual fixed effect model (columns 5–7). Early study phases indicate students in their first 2–3 years of the undergraduate program, receiving general and basic medical education. Late study phases indicate students in their next 2–3 years, receiving clinical education and clerkship. Robust standard errors are clustered at the college-grade level. *** $p<0.01$, ** $p<0.05$, * $p<0.1$.

The first two columns of Table 4 present the estimation results of the average treatment effect of the nudge intervention specified by Eq. (4). Coefficients are identical to those from the difference-in-differences model and the individual fixed effects model in Table 3. Controlling for student-level covariates increases the statistical power. On average, the information highlighting the critical role of prosocial healthcare workers further reduced medical students’ intent-to-drop-out rate by 0.4 percentage point ($p<0.01$). Results were also similar when we separately estimated the treatment effects for students in early or late study phases.

Table 4.

Prosocial nudge effects on medical students’ intent-to-drop-out by pre-COVID-19 intentions.

		(1)	(2)		(3)	(4)		(5)	(6)
		Outcome: Post-COVID-19 intent-to-dropout=1
Sample		All			Pre-COVID-19 dropout=1			Pre-COVID-19 dropout=0
Control mean		0.068			0.232			0.042
Nudge		$-$0.004***	$-$0.004***		$-$0.003	$-$0.001		$-$0.004***	$-$0.005***
		(0.002)	(0.002)		(0.008)	(0.008)		(0.001)	(0.001)
Constant		0.068***	0.053***		0.232***	0.219***		0.042***	0.055***
		(0.002)	(0.003)		(0.007)	(0.014)		(0.002)	(0.003)
Covariates		No	Yes		No	Yes		No	Yes
Observations		98,668	98,668		13,479	13,479		85,189	85,189
R-squared		0.000	0.086		0.000	0.048		0.000	0.014

Notes: This table presents estimates of the nudge effects on medical students’ intent-to-drop-out by pre-COVID intentions using Eq. (4). Covariates in columns (2), (4), (6) include student-level attributes in Table 1 and college and grade fixed effects. Robust standard errors are clustered at the college-grade level. *** $p<0.01$, ** $p<0.05$, * $p<0.1$.

Students’ pre-COVID-19 dropout intention not only serves as a baseline for the COVID-19 impact assessment but also offers valuable information on students’ prior attitudes before the randomized experiment for heterogeneity analysis on the nudge effectiveness.12

In columns (3) to (6) of Table 4, we separately evaluate the nudge effects for students who held a dropout intention before the epidemic and those who did not. An across-column comparison reveals that the nudge information effectively retained the students who were willing to continue in the medical profession before the epidemic. Specifically, for students with no dropout intention before the epidemic, the nudge intervention effectively reduced the likelihood of withdrawing by 0.5 percentage points ($p<0.001$, 12 percent of the control mean). This estimated impact is on par with the 0.49 percentage points found in a recent meta-analysis of 45 nudge RCTs in the “workforce and education” field covering over 134,726 individuals (DellaVigna and Linos, 2020). However, a costless gentle information nudge did little in promoting the students who planned to quit. In other words, a temporary behavioral motivation seems to have a much more limited impact on shifting long-term career projections when the direction of motivation goes against the initial plan.13

A reduced dropout intention among medical students during the COVID-19 epidemic could be caused by motivated intrinsic incentives in response to prosocial role models or improved extrinsic incentives due to increasing demand for doctors and higher expected career benefits. Our information nudge highlighted the role of heroic healthcare workers in fighting against the virus and thus could work through reinforcing students’ intrinsic incentives and enhancing their self identity as a member of a prestigious occupation. On the other hand, the presented information could induce students to make reasonable expectations regarding potential improvements in the career prospect of healthcare professionals (e.g., increased salaries and higher social respect). These optimistic inferences may shift students extrinsic incentives and promote their willingness to become a doctor. From the experimental design, we were unable to prove an underlying mechanism since students’ intrinsic and extrinsic incentives were not directly measured. In Section 6.1, we use non-retrospective data on students’ dropout intentions from an alternative survey with subjectively reported intrinsic incentives to offer indirect evidence on the behavioral channels at work. Results from a linear model with interactive terms capturing heterogeneous treatment effects showed that the COVID-19 epidemic likely retained medical students through increasing their intrinsic incentives.

5. Behavioral mechanisms underlying the heterogeneous nudge effects

In this section, we explore and discuss the possible underlying behavioral mechanisms for the nudge treatment effect heterogeneity. Following the literature of heuristic-systematic model (HSM) of information processing (Chaiken, 1980), we assume that a student processes an incoming message either instantly, heuristically, with minimum cognitive effort or deliberately using careful judgement of the content. The use of either processing method is endogenously determined by a student’s personal attributes and preference. One may more easily absorb or seek out information that goes along with his or her existing attitude or intentionally avoid information that contradicts with it (an “ostrich effect”) (Golman, Loewenstein, Molnar, Saccardo, Bohner, Dickel, 2011).

To test this hypothesis, we investigated the reading time of the nudge information among the treated students. Our survey did not record the time that students spent on each screen. However, the treatment and the control group only differed in the presence of the screen that contained the nudge information. Thus, we may infer the likely reading time of the text-image combined information. In Figure S1 left panel, we show the overall time spent on the survey among treated students. Those in upper grades generally spent less time completing the survey. Next, we regressed the reading time on student- and college-level covariates displayed in Table 2 using control students only and used the estimates to predict the required time of completing the survey for treated students. This provided us with an estimated reading time of the nudge information for each treated student (i.e., subtracting the predicted time from the actual time spent on the survey).14 Figure S4 right panel reveals a consistent pattern with senior students (grade 4 and up) more likely to process the nudge information heuristically. Those at the general education and basic medical education stages (grade 2–3) generally used more cognitive effort to intake and assess the information.

Next, we examined if a longer reading and processing time would translate to a higher likelihood of being successfully nudged. We categorized the reading time in brackets of 10 s and calculated the percentage of students being positively nudged (i.e., from dropping out to staying in the medical field) and negatively nudged (i.e., from staying in medicine to dropping out) in each bracket. Figure 3 depicts how the effectiveness of nudge changes with reading time. We found that the longer a student spent processing the information, the more likely a positive nudge effect would be observed. This pattern persisted across study phases (Figure S4) and pre-COVID-19 dropout intentions (Figure S4).

Fig. 3.

Predicted reading time on the nudge information and changes in the dropout decision. Notes: This figure plots the correlations between the time spent in reading the nudge information and the percentages of students changing from dropping out to remaining in the medical field (i.e., being positively nudged), from remaining in medicine to dropping out (i.e., being negatively nudged), and without any change in decision. Any change in decision should reflect a joint effect of COVID-19 and nudge instead of the nudge alone.

Below we propose several behavioral mechanisms underlying the observed phenomena and discuss the policy implications behind our findings.

5.1. Prior attitude affects how students respond to the nudge information

As pointed out in Section 2, we observed that the students who intended to stay in medicine before the epidemic spent on average twice the time in reading the message compared to those who intended to drop out (6.4 s vs. 3.2 s). This sharp difference suggests that students with dropout intentions were more likely to process the information heuristically and avoided things that conflicted with their prior attitude. As a result, they tended to under-update the information that went against their initial attitude (information avoidance), ignored the information and resisted to change (“ostrich effect”), or denied the information even when it was objective (reality denial).

Although information perception is endogenous, Fig. 3 shows instead that it is possible to improve the nudge effectiveness by forcing the participants to extend the reading time to activate their systematic processing scheme. This can be done by slightly modifying the experimental design, i.e., requiring respondents to stay on the information nudge screen for a short period. We leave it for future researchers to explore how variations of required reading time can affect the persuasiveness of information and whether the direction of the nudge effect can reverse with an excessively extended reading time (we briefly discussed one such case in Section 5.2).

5.2. Prior exposures to information affects students with different prior beliefs differently

Another factor that may shift a student’s internal incentive to respond to a nudge is the pre-existing exposure to COVID-19 information, which could take different forms. A socially engaged college student may search online and pay attention to social media to stay updated on current events. One may also gather information from teachers, mentors, family members, close relatives, and friends within his or her social network. Besides, those with home internet access and being geographically closer to Wuhan were more likely to absorb COVID-19-related information. When such pre-exposure was present, prior beliefs and attitudes were already formed at the time of survey (Sharot, Kanai, Marston, Korn, Rees, Dolan, 2012, Laudenbach, Malmendier, Niessen-Ruenzi, 2019). The text-image combined information became a gentle reminder of a student’s existing knowledge and worked through raising the salience of the epidemic in her career decision. Correspondingly, we would expect the nudge intervention to have a larger impact on students who were not extensively exposed to COVID-19-related information before the experiment.

We tested this hypothesis using a series of exposure measures: (1) whether a student reported to have paid attention to COVID-19, (2) whether she had a doctor parent, (3) whether faculty members or schoolmates in her close social network served at the frontline of COVID-19, (4) whether she had home internet access, (5) whether her college was in Wuhan’s neighboring provinces, and (6) whether her hometown province had a number of confirmed cases above the national median as of the survey day (data obtained from the National Health Commission of China). From these indicators, we constructed an exposure-to-COVID-19 index using a principal component analysis and categorized this index into low and high levels using median split. The two green bars in Fig. 4 show that among students who did not hold a dropout intention before COVID-19, the nudge information had a statistically insignificantly smaller effect on students with high exposures to COVID-19 information ($\widehat{\beta }=-0.0039$ relative to a control mean of 0.044, reflecting a 8.9% decline with a $p$-value of 0.035, compared with $\widehat{\beta }=-0.0051$ relative to a control mean of 0.040, which reflected a 12.8% decline with a $p$-value of 0.001). In Table s1:rct-hetero, we estimate the heterogeneous treatment effects for each of the six “information exposure” measures. While different measures may represent different channels and ways a student absorbed COVID-19-related information, results are qualitatively consistent with Fig. 4. In particular, the nudge information had no impact on students who knew their teachers and/or schoolmates serving at the frontline in Wuhan but instead reduced the dropout intention of those without such close relationships. This result suggests that “uninformed” students with no prior information exposure were more easily to be nudged.

Fig. 4.

Nudge intervention effect heterogeneity by pre-COVID dropout intention and estimated exposure to COVID-related information. Notes: This figure plots the estimated nudge intervention effects and corresponding 95% confidence intervals in gray lines for the four crosstab-ed groups in percentage relative to each subgroup’s control mean. Exposure to COVID index was estimated using a principal component analysis to summarize six main measures: (1) whether a student reported to have paid attention to COVID-19, (2) whether she had a doctor parent, (3) whether faculty members or students in her close social network served on the frontline of COVID-19, (4) whether she had home internet access, (5) whether her college was in Wuhans neighboring provinces, and (6) whether the number of her hometown province had confirmed cases above the national median as of the survey day. Low (High) exposure was defined as having the estimated exposure index smaller (larger) than sample median. The intervention effect was estimated using the same Model 5 in Table 4.

Prior attitude could work with pre-exposure to moderate the effect of nudge information. As the two orange bars in Fig. 4 show, when a student had formed an intention to withdraw the medical career before COVID-19, the nudge information generated insignificantly opposite effects when she had been lightly or heavily exposed to COVID-19 information before the experiment. When the pre-exposure level was low, the dropout intention was marginally declined with a gentle nudge ($\widehat{\beta }=-0.0059$ relative to a control mean of 0.223, $p=0.605$). However, the information could backfire when a student was re-exposed to a persuasive message that went against her prior attitude ($\widehat{\beta }$=0.0060 relative to a control mean of 0.243, $p=0.591$). This result suggests that although a reasonably stronger nudge (e.g., measured by a longer reading time) is likely to be more effective to some degree, it is not a one-size-fit-all tool of behavioral modification. Individuals who intentionally avoid such information because of personal distastes or heavy prior exposure may resist to change or even rebel against the nudge.

6. External validity and robustness checks

6.1. Validity of the impact of COVID-19

A critical concern regarding our empirical results lies in the nature of retrospectively reported dropout intentions among medical students during the COVID-19. One may question the reliability of the respondents’ stated willingness to continue in the medical profession prior to COVID-19 when the survey was taken after the COVID-19 outbreak, given that the epidemic has reshaped the medical education globally and affected the career perspective of healthcare workers. Moreover, a student’s dropout intention reported in the survey may not predict her actual decision. In this subsection, we use an alternative multi-year survey that reports medical school graduates’ real dropout outcomes of the 2019 cohort versus the 2020 cohort to provide supplementary evidence to our findings on the impact of COVID-19.

This survey is an annual “exit survey” conducted in each summer by the NCHPED as official medical school quality measures, similar to the Association of American Medical Colleges (AAMC)’s national sample survey. The survey in 2019 and 2020 consisted of 7419 and 25,754 randomly selected new graduates who have completed fifth-year clinical medicine training from 32 and 104 Chinese undergraduate medical programs, respectively. The first case of COVID-19 was reported in China on Jan 3, 2020, and the community transmission has grown rapidly since then. Thus, we assumed that the participants who completed the survey in 2020 were exposed to the COVID-19 and those in 2019 were not.

The survey recorded a binary indicator of dropout outcome upon graduation at the time of interview in July. Dropout is defined as a student chose to leave the medical profession upon graduating from the undergraduate clinical medicine program. In other words, one neither continued to enroll in a graduate medical program nor worked in the medical profession. On average, 13.04% of the medical students eventually withdrew from the medical profession in 2019, a number very close to the 13.7% pre-COVID-19 intent-to-drop-out rate in our main sample in Table 1. This consistency supports our identification assumption that pre-COVID-19 intention could serve as a counterfactual to the post-COVID-19 intention.

Besides the dropout outcome, the survey also included rich student background information, including gender, ethnic minority status, single/non-single child, geographic location (i.e., province, rural vs. non-rural), parental education, indicator of having a doctor parent, family socioeconomic status, high school attributes (type of high school, college entrance exam score), and enrolled medical program. In addition, the survey directly measured whether a student had a long-term career commitment in the medical profession since high school and a student chose the clinical medicine as his or her undergraduate major because of intrinsic incentives (e.g., saving lives and helping the injured) instead of extrinsic incentives (e.g., salaries).

Table 5 reports the estimated impacts of COVID-19 on the decision of dropping out of the medical profession upon graduation using students’ real, non-retrospective graduation outcome data. The model is based on a cross-cohort pre-post comparison design and the assumption that the COVID-19 epidemic exogenously affected the 2020 cohort. We employed a coarsened exact matching (CEM) method (Iacus et al., 2012) to account for full student-level covariates and ensure balance between pre- and post-epidemic cohorts. We found no statistically significant differences in student-level covariates across 14,409 matched individuals (4126 in 2019 and 10,408 in 2020). However, results were similar using the unmatched full sample (columns 1 versus column 2 in Table 5), controlling for family background, high school attributes, and college fixed effects. Consistent with our main findings in predicting medical school dropout (Table 2), rural students were less likely to withdraw from a medical career, possibly due to high job security and adequate economic compensation. Students being the only child in the family were less motivated to continue their medical degree, which may be explained by a dominating effect of potential psychosocial risk at the workplace.

Table 5.

The effect of COVID-19 on medical students’ dropout outcomes using non-retrospective data.

	(1)	(2)	(3)	(4)	(5)	(6)
	Outcome: Dropout (leaving the medical profession) = 1
Post-COVID-19 cohort (class of 2020)	$-$0.040***	$-$0.029***	$-$0.041***	$-$0.044***	$-$0.027***	$-$0.005
	(0.007)	(0.009)	(0.012)	(0.013)	(0.009)	(0.011)
Female	0.001	$-$0.001	$-$0.001	$-$0.018*	$-$0.001	$-$0.001
	(0.003)	(0.005)	(0.005)	(0.011)	(0.005)	(0.005)
Rural	$-$0.013***	$-$0.016***	$-$0.034**	$-$0.016***	$-$0.016***	$-$0.016***
	(0.003)	(0.006)	(0.014)	(0.006)	(0.006)	(0.006)
Minority	0.016***	0.016	0.015	0.016	0.052	0.016
	(0.006)	(0.020)	(0.020)	(0.020)	(0.049)	(0.020)
Only child	0.005	0.020***	0.020***	0.020***	0.020***	0.020***
	(0.004)	(0.006)	(0.006)	(0.006)	(0.006)	(0.006)
Long-term career commitment	$-$0.082***	$-$0.069***	$-$0.069***	$-$0.069***	$-$0.069***	$-$0.046***
	(0.004)	(0.006)	(0.006)	(0.006)	(0.006)	(0.013)
Intrinsic incentives	$-$0.032***	$-$0.035***	$-$0.035***	$-$0.035***	$-$0.035***	$-$0.022**
	(0.004)	(0.005)	(0.005)	(0.005)	(0.005)	(0.008)
Post COVID $\times$ rural			0.025*
			(0.014)
Post COVID $\times$ female				0.024*
				(0.013)
Post COVID $\times$ minority					$-$0.050
					(0.047)
Post COVID $\times$ long-term career commitment						$-$0.019*
						(0.010)
Post COVID $\times$ intrinsic incentives						$-$0.022**
						(0.008)
Constant	0.125***	0.131	0.132*	0.136*	0.133*	0.109
	(0.035)	(0.079)	(0.079)	(0.080)	(0.079)	(0.079)
Family background & high school attributes	Yes	Yes	Yes	Yes	Yes	Yes
College FE	Yes	Yes	Yes	Yes	Yes	Yes
CEM matched sample	No	Yes	Yes	Yes	Yes	Yes
Observations	32,905	14,409	14,409	14,409	14,409	14,409
R-squared	0.047	0.042	0.042	0.042	0.042	0.043

Notes: This table shows estimates on the impact of COVID-19 on medical students’ intent-to-dropout using non-retrospective data of graduation classes of 2019 and 2020. Class of 2020 is defined as the post-COVID-19 cohort. Long-term career commitment was measured from a survey question asking whether students had the career goal of being a doctor in high school. Intrinsic incentives were measured from a set of Likert scale questions asking the reasons that students chose clinical medicine as their undergraduate majors. The coarsened exact matching follows the standard approach proposed by Iacus et al. (2012) using student-level covariates and controlling for college fixed effects. Robust standard errors are clustered at the college-grade level. *** $p<0.01$, ** $p<0.05$, * $p<0.1$.

Long-term career commitment and intrinsic incentives work with COVID-19 in retaining medical students. Students with strong intrinsic incentives and a long-term career commitment since high school were much less likely to leave the medical profession. These findings are consistent with the prosocial nudge effects reported in the previous sections. If the underlying mechanism of “the COVID-19 impact” works through extrinsic incentives, we should expect to observe that students with extrinsic incentives for medicine are equally or more likely to be affected by the epidemic. Our results show the opposite and provide supporting evidence that the COVID-19 epidemic is more likely to reduce dropout through promoting medical students’ intrinsic incentives.

6.2. Validity of the impact of the nudge intervention

A practical challenge in validating the estimated nudge effect is that the intervention must be given after the epidemic. Thus, the retrospective nature of our data may produce a stronger bias in the reported pre-COVID-19 intentions for students who were more likely to be affected by COVID-19. To address this potential reporting bias, we replaced the dichotomous outcome with a continuous measure of dropout probability for robustness checks. This alternative measure was generated using predicted intent-to-drop-out from a set of models controlling for full individual-level and college-level covariates in Table 1.

Table s2:prediction compares the classification accuracy of four different prediction methods: Linear (as in Table 2), Logit, LASSO, and Random Forest. In the latter two methods, we split the sample into a training group (75%) and a validation group (25%). Overall, the four methods generate comparable levels of classification accuracy. Logit and Random Forest tend to have a slightly better performance in-sample and out-of-sample predictions. The top 5 important predictors in the Random Forest model (column 8) are undermatched in college admission, having paid attention to COVID-19, with home internet access, study phase, and having teachers/schoolmates at the frontline. Nevertheless, about 40% of the variations in students’ dropout intentions remained unexplained by data collected in the survey, calling for future research and data collections to better understand medical students’ dropout intentions and behaviors.

Using the predicted dropout probabilities from Table S2 column 8, we estimated the nudge effect within each predicted dropout probability decile. Figure 5 shows that the nudge intervention has consistently affected the students with a low predicted pre-COVID-19 dropout probability but did not change the minds of those who had alternative career plans before the epidemic. We got similar results using the pre-COVID-19 dropout probabilities from a Logit model. These results suggest that a human decision can be most effectively influenced by a behavioral nudge if there is no strong prior belief that goes against it.

Fig. 5.

The treatment effect heterogeneity by pre-COVID dropout intention predicted by random forest Notes: This figure plots estimated nudge intervention effects by pre-COVID-19 dropout probability deciles. The pre-COVID-19 dropout probability was predicted using random forest with the full individual-level and college-level covariates in Table 1. See details about different prediction models in Table s2:prediction. The intervention effect was estimated using Eq. (4) in column 2 of Table 4.

6.3. External validity concern due to survey non-response

While our survey reached all undergraduate students from 90 randomly selected medical schools in China, 98,688 (44%) of the invited students participated in the study. In Section 2.2 and 6.1, we have discussed this potential selection bias and offered several strategies to minimize its impact on our estimation results. In this subsection, we report the results from additional robustness checks to address the selection issue.

First, we used a bootstrap resampling method to assess the robustness of our results. The idea was that, when biased estimates arise from a particular group of students with unobserved characteristics that were correlated with both survey completion and dropout intentions, a bootstrapping method with an equal probability in resampling would generate unbiased estimates on model parameters. We started by drawing a 75% random sample of the 98,688 survey participants and re-estimated Eq. (4). We then repeated this procedure with 10,000 random draws. Figure S7 left panel plots the 10,000 nudge treatment effect estimates and their corresponding 95% confidence intervals. The 10,000 estimates are not significantly different from the main treatment effect estimate as reported in column 2 of Table 4. In the right panel of Figure S7, we make a more conservative assumption that 50% of the sample might suffer from the selection bias and randomly draw 10,000 sub-samples of 50% of the survey participants. Results are qualitatively identical.

Second, we refined our sample based on the field notes from the survey administration that non-responses were largely driven by between-college variation. Since the average count of student respondents per school was roughly 1100 (98,688/90), we further restricted our sample to 24 colleges with at least 1500 survey respondents. These colleges were likely to have a very high response rate due to campus culture or administrative effort. We found very similar results: the estimated nudge effect is $-$0.0044 ($p$-value = 0.037) based on the sub-sample of 58,246 students, which was very close to the full-sample estimate of $-$0.004 ($p$-value = 0.007).

7. Conclusions

Whether we are in difficult pandemic times or back in normal day-to-day life, closing the shortages in the supply of medical workers is essential for a society’s operation, functioning, welfare, and sustainability. This paper shows the potential as well as opportunities for motivational interventions to achieve desired social outcomes in the medical profession or other areas (DellaVigna, La Ferrara, 2015, Laudenbach, Malmendier, Niessen-Ruenzi, 2019, Bo, Chen, Song, Zhou, 2020). Utilizing the first nationwide survey for health professional education and development in China, we documented that public health emergencies can heavily influence the career decisions of medical school students, leading to a significantly lowered dropout rate upon college graduation. Further analyses using supplementary data sets reveal that the shifts in career plans come mostly from motivated intrinsic incentives to become a doctor.

In addition, we conducted a prosocial nudge experiment to test the effectiveness of a gentle behavioral intervention in retaining medical students in the event of public health emergencies. While the estimation results showed encouraging evidence that costless nudge information may work to influence college students’ career outlook, we also uncovered large heterogeneity underneath this effect. Our findings point to an important role that prior attitude and information exposures jointly play in determining the degree to which one’s career plan can be modified by a light-touch intervention.

According to the recent OECD data, China has roughly 2.0 doctors per 1000 inhabitants, much lower than many developed countries such as Canada (2.6), France (3.3), Switzerland (4.3), and the U.S.A. (2.6). Training doctors is a costly and prolonged process, and losing a well-trained doctor incurs a significant loss of human capital. Our study offers policy implications for countries facing a supply shortage in health workers by suggesting that low-cost prosocial nudge interventions can work effectively to retain medical students in the healthcare industry.

These findings lend support to behavioral interventions in other occupations (e.g., nursing and teaching professions) that experienced an upheaval in the pandemic to retain the workforce. On the other hand, low-cost interventions such as an information nudge might be helpful to keep college students in school and prevent early withdrawal. It can be used as a compensating tool with high-cost methods such as financial assistance to achieve the optimal educational outcome.

Appendix A. Supplementary materials

Supplementary Data S1

Supplementary Raw Research Data. This is open data under the CC BY license http://creativecommons.org/licenses/by/4.0/

Supplementary Data S2

Supplementary Raw Research Data. This is open data under the CC BY license http://creativecommons.org/licenses/by/4.0/