Abstract
Objective To determine whether patient outcomes differ between general internists who graduated from a medical school outside the United States and those who graduated from a US medical school.
Design Observational study.
Setting Medicare, USA.
Participants 20% national sample of data for Medicare fee-for-service beneficiaries aged 65 years or older admitted to hospital with a medical condition in 2011-14 and treated by international or US medical graduates who were general internists. The study sample for mortality analysis included 1 215 490 admissions to the hospital treated by 44 227 general internists.
Main outcome measures Patients’ 30 day mortality and readmission rates, and costs of care per hospital admission, with adjustment for patient and physician characteristics and hospital fixed effects (effectively comparing physicians within the same hospital). As a sensitivity analysis, we focused on physicians who specialize in the care of patients admitted to hospital (“hospitalists”), who typically work in shifts and whose patients are plausibly quasi-randomized based on the physicians’ work schedules.
Results Compared with patients treated by US graduates, patients treated by international graduates had slightly more chronic conditions. After adjustment for patient and physician characteristics and hospital fixed effects, patients treated by international graduates had lower mortality (adjusted mortality 11.2% v 11.6%; adjusted odds ratio 0.95, 95% confidence interval 0.93 to 0.96; P<0.001) and slightly higher costs of care per admission (adjusted costs $1145 (£950; €1080) v $1098; adjusted difference $47, 95% confidence interval $39 to $55, P<0.001). Readmission rates did not differ between the two types of graduates. Similar differences in patient outcomes were observed among hospitalists. Differences in patient mortality were not explained by differences in length of stay, spending level, or discharge location.
Conclusions Data on older Medicare patients admitted to hospital in the US showed that patients treated by international graduates had lower mortality than patients cared for by US graduates.
Introduction
International medical graduates make up a quarter of the physician workforce in the United States, United Kingdom, Canada, and Australia.1 2 3 4 5 In the UK, these graduates are largely from India, Republic of Ireland, and Pakistan, and in the US, largely from India, the Philippines, and Pakistan (not including US citizens who have gone abroad for medical education and returned to the US to practice).3 The licensure process for international medical graduates varies between these countries: in the UK, students have to pass an English language and Professional and Linguistic Assessments Board tests so they can register with the UK General Medical Council to practice.6 In the US, students must pass two examinations that test medical knowledge and one examination that assesses clinical skills, and they must complete accredited residency training in the US (see supplementary appendix 1 for details).7 Although international graduates are required to pass these examinations to practice medicine in the UK and US, there have been no UK based or US based standardized accreditations of foreign medical schools, and consequently both policymakers and the public have expressed concerns about the quality of care provided by these graduates.8 9 10 11 In addition, studies have suggested that there may be implicit biases that the quality of care international graduates deliver may be inferior to that of doctors trained domestically, potentially leading to discriminations against physicians who have trained abroad.12 13 14 In response, in the US, the Educational Commission for Foreign Medical Graduates recently announced the requirement for all foreign medical schools to be accredited through a formal process by 2023.15 However, it is unclear whether current licensure processes for international medical graduates provide adequate safeguards for ensuring high quality care.
Though concerns about the quality of care delivered by international medical graduates are often raised, available data comparing patterns of care and outcomes between international medical graduates and doctors trained domestically are limited. The evidence that does exist is inconsistent and has focused on measures such as test scores16 17 18 and process measures of quality.19 The studies that have examined patient outcomes have been small and from a few US states or a single Canadian province, making it difficult to draw conclusions about the quality of care provided by international graduates more broadly.20 21 22 23 Given the substantial public interest in and ongoing concerns about the quality of care such graduates provide in the US, UK, and other high income countries8 9 10 11 and policymakers’ efforts to assure more consistency in foreign medical education, it would be helpful to have empirical data on how international medical graduates perform.
Using a national sample of elderly patients admitted to hospital in the US, we sought to answer three key questions. First, do patients treated by internists who graduated outside the US have different mortality compared with patients treated by internists who graduated from a US medical school? Second, given substantial efforts and interest in reducing rates of readmissions and costs of care, do international graduates have higher readmission rates or provide more costly care than US graduates? Finally, do differences in patient outcomes and costs of care between the two types of graduates, if any, vary by clinical condition?
Methods
We linked four data sources: the 100% Medicare Inpatient Files (2011-14), the 20% Medicare Carrier Files (2011-14), the American Hospital Association annual survey on hospital characteristics, and a comprehensive physician database collected by Doximity (see supplementary appendix 1 for details of this database, including its validation24 25 26). We identified Medicare fee-for-service beneficiaries age 65 years or older who were admitted to the hospital with a medical condition during 1 January 2011 to 31 December 2014, as defined by the presence of a Medicare severity-diagnosis related group on hospital admission. The unit of analysis was hospital admissions. Our sample was restricted to patients admitted to the hospital who were treated in acute care hospitals and excluded elective admissions and those in which a patient left against medical advice. To allow sufficient follow-up periods, we excluded patients admitted in December 2014 from analyses of 30 day mortality and patients discharged in December 2014 from 30 day readmission analyses. For the cost analyses, we restricted our sample to patients with admission and discharge dates in 2011-2014 to ensure that we observed all costs during hospital stay.
We assigned each hospital admission to a physician based on the national provider identifier in the carrier file that accounted for the most Medicare part B spending during that hospital stay.26 27 28 In the US, costs for care of patients admitted to hospital consist of part A spending (payments to hospitals) and part B spending (professional and other fees determined by the physician). We focused on part B spending because it encompasses professional and other services at the discretion of physicians; and for patients admitted to hospital, part A spending is largely invariant to physician decisions because of the fixed payment by Medicare severity-diagnosis related group. On average, 51.1%, 22.0%, and 11.1% of total part B spending was accounted for by the first, second, and third highest spending physicians, respectively. We focused on general internists to avoid comparing physicians across different specialties. Physicians’ specialties were identified using multiple sources, including the data collected by the American Board of Medical Specialties and other specialty societies. We also restricted our analysis to graduates of allopathic medical schools.
Identification of international medical graduates
The Doximity database includes information on the medical school from which a physician graduated. We identified countries where each medical school was located using multiple sources, including the International Medical Education Directory organized by the Foundation for Advancement of International Medical Education and Research,29 the Association of American Medical Colleges database,30 and web pages of individual medical schools. In our study sample, data on graduate medical schools were available for approximately 85% of general internists.
Patient outcomes
The primary outcome was 30 day mortality of patients. Secondary outcomes were 30 day readmission rates and costs of care. We defined the costs of care as total part B spending for each hospital admission.
Adjustment variables
We adjusted for patient characteristics, physician characteristics, and hospital fixed effects (ie, hospital specific indicator variables included as covariates). Patient characteristics included age in five year increments (65-69 years through 90-94 years, and ≥95 years), sex, race or ethnic group (non-Hispanic white, non-Hispanic black, Hispanic, other), primary diagnosis (defined by Medicare severity-diagnosis related group), Elixhauser comorbidity index31 (27 coexisting conditions), median household income estimated from residential zip codes (in deciles), an indicator for dual Medicare-Medicaid coverage, year indicators, and day of the week. Physician characteristics consisted of age in five year increments (<35 years, 35-39 years, and so on through 65-69 years, and ≥70 years), sex, and patient volume (number of Medicare beneficiaries treated as a continuous variable with quadratic and cubic terms to allow for a non-linear relationship). Hospital fixed effects account for both measured and unmeasured characteristics of hospitals, allowing us to effectively compare patient outcomes between international and US medical graduates within the same hospital.32 33 34
Statistical analysis
We first examined whether patient mortality differed between international and US medical graduates using three models. Model 1 compared patient outcomes between the two graduate groups, adjusting for patient characteristics using a multivariable logistic regression model. Model 2 adjusted for all variables in model 1 plus physician characteristics, thereby comparing adjusted patient outcomes of the international graduates with US graduates across hospitals. Model 3 adjusted for all variables in model 2 plus hospital fixed effects, effectively comparing international graduates with US graduates within the same hospital.32 33 34 We clustered standard errors at the physician level.35
We then evaluated whether readmission rates and costs of care differed between the international and US graduates. For the readmission analyses, we used the same set of multivariable logistic regression models as we did for the mortality analyses. For our analyses on costs, we used multivariable ordinary least squares regression models, with the same set of adjustment variables as those used for the analyses of mortality and readmissions. Because we evaluated three outcomes, we considered a P value of less than 0.016 to be statistically significant.
Finally, we assessed whether differences in patient outcomes varied according to the primary condition for which a patient was admitted. We evaluated six major conditions treated by general internists, selected based on frequency: sepsis, pneumonia, congestive heart failure, chronic obstructive pulmonary disease, urinary tract infection, and arrhythmia (see supplementary table A for a list of ICD-9 (international classification of diagnosis, ninth revision) codes for these conditions).
Sensitivity analyses
We conducted sensitivity analyses. First, to address the possibility that the international graduates might treat patients with lesser or greater unmeasured severity of illness, we restricted the study population to patients treated by physicians who specialize in the care of patients admitted to hospital (“hospitalists”) (we focused on general internists (including both hospitalists and non-hospitalist general internists) for main analyses, and hospitalists for the sensitivity analysis). These physicians typically work in shifts, and therefore, within the same hospital, patients treated by them are plausibly quasi-randomized to a given physician based on that hospitalist’s work schedule.26 36 Second, since there are multiple ways to assign physicians to patients, we tested two alternative methods to assign physicians to patients: assigning physicians who had the largest number of evaluation and management claims, and assigning physicians who billed the first evaluation and management claim for a given hospital admission.37 Third, to account for the influence of international medical graduates who were US citizens, we excluded international graduates who graduated from medical schools in Central America and the Caribbean, because three quarters of US citizen international medical students graduate from medical schools in these countries.11 38 39 Fourth, since differences in length of stay, utilization of care (total part B spending per hospital admission), or discharge location might explain differences in patient outcomes between the graduates, we further adjusted our regression models for these variables. Fifth, to deal with the impact of unobserved care preferences of patients, we excluded patients with cancer and patients who were discharged to hospice care.40 Sixth, as international graduates might be more or less likely to work as intensivists in intensive care units, we excluded hospitals with a medical intensive care unit. Seventh, it is possible that residents bill Medicare claims on behalf of their attending physicians at teaching hospitals, and differences in patient outcomes might be related to the quality of care delivered by residents. To address this, we stratified our sample by teaching status of hospitals (major teaching, minor teaching, and non-teaching hospitals), and within each group we compared international medical graduates with US medical graduates (adjusted for patient and physician characteristics and hospital fixed effects). Finally, we examined whether patient outcomes varied by countries where international medical graduates were trained, after restricting to eight countries with the largest number of international medical graduates going to the US (India, Pakistan, Philippines, Syria, Nigeria, Mexico, Egypt, and China) to avoid unstable estimates (see supplementary appendix 1 for more details).
Patient involvement
No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results of the research to study participants or the relevant patient community.
Results
Hospital, physician, and patient characteristics of international graduates
Overall, 44.3% (19 589/44 227) of general internists in the US were international medical graduates. Among the internists, international graduates tended to be younger than US graduates (46.1 v 47.9 years, P<0.001; table 1) and were more likely to work in medium sized, non-teaching for profit hospitals, and hospitals without intensive care units.
Table 1.
Physician and hospital characteristics of general internists who graduated from a medical school outside the US compared with graduates of a US medical school. Values are numbers (percentages) unless stated otherwise
| Characteristics | International graduates (n=19 589) | US graduates (n=24 638) |
|---|---|---|
| Physicians | ||
| Mean (SD) age (years) | 46.1 (10.5) | 47.9 (11.4) |
| Women | 5974 (30.5) | 7709 (31.3) |
| No of hospital admissions per year* | 150.0 | 101.1 |
| Hospitals | ||
| Size (No of beds): | ||
| Large (≥400) | 6632 (34.1) | 10 455 (42.7) |
| Medium (100-399) | 11 426 (58.8) | 12 078 (49.3) |
| Small (1-99) | 1368 (7.0) | 1953 (8.0) |
| Teaching status: | ||
| Major | 3739 (19.3) | 7296 (29.8) |
| Minor | 6667 (34.3) | 8344 (34.1) |
| Non | 9020 (46.4) | 8846 (36.1) |
| Ownership: | ||
| For profit | 2927 (15.1) | 2643 (10.8) |
| Not for profit | 14 790 (76.1) | 18 768 (76.7) |
| Public | 1709 (8.8) | 3075 (12.6) |
| Region: | ||
| North east | 4844 (25.0) | 5220 (21.3) |
| Midwest | 4876 (25.2) | 5626 (23.0) |
| South | 6662 (34.4) | 8465 (34.6) |
| West | 2986 (15.4) | 5149 (21.1) |
| Rural-urban status: | ||
| Urban | 16 711 (86.8) | 20 997 (86.7) |
| Suburban | 437 (2.3) | 426 (1.8) |
| Large rural | 1703 (8.8) | 2234 (9.2) |
| Small rural | 410 (2.1) | 556 (2.3) |
| Intensive care unit: | ||
| Present | 16 712 (85.3) | 21 904 (88.9) |
| Absent | 2877 (14.7) | 2734 (11.1) |
Percentages for hospital characteristics represent proportion of physicians practicing in that type of hospital. P<0.001 for differences in physician characteristics for age and number of admissions to hospital per year, and 0.12 for sex. P<0.001 for differences in hospital characteristics between US graduates and international graduates for all variables except rural-urban status (P=0.12).
*Estimated based on data that the proportion of Medicare beneficiaries with Medicare Advantage plans is 30%, and Medicare beneficiaries comprise approximately 40% of all hospital admissions in the US.
In general, compared with patients treated by US graduates, those treated by international graduates were more likely to be non-white, have lower median household income, have Medicaid coverage, and have more comorbid conditions, including congestive heart failure, chronic obstructive pulmonary disease, and diabetes (table 2).
Table 2.
Characteristics of patients treated by general internists who graduated from a medical school outside the US compared with graduates of a US medical school. Values are numbers (percentages) unless stated otherwise
| Characteristics | International graduates (n=19 589) | US graduates (n=24 638) |
|---|---|---|
| No of patients | 657 565 | 557 925 |
| Mean (SD) patient’s age (years) | 80.6 (8.5) | 80.9 (8.5) |
| Women | 401 396 (61.0) | 337 538 (60.5) |
| Race: | ||
| White | 523 310 (79.6) | 462 675 (82.9) |
| Black | 75 035 (11.4) | 56 819 (10.2) |
| Hispanic | 40 420 (6.2) | 21 269 (3.8) |
| Other | 18 800 (2.9) | 17 162 (3.1) |
| Mean (SD) household income* ($) | 56 146 (22 135) | 57 128 (23 111) |
| Medicaid status | 172 505 (26.2) | 122 375 (21.9) |
| Coexisting condition: | ||
| Congestive heart failure | 134 054 (20.4) | 111 215 (19.9) |
| Chronic obstructive pulmonary disease | 173 198 (26.3) | 142 539 (25.6) |
| Diabetes | 217 387 (33.1) | 174 289 (31.2) |
| Renal failure | 144 687 (22.0) | 121 016 (21.7) |
| Neurological disorders | 106 692 (16.2) | 88 380 (15.8) |
| Cancer | 44 938 (6.8) | 41 250 (7.4) |
| Mental illness | 101 851 (15.5) | 85 676 (15.4) |
| Discharge location: | ||
| Home | 376 651 (57.3) | 324 360 (58.1) |
| Skilled nursing facility | 178 510 (27.2) | 143 658 (25.8) |
| Rehabilitation facility | 16 914 (2.6) | 13 508 (2.4) |
| Hospice | 28 557 (4.3) | 26 479 (4.8) |
| Other | 56 933 (8.7) | 49 920 (9.0) |
$1.00 (£0.83; €0.94).
P<0.001 for differences between US graduates and international graduates for all patient characteristics except renal failure (P=0.005) and mental illness (P=0.22).
*Estimated from beneficiary zip code of residence.
Mortality
Our final sample for mortality analysis included 1 215 490 patients who were treated by 44 227 physicians. The overall 30 day mortality was 11.4% (138 519/1 215 490); unadjusted mortality was 11.0% for international graduates (72 269/657 565) compared with 11.9% for US graduates (66 250/557 925). Patients treated by the international graduates had lower risk adjusted mortality compared with those treated by US graduates (adjusted mortality 11.1% v 11.7%; adjusted odds ratio 0.94, 95% confidence interval 0.92 to 0.95; P<0.001; table 3). This relationship remained essentially unchanged after additional adjustment for physician characteristics (adjusted mortality 11.2% v 11.6%; adjusted odds ratio 0.96, 0.94 to 0.98; P<0.001), or after further adjustment for hospital fixed effects, which effectively compared the international graduates with US graduates within the same hospital (adjusted mortality 11.2% v 11.6%; adjusted odds ratio 0.95, 0.93 to 0.96; P<0.001).
Table 3.
Patient 30 day mortality between general internists who graduated from a medical school outside the US compared with graduates of a US medical school. Adjusted patient outcomes are percentages
| Models | No of hospital admissions (No of physicians) | Adjusted 30 day mortality (95% CI) | Adjusted odds ratio (95% CI) international v US graduates | P value | |
|---|---|---|---|---|---|
| International graduates | US graduates | ||||
| Model 1: risk adjusted 30 day mortality* | 1 215 490 (44 227) | 11.1 (11.1 to 11.2) | 11.7 (11.6 to 11.8) | 0.94 (0.92 to 0.95) | <0.001 |
| Model 2: model 1+physician characteristics† | 1 181 913 (42 854) | 11.2 (11.1 to 11.3) | 11.6 (11.5 to 11.7) | 0.96 (0.94 to 0.98) | <0.001 |
| Model 3: model 2+hospital fixed effects‡ | 1 180 879 (42 710) | 11.2 (11.1 to 11.3) | 11.6 (11.5 to 11.7) | 0.95 (0.93 to 0.96) | <0.001 |
*Risk adjustment using patient age, sex, race, primary diagnosis, coexisting conditions (Elixhauser comorbidity index), median household income, Medicaid status, year indicators, and day of week.
†Include age, sex, and patient volume.
‡Hospital fixed effects allowed an effective comparison of patient outcomes between international and US graduates within the same hospital.
Readmission rates and costs of care
Our final sample, for the analysis of readmission, comprised of 1 182 268 patients who were treated by 44 201 physicians. The overall 30 day readmission rate was 15.4% (182 239/1 182 268); the unadjusted readmission rate was 16.0% for the international graduates (102 173/639 661) and 14.8% for the US graduates (80 066/542 607). Patients treated by the international graduates had higher risk adjusted readmission rates than those treated by the US graduates (15.9% v 14.9%; adjusted odds ratio 1.08, 1.07 to 1.09; P<0.001; table 4). This relationship remained unaffected after additional adjustment for physician characteristics. However, when we further adjusted for hospital fixed effects, the readmission rates no longer differed between the graduates (adjusted readmission rate 15.4% v 15.5%; adjusted odds ratio 1.00, 0.98 to 1.01; P=0.54), indicating that higher unadjusted readmission rates for the international graduates were driven by differences in the hospitals in which they work (ie, international graduates tend to practice in hospitals with higher readmission rates).
Table 4.
Patient 30 day readmission rates and costs of care between general internists who graduated from a medical school outside the US compared with graduates of a US medical school. Adjusted patient outcomes are percentages, unless stated otherwise
| Variables | No of hospital admissions (No of physicians) | Adjusted patient outcomes (95% CI) | Adjusted odds ratio/difference (95% CI) international v US graduates | P value | |
|---|---|---|---|---|---|
| International graduates | US graduates | ||||
| 30 day readmission rate: | |||||
| Model 1: risk adjusted 30 day readmissions* | 1 182 268 (44 201) | 15.9 (15.8 to 16.0) | 14.9 (14.8 to 15.0) | 1.08 (1.07 to 1.09) | <0.001 |
| Model 2: model 1+physician characteristics† | 1 149 716 (42 833) | 15.8 (15.7 to 15.9) | 15.0 (14.9 to 15.1) | 1.06 (1.05 to 1.07) | <0.001 |
| Model 3: model 2+hospital fixed effects‡ | 1 149 024 (42 667) | 15.4 (15.3 to 15.5) | 15.5 (15.4 to 15.6) | 1.00 (0.98 to 1.01) | 0.54 |
| Total part B costs per hospital admission: | |||||
| Model 1: risk adjusted costs per hospital admissions* | 1 276 559 (44 680) | $1192 ($1184 to $1201) | $1041 ($1034 to $1047) | $152 ($141 to $162) | <0.001 |
| Model 2: model 1+physician characteristics† | 1 240 872 (42 944) | $1195 ($1187 to $1203) | $1038 ($1031 to $1046) | $157 ($146 to $168) | <0.001 |
| Model 3: model 2+hospital fixed effects‡ | 1 240 867 (42 944) | $1145 ($1140 to $1150) | $1098 ($1093 to $1103) | $47 ($39 to $55) | <0.001 |
$1.00 (£0.83; €0.94).
*Risk adjustment using patients’ age, sex, race, primary diagnosis, coexisting conditions (Elixhauser comorbidity index), median household income, Medicaid status, year indicators, and day of week.
†Include age, sex, and patient volume.
‡Hospital fixed effects allowed an effective comparison of patient outcomes between international and US graduates within the same hospital.
Our final sample, for analysis of costs, consisted of 1 276 559 patients who were treated by 44 680 physicians. Costs of care were slightly higher for the international graduates than US graduates. After adjusting for patient and physician characteristics and hospital fixed effects, the international graduates had slightly higher costs of care ($1145 v $1098; adjusted difference $47, 95% confidence interval $39 to $55; P<0.001).
Patient outcomes for specific clinical conditions
The international graduates had lower patient mortality than the US graduates for pneumonia and congestive heart failure. We observed a trend toward lower mortality for all conditions except urinary tract infections, but the differences were not statistically significant for some conditions (fig 1 and supplementary table B). We observed no statistically significant difference in adjusted readmission rates between the US graduates and international graduates across all conditions we studied. Costs of care were slightly higher and statistically significant for the international graduates across all conditions we studied (fig 2).
Fig 1 Patient outcomes between international (IMGs) and US medical graduates (USMGs), by primary diagnosis. (Top) Adjusted 30 day mortality. *P<0.05 significant. **P<0.01 significant. (Bottom) Adjusted 30 day readmission rate. For all conditions P>0.05 for difference. Risk adjusted for patient and physician characteristics and hospital fixed effects. Standard errors were clustered at physician level
Fig 2 Adjusted total part B costs per hospital admission between international (IMGs) and US medical graduates (USMGs). ***P<0.001 significant. Risk adjusted for patient and physician characteristics and hospital fixed effects. Standard errors were clustered at physician level
Additional analyses
Hospitalists who were international graduates treated patients who were similar across a broad range of characteristics compared with hospitalists who were US graduates, supporting our hypothesis that patients treated by hospitalists are quasi-randomized based on physician work schedules (see supplementary table C). Among hospitalists, the international graduates had lower patient mortality than the US graduates (adjusted mortality 10.8% v 11.4%; adjusted odds ratio 0.94, 0.91 to 0.96; P<0.001; see supplementary table D). Our overall findings among general internists (including both hospitalists and non-hospitalist general internists) were not qualitatively affected by assigning patients to physicians on the basis of evaluation and management claims; exclusion of international graduates who trained at medical schools in Central America and the Caribbean; additional adjustment for length of hospital stay, utilization of care, or discharge location; exclusion of patients with cancer or who were discharged to hospice care; and exclusion of hospitals with medical intensive care units (see supplementary tables E-G). Differences in patient outcomes between the international and US graduates were consistent across groups defined by the teaching status of hospitals (see supplementary table H). Patient outcomes varied by country where the students were trained, although our statistical power to detect differences by individual countries was limited (see supplementary figs A-C).
Discussion
Using a nationally representative sample of elderly patients admitted to the hospital in the United States, we found no evidence that patient outcomes for graduates who had trained outside of the US were worse than for graduates from a US medical school. If any, patients treated by the international graduates had lower 30 day mortality than those treated by the US graduates. These differences persisted across a broad range of clinical conditions, and even among hospitalists, where patient selection might be less of a concern. We found no differences in readmission rates between patients cared for by international or US graduates, and slightly higher spending for the international graduates. Taken together, our findings should reassure policymakers and the public that our current approach to licensing international medical graduates in the US is sufficiently rigorous to ensure high quality care.
There are several explanations for why international graduates might have better patient outcomes than US graduates. The current approach for allowing international medical graduates to practice in the US may select for, on average, better physicians. Indeed, the match rate for US residency programs is substantially lower for international medical graduates (49.4% ) than for US medical graduates (94.0% for graduates of US allopathic medical schools),41 and therefore, it is possible that international graduates who are successful in the US matching process might represent some of the best physicians in their country of origin. The fact that the international graduates outperform the US graduates in test scores in the US lends some credence to this hypothesis.16 In addition, many of the international graduates who are currently practicing in the US likely underwent residency training twice, once in their home country and once in the US, and such intensive and prolonged training may be another reason why they might perform better. It is also possible that the international graduates might be more concerned about professional failure, and therefore are more engaged in continuous training and updating their skills and knowledge base, although we lack evidence to support or refute this hypothesis.
The difference in patient mortality between the international and US graduates, an adjusted odds ratio of 0.95, is at most a modest clinical significance. Based on the risk difference of 0.4 percentage points, for every 250 patients treated by US medical graduates, one patient’s life would be saved if the quality of care were equivalent between the international graduates and US graduates. This difference in mortality is comparable to the reduction in mortality we have seen in all cause inpatient mortality between 2000 and 2010 across US hospitals,42 which is often recognized as the result of national quality improvement efforts, development of clinical guidelines, and new treatments.
Strengths and limitations of this study
The main strengths of this study were its use of nationally representative data, and a natural experiment study design (analysis focusing on hospitalists) to minimize the impact of unmeasured confounders. Patients treated by hospitalists are plausibly quasi-randomized based on when patients seek medical care and on the hospitalist’s work schedules. Although it is still possible that the severity of illness of patients seen by the international and US graduates might differ in unobservable characteristics, our data showed that, if any, the international graduates appear to treat patients with higher rates of chronic disease and lower socioeconomic status than do the US graduates. For example, we found that patients treated by the international graduates had lower socioeconomic status (eg, more often identified as being from racial and ethnic minorities, being from lower income neighborhoods, and having higher Medicaid coverage), which is generally associated with worse outcomes. Moreover, we are unaware of any other studies that suggest that the international graduates treat systematically healthier patients than do the US graduates.
Our study has limitations. First, we could not distinguish between foreign born international medical graduates and US citizens who travel abroad for medical training. Approximately 73% of US citizens who are international medical graduates attend medical schools in Central America and the Caribbean.11 38 39 Excluding physicians from these countries in our sensitivity analysis did not affect our findings. However, the results of our analyses by country suggest that there might be some heterogeneity among the international graduates based on where they trained and their reasons for coming to the US. Second, patient mortality may appear lower for the international graduates if coding comorbidities for their patients was more complete. While this is theoretically possible, it is unlikely that coding practices would vary substantially within the same hospital. Further, we observed lower unadjusted mortality for patients treated by the international graduates, which was not affected by coding practice. Finally, although 30 day mortality and readmission rates are widely accepted measures of quality for hospital care, they are not comprehensive. There are other aspects of inpatient care, such as patient experience, which we did not measure.
Comparison with other studies
The findings from our study are consistent with those from a limited set of smaller studies. A study of physicians practicing in Pennsylvania, US found that in-hospital mortality among patients with medical conditions was lower for international graduates compared with US graduates.20 However, another analysis of medical conditions from Ontario, Canada,22 and a recent study of surgeons from two US states,23 reported no difference in mortality between international graduates and locally trained doctors, suggesting that the quality of care provided by the international graduates might depend on specialties, or on the rigorousness of the licensure process which varies from country to country.
Conclusions and policy implications
Using national data on Medicare beneficiaries admitted to hospital, we found no evidence that patient outcomes were worse for those treated by international medical graduates than for those treated by US medical graduates. To an extent there was a difference: patients treated by the international graduates appeared to have somewhat lower mortality than patients treated by the US graduates. Our findings indicate that current standards of selecting international medical graduates for practice in the US are functioning well for at least one important dimension: inpatient outcomes. As we consider expanding our physician workforce, these results suggest that systems modeled on the current rigorous approach to incorporate international medical graduates should allow for better access to care and good outcomes. Further research is warranted to better understand whether patient outcomes differ between international medical graduates and domestically trained doctors for other specialties (eg, psychiatry, intensive care, geriatric care), in the outpatient setting, in the younger populations, and in other high income countries.
What is already known on this topic
Few studies have examined patient outcomes between international medical graduates and physicians trained domestically in high income countries
Small studies found mixed results as to whether patient outcomes differ between these two groups of physicians
No study has investigated differences in patient outcomes between international medical graduates and US medical graduates using nationally representative data
What this study adds
Among Medicare beneficiaries admitted to hospital, 30 day adjusted mortality rates were lower for international medical graduates than for US medical graduates, despite international graduates caring for patients with higher rates of chronic conditions
Current standards of selecting international medical graduates for practice in the US appear sufficiently rigorous to ensure high quality care
Web extra.
Extra material supplied by the authors
Supplementary information: Supplementary tables and figures
Contributors: All authors conceived and designed the study, analyzed and interpreted the data, and critically revised the manuscript for important intellectual content. ABJ and AKJ acquired the data. YT and EJO carried out the statistical analysis. YT drafted the manuscript and is the guarantor.
Funding: This study received no support from any organization.
Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf. YT was supported in part by the Abe Fellowship Program (administered by Social Science Research Council with funds from the Japan Foundation Center for Global Partnership) and St Luke’s International University. ABJ was supported by the Office of the Director, National Institutes of Health (NIH early independence award, grant 1DP5OD017897-01). ABJ reports receiving consulting fees unrelated to this work from Pfizer, Hill Rom Services, Bristol Myers Squibb, Novartis Pharmaceuticals, Vertex Pharmaceuticals, and Precision Health Economics (a company providing consulting services to the life sciences industry).
Ethical approval: This study was reviewed and granted exemption by the institutional review board of Harvard Medical School.
Data sharing: No additional data available.
Transparency: The lead author (YT) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
References
- 1.Brotherton SE, Etzel SI. Graduate Medical Education, 2014-2015. JAMA 2015;314:2436-54. 10.1001/jama.2015.10473 pmid:26647275. [DOI] [PubMed] [Google Scholar]
- 2.General Medical Council. The state of medical education and practice in the UK. 2015.
- 3.Mullan F. The metrics of the physician brain drain. N Engl J Med 2005;353:1810-8. 10.1056/NEJMsa050004 pmid:16251537. [DOI] [PubMed] [Google Scholar]
- 4.Thompson MJ, Hagopian A, Fordyce M, Hart LG. Do international medical graduates (IMGs) “fill the gap” in rural primary care in the United States? A national study. J Rural Health 2009;25:124-34. 10.1111/j.1748-0361.2009.00208.x pmid:19785577. [DOI] [PubMed] [Google Scholar]
- 5.Whelan GP, Gary NE, Kostis J, Boulet JR, Hallock JA. The changing pool of international medical graduates seeking certification training in US graduate medical education programs. JAMA 2002;288:1079-84. 10.1001/jama.288.9.1079 pmid:12204075. [DOI] [PubMed] [Google Scholar]
- 6.General Medical Council. Applying for registration as an international medical graduate. www.gmc-uk.org/doctors/before_you_apply/imgs.asp.
- 7.American Medical Association. International medical graduates. Chicago, IL. www.ama-assn.org/life-career/international-medical-graduate.
- 8.Delamothe T. The plight of the international medical graduate. BMJ 2014;348:g2894 10.1136/bmj.g2894. [DOI] [Google Scholar]
- 9.Path to United States Practice Is Long Slog to Foreign Doctors. New York Times Aug 11 2013.
- 10.Doctors With Borders: How the U.S. Shuts Out Foreign Physicians. Atlantic 18 Nov 2014.
- 11.Eckhert NL, van Zanten M. U.S.-citizen international medical graduates--a boon for the workforce?N Engl J Med 2015;372:1686-7. 10.1056/NEJMp1415239 pmid:25923548. [DOI] [PubMed] [Google Scholar]
- 12.Desbiens NA, Vidaillet HJ Jr. Discrimination against international medical graduates in the United States residency program selection process. BMC Med Educ 2010;10:5 10.1186/1472-6920-10-5 pmid:20100347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Esmail A, Roberts C. Academic performance of ethnic minority candidates and discrimination in the MRCGP examinations between 2010 and 2012: analysis of data. BMJ 2013;347:f5662 10.1136/bmj.f5662 pmid:24072882. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Baldwin DC Jr, , Daugherty SR, Rowley BD. Racial and ethnic discrimination during residency: results of a national survey. Acad Med 1994;69(Suppl):S19-21. 10.1097/00001888-199410000-00029 pmid:7916815. [DOI] [PubMed] [Google Scholar]
- 15.ECFMG. Medical School Accreditation Requirement for ECFMG Certification Philadelphia, PA: ECFMG; 2010. www.ecfmg.org/about/initiatives-accreditation-requirement.html.
- 16.Garibaldi RA, Subhiyah R, Moore ME, Waxman H. The In-Training Examination in Internal Medicine: an analysis of resident performance over time. Ann Intern Med 2002;137:505-10. 10.7326/0003-4819-137-6-200209170-00011 pmid:12230352. [DOI] [PubMed] [Google Scholar]
- 17.McManus IC, Wakeford R. PLAB and UK graduates’ performance on MRCP(UK) and MRCGP examinations: data linkage study. BMJ 2014;348:g2621 10.1136/bmj.g2621 pmid:24742473. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Tiffin PA, Illing J, Kasim AS, McLachlan JC. Annual Review of Competence Progression (ARCP) performance of doctors who passed Professional and Linguistic Assessments Board (PLAB) tests compared with UK medical graduates: national data linkage study. BMJ 2014;348:g2622 10.1136/bmj.g2622 pmid:24742539. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Reid RO, Friedberg MW, Adams JL, McGlynn EA, Mehrotra A. Associations between physician characteristics and quality of care. Arch Intern Med 2010;170:1442-9. 10.1001/archinternmed.2010.307 pmid:20837830. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Norcini JJ, Boulet JR, Dauphinee WD, Opalek A, Krantz ID, Anderson ST. Evaluating the quality of care provided by graduates of international medical schools. Health Aff (Millwood) 2010;29:1461-8. 10.1377/hlthaff.2009.0222 pmid:20679648. [DOI] [PubMed] [Google Scholar]
- 21.Norcini JJ, Boulet JR, Opalek A, Dauphinee WD. Outcomes of cardiac surgery: associations with physician characteristics, institutional characteristics, and transfers of care. Med Care 2013;51:1034-9. 10.1097/MLR.0b013e3182a048af pmid:23929400. [DOI] [PubMed] [Google Scholar]
- 22.Ko DT, Austin PC, Chan BT, Tu JV. Quality of care of international and Canadian medical graduates in acute myocardial infarction. Arch Intern Med 2005;165:458-63. 10.1001/archinte.165.4.458 pmid:15738378. [DOI] [PubMed] [Google Scholar]
- 23.Zaheer S, Pimentel SD, Simmons KD, et al. Comparing International and United States Undergraduate Medical Education and Surgical Outcomes Using a Refined Balance Matching Methodology. Ann Surg 2016; Epub ahead of print 10.1097/SLA.0000000000001878 pmid:27429031. [DOI] [PubMed] [Google Scholar]
- 24.Jena AB, Khullar D, Ho O, Olenski AR, Blumenthal DM. Sex Differences in Academic Rank in US Medical Schools in 2014. JAMA 2015;314:1149-58. 10.1001/jama.2015.10680 pmid:26372584. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Jena AB, Olenski AR, Blumenthal DM. Sex Differences in Physician Salary in US Public Medical Schools. JAMA Intern Med 2016;176:1294-304. 10.1001/jamainternmed.2016.3284 pmid:27400435. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Tsugawa Y, Jena AB, Figueroa JF, Orav EJ, Blumenthal DM, Jha AK. Comparison of Hospital Mortality and Readmission Rates for Medicare Patients Treated by Male vs Female Physicians. JAMA Intern Med 2016; published online 19 Dec. 10.1001/jamainternmed.2016.7875 pmid:27992617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.McWilliams JM, Hatfield LA, Chernew ME, Landon BE, Schwartz AL. Early Performance of Accountable Care Organizations in Medicare. N Engl J Med 2016;374:2357-66. 10.1056/NEJMsa1600142 pmid:27075832. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.McWilliams JM, Landon BE, Chernew ME, Zaslavsky AM. Changes in patients’ experiences in Medicare Accountable Care Organizations. N Engl J Med 2014;371:1715-24. 10.1056/NEJMsa1406552 pmid:25354105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.FAIMER (Foundation for Advancement of International Medical Education and Research). International Medical Education Directory (IMED) Philadelphia, PA: Foundation for Advancement of International Medical Education and Research (FAIMER); www.faimer.org/resources/imed.html.
- 30.Association of American Medical Colleges. Association of American Medical Colleges. www.aamc.org.
- 31.Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care 1998;36:8-27. 10.1097/00005650-199801000-00004 pmid:9431328. [DOI] [PubMed] [Google Scholar]
- 32.Gunasekara FI, Richardson K, Carter K, Blakely T. Fixed effects analysis of repeated measures data. Int J Epidemiol 2014;43:264-9. 10.1093/ije/dyt221 pmid:24366487. [DOI] [PubMed] [Google Scholar]
- 33.Allison PD. Fixed effects regression methods for longitudinal data using SAS.Sas Institute, 2005. [Google Scholar]
- 34.Kaufman JS. Commentary: Why are we biased against bias?Int J Epidemiol 2008;37:624-6. 10.1093/ije/dyn035 pmid:18316349. [DOI] [PubMed] [Google Scholar]
- 35.Arellano M. PRACTITIONERS’CORNER: Computing Robust Standard Errors for Within-groups Estimators. Oxf Bull Econ Stat 1987;49:431-4 10.1111/j.1468-0084.1987.mp49004006.x. [DOI] [Google Scholar]
- 36.Hinami K, Whelan CT, Miller JA, Wolosin RJ, Wetterneck TB. Society of Hospital Medicine Career Satisfaction Task Force. Job characteristics, satisfaction, and burnout across hospitalist practice models. J Hosp Med 2012;7:402-10. 10.1002/jhm.1907 pmid:22271510. [DOI] [PubMed] [Google Scholar]
- 37.Mehrotra A, Adams JL, Thomas JW, McGlynn EA. The effect of different attribution rules on individual physician cost profiles. Ann Intern Med 2010;152:649-54.pmid:20479030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.McAvinue MB, Boulet JR, Kelly WC, Seeling SS, Opalek A. U.S. citizens who graduated from medical schools outside the United States and Canada and received certification from the Educational Commission for Foreign Medical Graduates, 1983-2002. Acad Med 2005;80:473-8. 10.1097/00001888-200505000-00015 pmid:15851461. [DOI] [PubMed] [Google Scholar]
- 39.Norcini J, Anderson MB, McKinley DW. The medical education of United States citizens who train abroad. Surgery 2006;140:338-46. 10.1016/j.surg.2006.06.001 pmid:16934591. [DOI] [PubMed] [Google Scholar]
- 40.Covinsky KE, Fuller JD, Yaffe K, et al. The Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments. Communication and decision-making in seriously ill patients: findings of the SUPPORT project. J Am Geriatr Soc 2000;48(Suppl):S187-93. 10.1111/j.1532-5415.2000.tb03131.x pmid:10809474. [DOI] [PubMed] [Google Scholar]
- 41.National Resident Matching Program. Results and Data: 2015 Main Residency Match®. Washington, DC; 2015. [Google Scholar]
- 42.Hall MJ, Levant S, DeFrances CJ. Trends in Inpatient Hospital Deaths: National Hospital Discharge Survey, 2000–2010.National Center for Health Statistics, 2013. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary information: Supplementary tables and figures