Critical elements of international academic partnerships in orthopaedic surgery: a modified Delphi approach

Abstract

Background:

Despite the recent emphasis on promoting international collaborations within orthopaedic surgery, criteria for determining the strengths of such partnerships has not been established. The purpose of this study was to evaluate orthopaedic experts' perceptions of the most valuable characteristics of international academic partnerships.

Methods:

This study was conducted using a modified Delphi methodology. Experts were identified through the Consortium of Orthopaedic Academic Traumatologists (COACT). Responses were collected from February to September 2022. Three rounds of surveys listing possible topics on a 5-point Likert scale were used to develop consensus among a group of experts. Consensus criteria for topic inclusion in the final scale was determined as a rating of “strongly agree” or “agree” by ≥70% of the participants in the third survey.

Results:

The Round 1 survey was distributed to 96 invited participants within the COACT network, of which 50 experts (52.1%) completed the first survey. Consensus was reached on 54 topics organized into the following 5 categories: Research, Advocacy/Leadership, Training/Surgical Skills, Education/Knowledge Exchange, and Sustainability and Safety (RATES Criteria).

Conclusions:

Determining the most valuable characteristics of successful international academic partnerships can lead to more sustainable, mutually beneficial collaborations. The criteria identified in this study can provide the foundation for developing new partnerships and assessing existing ones.

1. Introduction

Academic exchange between orthopaedic scholars from various countries and geographical regions can enhance musculoskeletal care worldwide. Within orthopaedic surgery, there has been an increased emphasis on promoting international partnerships in recent years. These partnerships seek holistic and sustainable collaboration, with initiatives including research, clinical exchanges, and fellowship programs to further orthopaedic surgical knowledge and techniques.^1–5 However, despite a recent focus on establishing international collaborations within orthopaedic surgery, the characteristics that comprise successful partnerships have not been established. Studies that have concentrated on the role of collaborative surgical training and education in orthopaedic surgery note that international partnerships should include mutual learning, directed goals, and site champions.⁶ However, no consensus has been determined on the most critical elements that comprise international academic partnerships. The purpose of this study was to evaluate orthopaedic experts' perceptions of the most valuable characteristics of international academic partnerships.

2. Methods

This study was conducted using a modified Delphi process, modeled after previous studies in the orthopaedic literature.^7–11 This work was approved by the Institutional Review Board (IRB) at Yale University, and participant consent was obtained. The secure database REDCap (Nashville, TN) was used to create, distribute, and store survey data.^12,13 Responses were collected from February to September 2022.

2.1. Delphi Round 1

The first stage used a survey soliciting characteristics that could be considered valuable for an international academic partnership. Experts were identified through the Consortium of Orthopaedic Academic Traumatologists (COACT), a network of experienced orthopaedic surgeons from the United States and Canada who currently participate in global outreach.¹⁴

To participate in the study, each identified expert must have had prior experience in international academic activities and an existing global partnership. Surgeon-experts were contacted via email with a survey link.

Participant demographic information was collected, including age, sex, country, years in practice, and practice type. Participants were encouraged to list important elements that they perceived as critical for international academic partnerships through a free-text box. There was no limit to the number of items that a participant could submit. Survey results were subsequently compiled by the primary research team and grouped into 5 general areas: Research, Advocacy/Leadership, Training/Surgical Skills, Education/Knowledge Exchange, and Sustainability and Safety (RATES Criteria, Appendix, http://links.lww.com/OTAI/A95).

2.2. Delphi Round 2

The topics identified in the Delphi Round 1 were listed on a second electronic survey and distributed to all participants. Participants were then asked to review each item for its perceived value to international academic partnerships and rate it on a 5-point Likert scale from 1 (“strongly agree”) to 5 (“strongly disagree”). Responses from Round 2 surveys were compiled and analyzed.

2.3. Delphi Round 3

Delphi Round 3 was the third survey sent to participants. Participants were asked to re-rate each item for its perceived value to international academic partnerships using the same Likert scale. This survey included a histogram illustrating the collective ratings for each topic from Round 2.

At the completion of Round 3, the results were calculated with an aggregate rating using STATA version 15.0. Each topic was given an overall mean score based on the Likert scale. Topics that were rated as “strongly agree” or “agree” by ≥ 70% of participants were identified.

3. Results

3.1. Demographics and Round 1

Initially, invitations to participate were distributed electronically to 96 potential participants through the COACT network, which included study information, informed consent, and a link to the Round 1 survey. Of those 96 potential participants, 50 experts consented to participate and subsequently completed the Round 1 survey. Most of the participants were from urban (90%) academic centers (78%) in the United States (74%) (Table 1). The majority of participants were male (90%), with a mean age of 51.5 years (range 34–77). Nearly half of the participants had more than 20 years of experience (42%). After consolidation and deduplication of responses, experts listed 95 unique topics.

TABLE 1.

Participant Demographics

Category Total	N = 50, N (%)
Country
Canada	4 (8)
Ghana	2 (4)
Haiti	1 (2)
Mexico	1 (2)
Nepal	1 (2)
Nicaragua	1 (2)
Switzerland	1 (2)
Tanzania	1 (2)
Uganda	1 (2)
United States	37 (74)
Mean age and range in years	51.5 (34–77)
Sex
Female	5 (10)
Male	45 (90)
Years in practice
0-5	7 (14)
6-10	7 (14)
11-15	11 (22)
16-20	4 (8)
>20	21 (42)
Practice type
Academic	39 (78)
Private	4 (8)
Public	4 (8)
Both	3 (6)
Practice setting
Urban	45 (90)
Rural	3 (6)
Both	1 (2)
Conflict zone	1 (2)

3.2. Rounds 2 and 3

Thirty-nine experts (75% response rate) completed the Round 2 survey. Overall ratings and the percentage of consensus among participants who selected either “strongly agree” or “agree” were determined (Table 2). There were 48 experts (96% response rate) who completed the Round 3 survey. In total, 54 topics reached consensus and were included in the scale, all of which were rated as “strongly agree” or “agree” by ≥70% of participants.

TABLE 2.

Overall Ratings of Partnership Topics and the Percentage of Consensus Among Participants

Topics (Round 1)	Round 2 Mean*	Round 3 Consensus†
Research
Two-way exchange of research data/findings	1.32	100.0%
Projects are developed collaboratively and are relevant to LRP	1.26	97.8%
LRPs listed as authors on publications	1.24	95.7%
Research mentorship/training available	1.61	91.3%
Published articles	1.74	87.0%
Conducting research project(s) with <3 level of evidence	1.78	84.8%
Presented work(s) at a conference/event	1.89	82.6%
Advocacy/leadership
Partners pursue shared goals	1.32	95.7%
Regular two-way feedback	1.41	95.7%
Bidirectional ethics/trust has been discussed in meetings	1.53	91.3%
Regular contact (in-person, email, video chat, phone calls, etc.)	1.47	91.3%
HRP has discussed their perceived role to LRP	1.69	89.1%
LRP desires to engage in partnership	1.32	87.0%
Established a Memorandum of Understanding (MOU)	1.95	82.6%
Bidirectional cultural humility awareness has been discussed in meetings	1.61	80.4%
Funding available for LRP to present at conferences	1.97	78.3%
Long-standing partnership (>3 years)	2.14	73.9%
Training/surgical skills
Funding available/received for LRP to attend trainings	1.53	93.5%
Training resources available	1.58	93.5%
Protocols in place for clinical teaching	1.83	91.3%
Virtual trainings available	1.91	91.3%
Provide didactic session(s)	1.81	87.0%
Provide surgical skills training session(s)	1.75	87.0%
Funding available/received for developing training resources	2.08	71.7%
Exchange of knowledge/education
Fellowship and/or observership available for surgeons	1.97	93.5%
LRP participates in curriculum development	1.69	91.3%
HRP volunteers at LRP institution	2.08	89.1%
Remote educational opportunities available	1.74	89.1%
Elective and/or fellowship available for trainees	2.08	87.0%
Grand rounds/case discussions/journal clubs available	1.77	87.0%
Journal/resource access available for LRP	1.61	87.0%
Access to recorded lectures	1.83	80.4%
Funding available for exchange(s)	1.89	76.1%
Length/timing of trip/exchange >2 weeks	2.00	73.9%
Two-way exchange opportunities available (both HRP and LRP exchange)	1.94	71.7%
Sustainability and safety
Regular review of challenges faced by partnership	1.81	95.7%
Improvement tracked over time	1.69	93.5%
Mechanisms available to ensure appropriate clinical follow-up	1.58	93.5%
Access to translators/translations	1.72	91.3%
Improvement in patient care as a result of the partnership	1.61	91.3%
Mechanisms available to ensure quality of care	1.50	91.3%
Partnership is economically stable	1.72	91.3%
LRP conducts research/surgical skills when HRP is not in-person	1.75	84.8%
Quality improvement projects conducted at LRP institutions	2.00	84.8%
Treat patients with and without insurance	1.67	82.6%
Housing available for visiting surgeons/trainees	1.81	80.4%
Measure of researcher/trainee safety in country available	2.06	80.4%
Clinical support available via WhatsApp	2.28	78.3%
Surgical implants available for LRP	2.03	78.3%
Leadership training available	1.92	76.1%
LRP has technological capabilities (Internet, Electronic Medical Record, Zoom, etc.)	2.00	76.1%
Audits on surgeries taught and conducted	2.08	73.9%
HRP provides needed research equipment	2.22	71.7%
Measure of surgeon decision making available	2.22	71.7%

HRP = high-resource partner; LRP = low-resource partner.

^*The mean score with 1 being “strongly agree” and 5 being “strongly disagree.”

^†Consensus was defined as ≥70% of participants rating the topic as “strongly agree” or “agree.”

Overall, the topics that reached consensus and were most highly rated by study participants were related to effective collaboration and communication, including bidirectional exchange of data (100%), collaborative project development (97.8%), representation of low- and middle-income countries (LMICs) in publications (95.7%), and the pursuit of shared goals (95.7%). Similarly, experts highly rated bidirectional feedback (95.7%) and review of challenges faced by the partnership (95.7%).

4. Discussion

This study used a modified Delphi method to evaluate orthopaedic experts' perceptions of the most valuable characteristics of international academic partnerships. The results showed that scholars within the field of orthopaedic surgery across geographical settings prioritized mutual engagement throughout the entirety of a collaboration. This engagement includes partners' concerted efforts to select common goals, effective communication regarding barriers, and equal representation throughout the publishing process. The results of this modified Delphi study indicate that experts value partnership equity.

The results of this study could provide a foundation to develop a more objective measure of a partnership. Scales have been created to identify international academic partnerships in various surgical specialty areas. In one study in the neurosurgical literature, the authors used a 3-level “engagement grade,” where level I represented the lowest score and III the highest score. This system included qualifiers of frequency, duration, and definition of a partnership to determine its level of engagement.¹⁵ Though not validated, the scale was also used to evaluate training and engagement among ophthalmology partnerships.¹⁶ Furthermore, in a study published in the orthopaedic literature, a team of surgeons, bioethicists, and residents created an ethical framework for partnerships using the principles of nonmaleficence, justice, beneficence, autonomy, and solidarity.¹⁷ While this effort provides guidance on ways to approach partnership development, it does not provide an objective mechanism for evaluating existing partnerships.

Communication was a common theme identified by experts throughout this study. It has been recognized as a key element for effective personal, social, and professional partnerships and relationships,^18–20 with the lack of addressing cultural-communication differences, miscommunication, and poor communication as problematic.^21,22 Disconcordant priorities or selfish intentions can lead to a dysfunctional partnership,²³ particularly between low- and high-resource institutions.^24,25 Pingray et al²⁶ highlighted that investigators from LMICs are less often listed as first and last authors. Although speculative, in this study, expert consensus on the inclusion of LMIC representation in publications may have stemmed from this concern. These results suggest that experts consider authorship as valuable in equitable partnerships, especially when used as a tangible representation of one's work or leadership in academia.^27,28

Another critical element identified by the expert group was sustainability and safety, demonstrating the importance for lasting and meaningful relationships in successful partnerships. The results of this work underscore relationship building over other factors, such as attending conferences or receiving awards. The sustainability and safety theme included funding stability in partnerships, infrastructures (eg, housing to support visiting scholars), quality improvement initiatives, and autonomy. In addition, patient-centered quality metrics, such as improvements in patient care, and appropriate clinical follow-up were considered highly valued elements within this domain.

Building sustainable international partnerships has the potential to improve musculoskeletal care for patients, particularly those in low-resource settings. Although there are limited studies examining global partnerships in orthopaedics, studies show that collaboration between low- and high-income partners can build research capacity and augment research quality.^4,29 Reported methods for building local capacity include training modules, group educational workshops, and bioskill laboratories for nurses and hospital staff.³⁰ Other partnerships focus on resident education to build local workforce and reduce the reliance on foreign surgical teams.²⁹ Similarly, institutional international partnerships have been shown to catalyze the development of unique clinical solutions based on using locally available resources.³¹ The results of this study are consistent with existing literature demonstrating the desirability to invest in the development of front-line professionals, particularly in low-resource settings.

A limitation of this study includes potential selection bias. While the response rates for Rounds 2 and 3 were high, only approximately half of those initially invited experts from the North American-based COACT network chose to participate. Furthermore, of those North American experts, only a limited number identified an international partner to also participate. Further bias may have resulted from the pool of experts who were largely male and based in urban academic centers. Although this pool included participants from LMICs, it would have been ideal to have a more equal representation between surgeons from low- and high-income settings.

In summary, the understanding of key characteristics of successful international academic partnerships in orthopaedic surgery can lead to more sustainable, mutually beneficial collaborations. The RATES Criteria identified in this study can provide the foundation for developing new partnerships and assessing existing ones. Future work can use these results to further develop methodologies to evaluate collaborations more objectively and critically between low- and high-resource countries.

APPENDIX 1. COACT Delphi Study Group

The COACT Delphi Study Group consists of Dino Aguilar, MD (dino.aguilar@clinicaortopedia.com); Christopher Born, MD (christopher_born@brown.edu); R. Richard Coughlin, MD, MSc (richard.coughlin@ucsf.edu); John R. Dawson, MD (john.dawson@bcm.edu); Andrew R. Evans, MD, FACS, FAAOS (andrew_evans@brown.edu); James Ficke, MD (jficke1@jhmi.edu); Richard A. Gosselin, MD (froggydoc@gmail.com); Billy T. Haonga, MD (bhaonga@gmail.com); Roman Hayda, MD (roman_hayda@brown.edu); Thomas F. Higgins, MD (thomas.higgins@hsc.utah.edu); Herman Johal, MD, MPH (hermanjohal@gmail.com); James Kellam, MD (james.f.kellam@uth.tmc.edu); Sariah Khormaee, MD, PhD (sariah.khormaee@gmail.com); Dominic Konadu-Yeboah, MD, MPH (domiyk@yahoo.com); Arjun Lamichhane, MBBS, MS (drajun@gmail.com); Cassandra A. Lee, MD (casslee@ucdavis.edu); Ross Leighton, MD (leightonr2@gmail.com); Michael A. MacKechnie, MD, CM, FRCSC (michael.mackechnie@gmail.com); Melvin C. Makhni, MD, MBA (mmakhni@gmail.com); Samir Mehta, MD (samir.mehta@pennmedicine.upenn.edu); Barry N. Messinger, MD (gbmess@comcast.net); Chinenye O. Nwachuku, MD (chinenye.nwachuku@gmail.com); Luis G. Padilla, MD (lupadilla@gmail.com); Andrew N. Pollak, MD (apollak@som.umaryland.edu); Saqib Rehman, MD (saqib.rehman@tuhs.temple.edu); Edward K. Rodriguez, MD, PhD (ekrodrig@bidmc.harvard.edu); Coleen S. Sabatini, MD, MPH (coleen.sabatini@ucsf.edu); Sanjeev Sabharwal, MD, MPH (sanjeev.sabharwal@ucsf.edu); Ashoke Sathy, MD (ashoke.sathy@utsouthwestern.edu); Verena M. Schreiber, MD (verena.schreiber@nicklaushealth.org); Marc Swiontkowski, MD (swion001@umn.edu); Nirmal C. Tejwani, MD (nirmal.tejwani@nyulangone.org); Todd Ulmer, MD (ulmertw@hotmail.com); Arvind G. von Keudell, MD, MPH (avonkeudell@bwh.harvard.edu); Michael J. Weaver, MD (mjweaver@bwh.harvard.edu).