A Framework and Blueprint for Building Capacity in Global Orthopaedic Surgical Outreach

Abstract

Background:

Nongovernmental organizations (NGOs) from high-income countries provide surgical outreach for patients in low and middle-income countries (LMICs); however, these efforts lack a coordinated measurement of their ability to build capacity. While the World Health Organization and others recommend outreach trips that aim to build the capacity of the local health-care system, no guidance exists on how to accomplish this. The objective of this paper is to establish a framework and a blueprint to guide the operations of NGOs that provide outreach to build orthopaedic surgical capacity in LMICs.

Methods:

We conducted a qualitative analysis of semistructured interviews with 16 orthopaedic surgeons and administrators located in 7 countries (6 LMICs) on the necessary domains for capacity-building; the analysis was guided by a literature review of capacity-building frameworks. We subsequently conducted a modified nominal group technique with a consortium of 10 U.S.-based surgeons with expertise in global surgical outreach, which was member-checked with 8 new stakeholders from 4 LMICs.

Results:

A framework with 7 domains for capacity-building in global surgical outreach was identified. The domains included professional development, finance, partnerships, governance, community impact, culture, and coordination. These domains were tiered in a hierarchical system to stratify the level of capacity for each domain. A blueprint was developed to guide the operations of an organization seeking to build capacity.

Conclusions:

The developed framework identified 7 domains to address when building capacity during global orthopaedic surgical outreach. The framework and its tiered system can be used to assess capacity and guide capacity-building efforts in LMICs. The developed blueprint can inform the operations of NGOs toward activities that focus on building capacity in order to ensure a measured and sustained impact.

Introduction:

The burden of surgical disease in low and middle-income countries (LMICs) is growing^1,2, with several studies highlighting the increase of many surgically treatable conditions^2–4. Approximately 5 billion people worldwide do not have access to safe and affordable surgical and anesthetic care, and 143 million additional surgical procedures are needed annually to save lives and prevent disability in LMICs^1,5–7. Because trauma, particularly from traffic accidents, kills more people than HIV/AIDS (human immunodeficiency virus/acquired immune deficiency syndrome), malaria, and tuberculosis combined, the burden of musculoskeletal trauma is great^6–10. While global surgery has historically been a low priority, a growing body of literature has highlighted its importance and cost-effectiveness^1,2.

More than 400 organizations operating in all 139 LMICs have been deployed to help address this burden¹¹. The organizational models that exist have great variability: “vertical” models provide care in an episodic manner focusing primarily on surgical care, while “horizontal” models focus on longer-term investments to strengthen local capacity. “Diagonal” models create a synergy between the immediate advantages of vertical models and the longer-term benefits of horizontal models. Empirically, many surgical outreach organizations employ a vertical model that does not prioritize long-term outcomes, accountability, or capacity-building^12–14. As such, there is rising concern regarding the true impact of these outreach trips on the LMIC community. In a systematic review of short-term reconstructive surgical missions, Hendriks et al. identified 41 studies of >48,000 patients and noted that only 10 studies reported follow-up of >6 months¹⁵. The studies reporting follow-up of >6 months noted a complication rate of 22.3% as compared with a complication rate of 1.2% in studies not reporting a time frame for follow-up. In orthopaedic surgery, a systematic review evaluating follow-up practices after short-term outreach trips identified only 18 studies, with a mean follow-up of 58% (weighted to adjust for patient numbers)¹⁶. Coordinated efforts to collect long-term outcomes and partnerships between surgeons in high-income countries (HICs) and LMICs have the potential to build surgical capacity, improve understanding of the impact of care, and, thus, improve surgical quality.

Capacity-building is an approach to health-care development that builds independence through infrastructure development, sustainability, and enhanced problem-solving while taking context into account. Capacity-building is recommended by multiple organizations (e.g., the World Health Organization [WHO]) and is often a cited goal for outreach organizations^17–20. While there is guidance for internal stakeholders in an LMIC regarding the process for increasing its medical community’s capacity to improve population health and provide surgical care, to our knowledge, a capacity-building framework to guide HIC nongovernmental organizations (NGOs) that deliver orthopaedic surgical care in an LMIC on designing their operations and measuring their effects does not exist. The purpose of this work was to identify what domains should be addressed for capacity-building in global orthopaedic surgical outreach in LMICs and provide the necessary components of a blueprint to guide the operations of NGOs seeking to build capacity.

Methodology:

Literature Review:

Two authors (J.M.W. and L.M.S.) conducted a literature review to identify frameworks for capacity-building relevant to surgical care and LMICs. They identified and reviewed 3 articles for common themes describing recommended frameworks to achieve capacity-building^21–23.

Semi-structured Interviews and Thematic Analysis:

A panel of orthopaedic surgeons, trainees, and administrators from an LMIC (local) and NGOs (international) was created. We included 16 participants from 7 countries (6 LMICs) to ensure saturation of themes and broad stakeholder input (Table I). Notably, the experience of many of these participants encompasses several countries (e.g., one director of an NGO who is based in the U.S. has experience in >15 countries). This panel was a convenience sample of stakeholders participating in outreach with NGOs. We attempted to achieve a broad perspective by including various stakeholders with varying experience in multiple settings and locations. Interpreters were used when necessary. Each member of the panel completed a semi-structured interview that had been designed to understand how outreach NGOs can build capacity. A semi-structured interview guide was created based on the themes identified during the literature review (see Appendix A). Interview transcripts were coded and analyzed using thematic analysis. Data were analyzed in real time to inform revisions to the interview guide, identify missing data, and assess saturation. All interviews were conducted by 1 author (J.M.W.) via video conference or telephone from March to May 2021. Interviews were recorded and transcribed verbatim.

TABLE I -.

Interviewed Stakeholders^*

Domain	Number	Notes	Countries Represented†
Surgeons	21	Local surgeons (11)‡, NGO surgeons (10)	Vietnam§#, Tanzania§, Costa Rica#, Argentina§, Haiti§, Ethiopia§, Zimbabwe#, Cambodia§, United States§#
Administrators	3	Liaisons/interpreters, NGO administrators	Vietnam§#, United States§#

^*Represents combined initial semistructured interviews for thematic analysis and subsequent interviews for member-checking.

^†Nine countries are represented here as participants’ primary location; however, many surgeons, trainees, and administrators practice in and thus represent a great number of additional countries.

^‡Four of these surgeons were trainees at the time of the interview.

^§Countries represented by initial semistructured interviews.

^#Countries represented by subsequent interviews for member-checking.

We used an iterative process to develop a codebook with NVivo software (version 12; QSR International). Two reviewers (J.M.W. and M.C.) with training and experience in qualitative methodology developed a preliminary codebook with deductive codes derived from previously identified themes of capacity-building. Standard qualitative methodology was followed (codes defined as labels or tags are assigned to phrases that assign meaning, and a codebook is a set of codes, definitions, and examples utilized to organize and guide analysis)^24,25. The reviewers independently coded transcripts and met to resolve coding discrepancies, added inductive subcodes as new themes emerged, and iteratively revised the working codebook. This process continued until consensus was reached on code definitions and no new subcodes were added. The Cohen kappa coefficient was calculated to assess intercoder reliability for the final codebook. The authors independently applied final codes to all transcripts. We performed a thematic analysis of the coded data to identify relevant domains²⁶. The resulting themes were used to inform the nominal group technique (NGT) and develop consensus on the domain.

Nominal Group Technique:

A technical expert panel of 10 U.S.-based surgeons with experience in surgical outreach and interest in promoting the delivery of safe and high-quality care in LMICs (mean of >16 years of outreach experience, having completed >3,000 surgeries in 24 countries) was convened for a day-long face-to-face meeting. These surgeons are members of Global QUEST (Global Quality in Upper Extremity Surgery and Training), a consortium of orthopaedic and plastic surgeons that treats upper and lower-extremity injuries^27,28. We followed the modified NGT to develop a consensus on capacity-building domains and an operational blueprint^29–31. The NGT is a frequently utilized method to establish consensus that provides a structured process for obtaining information from experts in a group setting in which ideas can be generated, discussed, clarified, and prioritized. Figure 1 details the modified NGT methodology that we used. The group was tasked with the above purpose and was provided background information (e.g., articles identified in the literature review and the results of thematic analysis). Ideas were generated in an independent manner; at the meeting, a round robin session was conducted to elicit more ideas, and a discussion was conducted to ensure clarification. During this discussion, ideas were grouped into domains based on similarity (e.g., surgical skills and quality improvement were grouped into professional development). A preliminary vote was performed on each theme to identify its importance. Domains were identified as important if >80% of members voted “important” as opposed to “not important.” A second discussion occurred to further elaborate on each domain, with the purpose of segmenting the necessary activities under each domain to create a hierarchical framework, as conducted previously. An operational blueprint, informed by the WHO²¹, was created to guide NGOs seeking to promote capacity-building based on the domains of capacity-building and the outreach experiences of the stakeholders. The framework and the blueprint were validated by member-checking interviews with 8 new stakeholders from 4 LMICs³², who were selected as a convenience sample to gain a broad perspective. Interpreters were used when necessary.

Figure 1:

Modified nominal group technique methodology

Results:

Seven domains were identified, all of which were voted to be important for capacity-building. The final codebook is shown in Appendix B. The Cohen kappa coefficient was 0.77. The domains were stratified into tiers of increasing independence (Fig. 2). No new domains emerged after member-checking. Definitions, examples, and representative quotes for each domain are shown in Table II. All representative quotes are included in Appendix C.

Figure 2:

Organizational framework developed to guide the operations of NGOs providing surgical care in order to build surgical capacity in LMICs.

TABLE II -.

Definitions, Examples, and Representative Quotes for Each Domain^*

Domain	Definition	Representative Quote(s)	Example
Partnership	Long-term connections between a surgical outreach organization, local health-care system/community, and governments	“I focus more on our work in Africa and so a lot of that has happened through creating a partnership with the College of Surgeons of Eastern Southern and Central Africa (COSESCA) who is the leading body of certifying and training surgeons in Africa.” “I just think with institutions and having that cross learning and collaboration and you’re always going to have residents and students who are interested in traveling and global health, and there’s just so much mutual benefit from the institutions learning from each other as well.”	Local surgeons participate in annual meetings of professional societies (e.g., speaking at the Annual Meeting of the American Academy of Orthopaedic Surgeons)
Professional development	The advancement of the clinical, educational, and research activities of the local health-care system/community	“I think a component of the educational program should be based on what they need, and we don’t go to a country unless they’ve asked us to come and then we ask them what they need and that’s what we target our education and the program.” “Why research? Once you start doing research and you learn how to do research and the methodology, you can start actually doing research studies that impact your people, your demographic. And then that really starts turning local hospitals in academic centers.”	Local hospitals conduct their own QI programs (e.g., residents conducting QI projects on decreasing time to antibiotics for open fractures)
Governance	The structures and processes by which a surgical outreach organization and local health-care system/community are directed	“Each country has a big role in their own capacity-building. But I think, also at that level it’s great having the ministries of health committing to that capacity-building, but again, a lot of times they don’t have the implementation capability or funding to make it happen, and that’s where I do think nonprofits can fill the gaps, especially in the beginning stages for a lot of countries.”	Policy sector officials and policymakers are engaged during trips and throughout the partnership (e.g., ministries of health help guide activities and programming; accountability steps and guidelines are established)
Community impact	The effect of activities on the local community	“…but sometimes we can be doing more harm by taking over the operating rooms and pushing out the local surgeons and if there’s any patient complications, and then the surgeon’s gone, there’s lots and lots of evidence on why that can be a problematic approach.”	Locally led follow-up and outcomes data collection on measures important to the community (e.g., local hospital/clinic collecting HOOS JR or KOOS JR at6 months postoperatively, focus on upstream prevention efforts)
Finance	The capital required to fund activities	“…they don’t have a steady flow of implants. Everything they got were basically donations of multiple implants and that’s what actually dictated a lot of their ability to take care of patients.” “…and then can we create a partnership somehow that would allow us to get implants that steadily live there, as opposed to looking for donations every month.”	Locally driven resource allocation (e.g., community- guided supply chain efforts to ensure that appropriate implants are present)
Culture	The shared beliefs of the surgical outreach organization and the local health-care system/community	“…the inappropriateness or the error in coming in and applying your values to assessing a situation or a modelor even how things are done, so, I do think that understanding of what they need and what they want, is going to guarantee success.”	Local care-team members, ministries of health-led pre- and post- trip cultural education programs (e.g., educational programming on local community culture and language prior to trip, understanding the health-care context and other key providers)
Coordination	The communication and strategic planning for the execution of activities	“It is incredibly helpful to have an understanding before you go of what their resources are, where they’re at, what their goals are, and think about that in a way that you can have a plan.”	A locally driven needs assessment tool is completed and reviewed to inform trip activities (e.g., needs assessment tools help to guide educational programming, implant, and case selection, etc.)

^*QI = quality improvement, HOOS JR = Hip disability and Osteoarthritis Outcome Score, Joint Replacement, and KOOS JR = Knee injury and Osteoarthritis Outcome Score, Joint Replacement.

Capacity-Building Domains:

Partnerships were recognized in all of the articles and were similarly discussed in a majority of the interviews. The subcodes included strengthening of the existing partnerships as well as the development of new partnerships, with the recognition that partnerships can occur between many stakeholders, including the local providers and NGOs as well as the local providers and other stakeholders (e.g., supply chain networks, funders).

Professional development was discussed in all of the articles. The key subcodes included infrastructure, education and training programs, and research and quality improvement. Frequently highlighted aspects of such subcodes (e.g., needs-based, measurable) were included in the framework and the blueprint. In practice, while early professional development activities may include the assessment and prioritization of NGO needs, eventually, local providers and NGOs can collaborate and/or communities can be independent in curriculum development and certification processes and begin asking questions to inform improvement efforts and research.

Governance was discussed in all of the articles in a variety of ways (e.g., policies and organizational structures). These themes ran through the qualitative interviews, with subcodes that included policies and/or procedures and management systems. Interviewees highlighted that governance, which occurs on multiple levels, may be different from what NGO members are used to. Early capacity-building efforts require the evaluation of rules, regulations, and policies as well as an adaptation of processes to fit the needs of the local community.

Community impact was highlighted in the literature as alignment with local priorities and accountability and/or measurement. The majority of interviewees reiterated these notions and also emphasized the importance of minimizing negative external factors. Examples include measuring long-term impacts, measuring the impact of efforts on the community (not just the patient), and avoiding taking cases away from local surgeons.

Financing was highlighted in all of the articles. Themes included the securing of financial support, resource allocation, the coordination of funds, and the recognition of health insurance systems. The subcodes included funding sources, local payment structure, and internal and external costs. Many interviewees highlighted the importance of securing funding to support longitudinal involvement in capacity-building efforts.

The culture domain permeated throughout the articles and interviews. The subcodes included accountability and customs and/or beliefs. Because this domain was not mutually exclusive of the other domains, it runs throughout the framework and the blueprint. Interviews and discussion during the NGT highlighted cultural awareness and humility, and continually iterated processes and goals tailored to the local context.

The coordination domain was similarly present in all articles and interviews. Qualitative analysis included subcodes related to strategic planning efforts (i.e., stakeholders, timing). Bidirectional communication (pre- and post-trip) was emphasized.

Operational Blueprint:

The operational blueprint (Fig. 3) was developed to guide NGOs with actionable steps toward building capacity at a specific site. Of note, this detailed process begins after appropriate site selection, community selection, and/or partnership (which we believe is a separate topic outside the scope of this paper). For example, the current political stability of a community prior to a partnership should be evaluated before capacity-building efforts begin. A change in political stability between trips may be assessed and recognized with repeat application of the framework.

Figure 3:

Operational blueprint developed to guide the operations of NGOs providing surgical care in order to build surgical capacity in LMICs.

Summary:

As the number of surgical outreach trips grows, guidance and measurement systems for capacity-building are needed^33–35. Orthopaedic outreach programs that build capacity for the local providers and community yield several advantages and have been advocated for by governing bodies^33,35,36. Several orthopaedic organizations have begun capacity-building initiatives (e.g., Health Volunteers Overseas and Institute for Global Orthopaedics and Traumatology)^10,27,33,37; however, there is little guidance on exactly how organizations can achieve capacity-building goals and what proxy measures can be used along the way. The results of this study serve to guide NGOs seeking to adapt to a model that builds capacity on surgical outreach trips.

The identified and included domains can serve as a foundation toward capacity-building for the operations of an NGO. The framework and the domains can be used to inform outreach in a number of ways. An NGO can use the framework and the blueprint to measure and benchmark its current practices at a specific LMIC site. Second, the level of capacity-building relative to current operations can be changed to meet the needs of a site. For example, while local providers in a specific hospital may demonstrate an excellent understanding of the topics that have been taught, they may desire certificates of completion. Third, improvements in capacity over time can be assessed to ensure that operations are achieving their goals and to inform ongoing changes to the operations as well as resource allocation. A radar chart is demonstrated in Appendix Figure 1. As an example, if an NGO is seeking to enhance the professional development that it provides on outreach trips, it may realize that the current educational curriculum is not learner- or needs-driven but is based on the convenience or knowledge of the NGO itself. While that knowledge may serve to improve the education of the local providers, a greater impact may be realized if the NGO is able to direct the educational curriculum to the needs of the learner or incorporate knowledge testing to evaluate the effectiveness of its efforts.

Capacity-building initiatives in global orthopaedic surgical care are often measured by clinical outcomes (e.g., patient mortality) and fall short of measuring the nonclinical aspects of capacity-building^38,39. While we cannot presume a causal effect of capacity-building on reducing morbidity and mortality, it is possible that benchmarking and evaluating change in the capacity-building tiers can improve the health of the community. As such, beyond guiding NGO operations, the developed framework can inform the creation of quality measures that policymakers and organizations can use to evaluate increases in capacity and downstream impact of changes.

Given the great burden of musculoskeletal conditions and the increased interest in capacity-building in orthopaedic surgery^{10,33,34,40,41}, this framework was developed with orthopaedic surgery guidance and application. While the examples and applications of capacity-building will be context-specific (different for each site and/or community), Table II provides examples that can be used. While implementation of this framework and blueprint will vary by organization, there are some potential next steps that leaders of NGOs can take. First, a process measure to assess utilization of the framework and the blueprint could be created to promote its use. Second, transparent reporting to a centralized organization (e.g., the American Academy of Orthopaedic Surgeons or the American Orthopaedic Association) could be used to share knowledge on capacity-building. Third, capacity-building progress with this framework could be used for impact assessment and to support NGO fundraising efforts, while also informing resource allocation for NGOs. Additional work is underway to make these tools more actionable for specific partnerships. Ultimately, we hope that this work can be expanded and iterated for other subspecialties that provide outreach.

This work should be viewed in the context of its limitations. While the NGT has been demonstrated as valid, it has not yet been utilized in real time. Although there is no current framework on which to base activities, our results provide the foundation for implementation and testing. Future work includes making this framework more usable such that capacity-building can be measured across time and NGO. Therefore, measures aside from clinical outcomes (e.g., morbidity) can be evaluated to understand the impact of outreach on the various domains of capacity at a site. Additionally, we did not conduct a formal PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses)-compliant systematic review. Therefore, bias may have been introduced; however, expanding the review to “best practices” rather than just “frameworks” did not result in more novel domains, nor did stakeholder discussion. Similarly, given our stakeholders and experts, there may have been bias. This was mitigated by the use of stakeholders from several countries and experts who were able to draw on experiences from several countries. The framework and the blueprint were subsequently member-checked with 8 new stakeholders from 4 LMICs to ensure external validity. We recognize that while capacity-building is not the primary goal of every organization or trip (e.g., disaster work), the principles of capacity-building (e.g., long-term impact on the community) may still be applicable. Also, our results should be contextualized to each site. Similarly, accountability and expectations related to the trip and the provided care should be coordinated and monitored. Tracking of data related to patients and capacity-building as well as trip feedback can help ensure accountability and inform future outreach efforts. Lastly, these tools are not meant to be static, and as further investigations into surgical capacity-building and validation studies regarding these tools are performed, feedback can be integrated to make iterative improvements in these tools.

This work, which was motivated by the need for clearer guidance on building capacity in global surgical outreach, provides an organizational framework and an operational blueprint to guide the efforts of NGOs. Not only can the framework be utilized to benchmark capacity-building domains and assess improvements over time, its components can serve as proxy measures to better understand the impact of capacity-building on clinical outcomes and other health indicators. The blueprint serves to inform changes to the operations of NGOs and the workflows of teams to enhance capacity-building in order to ensure a measured and sustained impact. As governing bodies continue to advocate for capacity-building and NGOs seek to enhance the accountability and sustainability of their efforts, the resulting framework and blueprint can also help measure and improve the local and community impact of outreach.