Design and Implementation of a Distant-Learning Clinical Research Mentorship Program: The Accra-Toronto Collaboration

Abstract

PURPOSE

For many oncology training programs in low- and middle-income countries, dedicated time for research education and mentorship of trainees is limited. Here, we report a 1-year–long collaboration between a cancer center in Canada and one in Ghana with the aim of imparting clinical research skills and mentoring the research of radiation oncology residents.

METHODS

On the basis of a needs assessment conducted in Ghana, we designed a curriculum consisting of 13 weekly seminars delivered via videoconference, followed by a 1-year–long mentorship program to support research projects. The primary outcome was the feasibility of the program from seminars to manuscript preparation. We used multiple secondary outcomes to capture the learning experience with study-specific questionnaires. We evaluated critical thinking ability using the Berlin questionnaire. Funding was made available for research and travel to international conferences.

RESULTS

Five Ghanaian trainees submitted research proposals. Nine Canadian faculty members delivered the seminars and two served as methodology mentors, and two Ghanaian faculty acted as local supervisors. Feedback questionnaires from all participants showed that they agreed strongly that they would recommend the sessions to another resident (75%), that the objectives were clear (71%), and that the topics were useful for their training (73%). At the end of the program, two Ghanaian trainees finalized their manuscripts and one was published.

CONCLUSION

Here, we report on the implementation of a mentorship program focused on research methods and evidence-based medicine in sub-Saharan Africa. The program was successful in the drafting and publication of abstracts and manuscripts by local trainees.

INTRODUCTION

In 2013, an International Atomic Energy Agency (IAEA) report described only 23 of 52 African countries with access to radiotherapy.¹ This is slowly improving² as governments and policymakers respond to the growing crisis through the implementation of long-term cancer strategies and capital investment in radiotherapy facilities and human resources.³ Training goes beyond the fundamentals of radiation medicine and the safe use of equipment. It must be complimented by critical appraisal and research skills, as well as a dedication to lifelong learning, to achieve the best possible outcomes for our patients.⁴

CONTEXT

Key Objective

Dedicated time for clinical research education and mentorship is challenging to find, especially in training programs in developing countries. We present a collaboration between the Princess Margaret Cancer Center in Toronto, ON, Canada, and the National Center for Radiotherapy in Accra, Ghana, focused on imparting clinical research skills and mentoring a group of trainees.

Knowledge Generated

We describe our needs assessment, program design, evaluation, and early impact, and find it to be a successful and worthwhile model to pursue.

Relevance

Our paper provides a blueprint for international partnerships to support clinical research for oncology trainees through distant-learning and longitudinal mentorship principles.

The National Center for Radiotherapy and Nuclear Medicine (NCR) in Accra, Ghana, and the Princess Margaret Cancer Centre in Toronto, ON, Canada, entered into a partnership to support training needs after bilateral academic exchanges. We hypothesized that a distance-learning mentorship program can be effective in supporting the development of clinical research skills in radiation oncology trainees. Ultimately, such collaborations are meant to improve clinical care and patient outcomes. In this report, we present the experience of the Accra–Toronto Clinical Research Mentorship Program (CRMP). We describe our initial needs assessment followed by the program’s design, evaluation, and early impact.

METHODS

Needs Assessment

We held a focus group in 2014 at the NCR with Ghanaian residents and fellows. The focus group was facilitated by one of the authors (H.V.), a resident from Toronto undertaking an elective in Accra at the time. The group was guided by open-ended questions that explored the level of interest in a collaborative mentorship program, potential research topics, and format of the program, including scheduling, teleconference capability, and class structure.

Program Design

In designing our program, we were guided by our needs assessment in Accra and iterative input from trainees and faculty in both Accra and Toronto, including faculty with expertise in education and research methodology.

The final design took the shape of a 1-year–long program summarized in Figure 1. The program has four key features.

FIG 1.

Schema for Ghana –Toronto clinical research mentorship program.

Selecting a research question.

To ensure learning through practice and application, all trainees who were enrolled in the radiation oncology training program (N = 5) were invited to submit a short research proposal (2 to 3 pages). The trainee was asked to identify one of the two Ghana consultants as their supervisor for the proposal. This was intended to ensure that the proposal had the support of the program for feasibility and scientific interest. Using a standardized rubric, a panel of assessors—2 faculty and 1 trainee—evaluated proposals on the basis of the project’s feasibility, prior knowledge of clinical research methods as demonstrated in the proposal, and anticipated impact on the clinical or training environment (Figure 2).

FIG 2.

Research proposal score sheet.

Defining a 1-year supervisor, student, and mentor relationship.

For this pilot project, to focus our efforts and faculty commitment, only the top 2 trainee proposals were selected for a mentor assignment, although all trainees were invited to participate in the rest of the program. The supervisor based in Accra provided clinical expertise on the research topic, facilitated problem solving from a systems perspective, and played a central role in ensuring the success of the international collaboration. The mentor based in Toronto was selected for their clinical epidemiology expertise and interest in global health. The trainee, supervisor, and mentor worked together through the year-long program with the goal of translating the research question to a viable research protocol, collecting data, interpreting results, and preparing an abstract and manuscript for knowledge dissemination. We deliberately used this terminology to reinforce the collaborative relationship. Trainees without an assigned mentor at Princess Margaret Cancer Centre were invited to participate in the seminar series and expected to work with their local supervisors to complete their research projects as part of their training program.

Clinical research methods teaching on the basis of a flipped classroom design.

All trainees were invited to participate in a seminar series at the start of the program. This consisted in 13 1.5-hour seminars delivered weekly through a Web-based video connection—to maximize interactivity—in the first 3 months of the program. These seminars used a flipped classroom design in which learners were provided with reading material, learning objectives, and practice assignments before each session. The goal was to allocate 50% of the seminar time for the facilitator to review key concepts, whereas the second half was dedicated to group discussion and applying the concepts to the learner’s research project. The curriculum included topics from framing a research question, clinical trial design, research ethics, biostatistics, and abstract and manuscript preparation.

Funding support.

A modest research grant was allocated to each learner ($2,000 Canadian). These funds were designed to cover conference fees, transportation costs, and lodging costs for trainees. All trainees who participated in the seminars (N = 5) were eligible to access these funds upon successful acceptance of their abstract by an international conference—for example, the American Society for Radiation Oncology or the Canadian Association of Radiation Oncology. Only 80% of the funds were accessible to support travel, while the remaining 20% was accessible upon preparation of a manuscript. The National Center for Radiotherapy–Princess Margaret CRMP was supported by an unrestricted grant from the University Health Network Global Capacity Fund to support the research and travel stipend for students.

Outcomes of Interest

The primary outcome of the project was feasibility, as measured by the number of learners who completed the seminar component and the rest of the mentorship program, up until manuscript preparation. We used multiple secondary outcomes to capture the learning experience aligned with the Kirkpatrick levels of evaluation—that is, reaction, learning, behavior change, and results.⁵ A 13-item study-specific questionnaire using a 5-point Likert scale (1 = strongly disagree; 5 = strongly agree) was designed to evaluate each seminar topic and the effectiveness of faculty facilitators (Figure 3). Qualitative feedback was solicited at the end of the seminar series through a videoconference and e-mail exchange from learners and the faculty in Ghana and Toronto. We evaluated the proficiency of learners with critical appraisal skills using the Berlin questionnaire, a validated instrument administered before and after the completion of the program.⁶ We reported the completion of research projects, abstract submissions, acceptance at international conferences, manuscript preparation, and publications. Finally, we explored interest in this kind of mentorship collaboration with other oncology training programs in Africa through personal introductions by our collaborator (V.V. & J.Y.).

FIG 3.

Study-specific questionnaire.

RESULTS

The program was designed, funding secured, and the call for proposals first launched in 2015. Five Ghanaian trainees responded to the call for research proposals. Three faculty members in Toronto (J.R., Z.K., and R.K.S.W.) with Masters-level clinical epidemiology training served as research mentors. Both oncologists at the NCR (V.V. and J.Y.) served as supervisors and supported the learners. As described in Methods, two learners were assigned a mentor in Toronto, although all trainees were invited to attend the seminars, and were eligible to receive travel funding upon acceptance of their results at the end of the program. Nine Canadian faculty members delivered the initial weekly seminars.

The research proposals submitted by the 5 trainees are summarized in Table 1. Feedback that was obtained using the study-specific questionnaire (N = 5) showed participants agreed strongly that they would recommend the sessions to another resident (75%), the objectives were clear (71%), the topics were useful for their training (73%), and the discussions were helpful to understand the topics (70%). Audio-video connection functionality scored the lowest (32% strongly agreed; Figs 4A and 4B). Qualitative feedback from faculty highlighted areas that worked well and those that did not. From the trainees, the strength of the teamwork that went into supporting the teaching was highlighted. Topics selected for teaching were felt to be highly appropriate. The relevance of “Design of quality assurance projects” was less obvious. Practical biostatistics sessions were particularly well received. The ability to apply the concepts to protocol design was useful to reinforce learning. Participants generally acknowledged that videoconferencing was effective in providing interactivity, and the ability of participants to see each other was important for building relationships. However, technology limitations were noted. Setting up the connection reliably was a consistent issue, and technical support was required for efficiency and to avoid frustration. Preclass preparation and learner participation was heterogeneous and the reasons are multifactorial. Preclass reading material and tasks needed to be received with more lead time—for example, more than 1 week. Cultural differences in learning and interaction must be recognized. In general, it was perceived by the Toronto faculty that the concept of the flipped classroom style of learning and engagement in discussion by trainees was less familiar. Trainees had limited time to take advantage of the preclass reading material. Clinical commitments also made it difficult to commence sessions as scheduled on occasion. It was suggested that the reading material be more selective and made available well in advance.

TABLE 1.

Research Proposals

FIG 4.

Evaluations of (A) seminars and (B) seminar leads (13-item questionnaire using a 5-point scale).

Five trainees, including the 2 trainees who were assigned mentors, completed the baseline Berlin questionnaire, and 4 completed the postseminar questionnaire. Results for 3 trainees showed an average 20% absolute improvement, whereas the score was unchanged for 1 trainee.

Two trainees have seen their research abstracts accepted at international conferences, including the Canadian Association of Radiation Oncology Annual Scientific Meeting and the International Gynecologic Cancer Society meeting. Two original research manuscripts have been finalized and one has been published in a major radiation oncology journal, with a second submitted. The first is a large retrospective review on cervical cancer outcomes at the NCR (no mentor assigned) and the other a review of the management and outcomes of pediatric patients with Wilms tumors in Ghana (mentor assigned). A third project investigating pain scores during brachytherapy for cervical cancer was initially postponed because of equipment delay but is currently underway (mentor assigned). Since the completion of the CRMP in Ghana, the program was offered to the oncology training program in Harare, Zimbabwe, where it is ongoing.

DISCUSSION

Our distant-learning clinical research mentorship program was effective in enabling the completion of 2 clinical research projects—with manuscripts documenting their findings—yielding novel clinical data with the potential to improve patient care. The seminar series was successfully delivered despite some technical limitations using a videoconferencing platform in an environment with limited Internet bandwidth and a 5-hour time difference. Learners successfully demonstrated an improvement in their critical appraisal skills after the seminar series, as evaluated using the Berlin questionnaire. The quality of the collaboration overall was strong and the program well received, garnering interest from other training programs. It is currently in its second iteration in another training program in sub-Saharan Africa.

Lessons learned from our assessment have highlighted specific areas for improvement. These include focusing the amount of preclass reading material, including trainees and local faculty in the initial scoring task to enhance learning, and modifying content in response to the research method need of different groups.

Not all trainees completed their research project. Reasons for incomplete projects were multifactorial. Participation in the research mentorship program was voluntary. In one mentor-assigned project, the research topic had a prospective design. Delays in commencing the clinical program of interest meant the project needed to be deferred. Resources to support residents extracting data, creating and managing databases, performing analyses, securing ethics approvals, and performing additional study specific investigations were limited. These limitations are inherent to the process of conducting research, particularly in resource-constrained settings.

Resources needed to enable research can be considered in terms of infrastructure and human resources. These include faculty time to prepare and teach seminars, as well as to provide supervision (Ghana) and mentorship (Toronto) to selected residents throughout the year; specific content expertise on each of the topics and research methodology; technical support to ensure videoconference connectivity from both sites; and a coordinator function to ensure a smooth experience for both trainee and faculty in coordinating the needs assessment, submitting the proposal, providing evaluation and feedback, designing curriculum, scheduling sessions, distributing reading materials, assignments, evaluation surveys, and collating responses (Table 2). In the spirit of collaboration, all of these resources were provided in kind. Funding was preferentially used to facilitate the trainees’ ability make presentations at conferences and disseminate their findings.

TABLE 2.

Resource Requirements

The NCR was established in 1997 in collaboration with the IAEA, serves as a large catchment area, and is a resource to other West African countries, including Togo, Benin, Côte D’Ivoire, Burkina Faso, Liberia, and Sierra Leone.⁷ Common to many other areas that are undergoing stages of rapid change, dedicated time for research education and mentorship is challenging to find. Relational partnerships between resource-rich and -limited settings can provide a particularly effective solution in this setting.⁸ Recent medical literature has acknowledged the strong need for mentorship in low- and middle-income countries for the advancement of global health.^9,10 The theoretical underpinning of our mentorship program is in direct alignment with Prasad and colleagues’⁹ conceptual framework for mentoring in global health. This framework highlights an ecologic systems model whereby the interaction between the mentor and mentee is securely embedded within several concentric circles, including a mesosystem (eg, interpersonal interactions and alignment of goals and objectives), exosystem (eg, institutional factors, such as policy and resources), macrosystem (eg, national economy and sociopolitical movements), and supramacrosystem (eg, global economy and global politics).⁹ Ongoing consideration within this framework facilitates a holistic approach to mentorship interactions.

When reflecting on enablers of and barriers to our collaboration, building a strong relational partnership is key. One author (J.Y.) spent 1 year as an international fellow supported by the IAEA and the Princess Margaret Cancer Centre. This relationship led to another author (R.K.S.W.) being invited to attend an African Radiation Oncology Group meeting in Ghana to experience firsthand the practice environment and to gain an understanding of the needs of Ghanaian trainees. Another author (H.V.), with a strong career interest in global health, traveled to Ghana for a 1-month elective supported by a Global Oncology Scholarship from the Canadian Association of Radiation Oncology. From a program design perspective, we felt that the joint mentorship and supervisory roles, the reverse classroom design, and a modest amount of funding to enable the dissemination of research findings were critical enablers. Oncology journals were open and encouraging of data reporting from sub-Saharan Africa. We encountered roadblocks, too. Securing visas for Ghanaian trainees to present at international conferences—in Canada and the United States—was challenging for both of our successful trainees. Communications with one of the trainees, who returned to Eritrea, was limited as a result of inconsistent Internet connectivity.

Technology-enabled learning has obvious advantages to narrow geographic distance and scale-up information transfer. The Lancet Commission on Medical Education calls for a transformation of medical education for the 21st century, highlighting the need for global networks and the use of information technology for learning.¹¹ In Africa, distance education using the Internet is growing. For example, the African Virtual University, a project of the World Bank, operates in 19 African countries as an intergovernmental organization.¹² The International Atomic Energy Agency’s Applied Sciences of Oncology provides online courses in oncology and research methods.¹³ Though invaluable, distant learning alone cannot replace the facilitating roles of expert teachers in applying knowledge. A blended approach using distant learning technologies and expert teachers, as used in our 1-year–long mentorship program, set our project apart from traditional e-learning solutions.

Strengthening research capacity has been touted as a powerful and cost-effective way to improve health and development.¹² The paucity of data on operational guidance, monitoring, and evaluation may be one of the reasons why progress is slower than desired. Well-conceptualized programs with a priori deliverables remain the minority, but are growing in number.¹⁴ The ESSENCE Good Practice framework for research capacity strengthening provides useful principles and practical considerations.^15,16 Our program design included many of the principles and deliverables recommended.

Our experience highlights enabling strategies for building a successful relational partnership in education collaboration, which we have operationalized into 8 steps. These include the following: identify potential partners, perform a needs assessment of learners, define the knowledge gap, design and refine the curriculum, select evaluation tools, define short- and long-term deliverables, consider the budget, and repeat the process for each successful group of learners or collaborating partners. Identifying appropriate partners aligns with the ESSENCE principles. The needs assessment of learners is a discrete step well described in educational design⁵ to understand the learning needs and perspective of the target audience, intentionally refining educational offerings to align with different learning styles and expectations. The choice of evaluation tools should be focused on instruments that are validated and aligns with the Kirkpatrick model¹⁷ of program evaluation. Adequate consideration of budgetary requirements to support the goals of learners and partnering institutions is also important. We expect this simple action cycle will enable experience to be gained, evidence to be built, and best practices to be followed, doing so with improved efficiency, sustainability, and growth. This is particularly important for academic institutions designing education collaborations with partners from different settings (Fig 5).

FIG 5.

Education collaboration design.

There are limitations to our project. Our report represents our first experience with a single institution and a modest cohort of 5 learners. Continued sustainability is required to demonstrate real success with our approach. There is no easy way to quantify the long-term impact of our program as incorporating the skills that have been learned into clinical practice is subject to multiple environmental factors. We expect that, through continuation of our project with successive cohorts of learners, a network of alumni from our mentorship program will provide a greater experience as well as opportunities for longer-term follow up. A qualitative study of trainees who have completed the program from Ghana and Zimbabwe is underway. We are optimistic that building bridges and promoting research capacity can ultimately improve the quality of care and meaningful clinical outcomes for patients in low- and middle-income countries.

In conclusion, we report here on the implementation of a mentorship program focused on research methods and evidence-based medicine in sub-Saharan Africa. To our knowledge, this is a unique model that was established in close collaboration with its intended recipients. It served to build bridges between institutions and fostered friendships and cooperation. The program was successful in the drafting and publication of abstracts and manuscripts by local trainees.

AUTHOR CONTRIBUTIONS

Conception and design: Horia Vulpe, Joel Yarney, Zahra Kassam, Rebecca K.S. Wong

Administrative support: Joel Yarney

Collection and assembly of data: Horia Vulpe, Verna Vanderpuyne, Joel Yarney, Jolie Ringash, Rebecca K.S. Wong

Data analysis and interpretation: Horia Vulpe, Verna Vanderpuyne, Sarah Tosoni, Zahra Kassam, Rebecca K.S. Wong

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jgo/site/misc/authors.html.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).

Horia Vulpe

Research Funding: Varian Medical Systems (Inst)

Joel Yarney

Honoraria: Roche, Johnson & Johnson, AstraZeneca

Speakers' Bureau: Johnson & Johnson

Travel, Accommodations, Expenses: Roche

No other potential conflicts of interest were reported.