A new wave of leaders: Early evaluation of the interdisciplinary Foundations of Leadership in Radiation Oncology (FLiRO) program

Highlights

• •170 interdisciplinary radiation oncology (RO) professionals from 36 countries have taken part in the Foundations of Leadership in RO (FLiRO) leadership program between 2018 and 2021.
• •95% of participants indicated that FLiRO learning would be very or extremely useful for application in their workplace environments.
• •Systematic leadership development for RO professionals is central to ensuring the global gap in radiation therapy access and utilization continues to be addressed.

Abstract

Purpose

Effective leadership across all areas of radiation oncology (RO) is vital to fully realise the benefits of radiation therapy in cancer care. We report outcomes of a novel interdisciplinary leadership program designed for RO professionals under a global joint society initiative.

Methods

The Foundations of Leadership in RO (FLiRO) program was designed for aspiring RO leaders. Initially delivered in a blended learning format, it was adapted to fully virtual in 2021. It comprised a webinar tutorial, on-line modules and homework followed by ‘live’ in-person/virtual workshops over an approximately 6-week period. Topics included personal awareness, effective teamwork, quality improvement skills, leading change and conflict management. An immediate post-program online survey was performed using Likert scales to measure self-reported educational value, interaction with others and the likely application of learning to practice. Open comments were invited.

Results

170 participants from 36 countries and 6 continents took part from 2018 to 2021 (99 doctors, 36 physicists, 32 radiation therapists/RTTs and 3 others). 141 (83%) participants responded to the post-program survey. Average weightings for responders’ views on whether pre-determined learning objectives were met ranged from 4.30 to 4.61 on a 5-point scale (1 = ‘not met at all’ and 5 = completely met). For the question addressing potential value of learning for application to their workplace, 124 of 130 (95%) of responders indicated that FLIRO would be ‘very useful’ or ‘extremely useful’.

Conclusion

Initial evaluation of the FLiRO program supports its continuation and expansion with ongoing evolution based on emerging evidence around leadership education and participant feedback.

Introduction

There is increasing recognition of the requirement for health professionals to develop skills to effectively lead improvements and create high-functioning teams in their workplaces [1], [2]. The ultimate result of successful clinician leadership is optimal care of patients and communities, at the same time maintaining the well-being of those that work within the health system [3], [4]. Radiation oncology (RO) is the archetypal discipline in which effective teamwork is vital for ensuring high quality patient care. Radiation therapy is a key component of effective cancer cure and control worldwide, yet our field faces some distinct challenges. Not least of these challenges is the need to keep practitioners continually upskilled in rapidly evolving technological advances and, in many countries, the fact that radiation therapy is an underutilised, poorly accessed and underfunded cancer treatment modality [5], [6], [7], [8]. If the benefits of radiation therapy are to be fully realised worldwide, all RO professionals need to take on responsibility for leadership across all relevant spheres - clinical, research, education, administration and advocacy - at local, national and global levels.

Examples of the growing acknowledgement amongst health professional educators that leadership development must become core learning is the incorporation of the Leader Domain within the 2015 edition of CanMEDS (Canadian Medical Education Directives for Specialists) [9], [10] and the publication of the Leadership and management standards for medical professionals by the NHS Faculty of Medical Leadership and Management in the UK [11]. In turn, specific competencies for the Leader role (or equivalent) have been articulated to support learning within many competency-based training program curricula around the world, [12], [13] including within RO [14], [15]. There have been several reports highlighting the gap in opportunity and education around leadership development during the period of training and beyond, most notably for medical team members i.e., radiation and clinical oncologists [16], [17], [18], [19]. In the latest revisions of the ESTRO Core Curricula for oncologists [20] and physicists [21] learning objectives for the Leader role have been incorporated. However, in common with other non-clinical or technical domains of health professional learning, embedding the structures, tools and opportunities for learners to systematically develop leadership skills is challenging. Not only are training program curricula already crowded, but lack of familiarity and expertise in teaching these capabilities, and lack of recognition by training organisations and/or senior managers may all present barriers to learning [22].

The Foundations of Leadership in Radiation Oncology (FLiRO) blended learning program was designed to start addressing the known gap in leadership education within RO. This article describes the genesis of the program, outlines the program design and implementation, and presents early evaluation outcomes from the first five rounds of the FLiRO program.

Materials and methods

Program development and implementation

In 2017, a collaboration was established between members of the ESTRO School, the Canadian Association of Radiation Oncology (CARO) and the Royal Australian and New Zealand College of Radiologists (RANZCR) Faculty of Radiation Oncology, to design and deliver a foundational course to teach leadership concepts and core leadership skills to RO professionals. The ESTRO School was instrumental in supporting development and implementation and included FLiRO in its annual calendar of educational events from 2018.

The overarching program goal was to improve the confidence and capability of RO professionals in acting as ‘little l’ leaders and/or as they move into formal leadership roles. By breaking down healthcare leadership into its elements and learning skills, strategies and behaviours which underpin effective leadership, aspiring leaders can be empowered to initiate improvements and effect positive change within RO teams and across services in which they work. The major FLiRO Learning Outcomes were that after the program participants would be able to:

• •Examine & reflect on their own behaviours, reactions & interactions with others
• •Describe basic leadership theory & styles as they might apply to healthcare situations
• •Describe the foundations of effective change management
• •Explore strategies to establish & lead effective teams
• •Apply basic quality improvement tools to approach an improvement process

FLiRO was designed as an interdisciplinary program for RO professionals (specifically doctors, radiation therapists/technologists [RTTs] and medical physicists) fostering development of knowledge and skills various elements of effective leadership. The program topics were mapped to align with 20 Leader role curriculum competencies for RO determined by an international modified Delphi consensus process [13] and organised within the domains of: A. Contribute to the improvement of cancer care delivery in teams and wider health systems, B. Engage in stewardship of cancer care resources, and C. Demonstrate elements of leadership in practice.

Potential co-directors representing each of the collaborating societies who had demonstrated academic and practical interest in the area of RO leadership development were approached by the Chair of the ESTRO Education Committee to build the program. There was no formal recruitment process for teaching faculty though it was felt important that the faculty comprise mixed RO (and clinical oncology) professionals of all types who have an interest in education generally, as well as, in many cases, expertise specific to leadership development. The program was aimed at learners towards the end of radiation/clinical oncology training or within the first 5 years of independent practice (for doctors) and for RTs and physicists with a few years of clinical experience. The program was promoted mainly via the relevant RO professional organisations and individual RO craft-group societies in some countries through various media channels, including social media. For faculty member active on social media, messaging was amplified through their personal accounts.

Applicants had to complete a brief statement of interest and motivation for learning, as well as give a commitment to completion of all program elements. A subpart of the teaching faculty assessed applications for suitability with maximum numbers set by the organising society in conjunction with co-directors. Where possible, the participants from non-medical professional groups were prioritised when application numbers exceeded the program limit though there were always many more physician applicants than other groups.

The program was designed as a blended learning resource comprising an online introductory webinar followed around six weeks later by one and a half days of interactive workshops, usually held in conjunction with an international RO meeting. In 2021, the in-person elements were replaced by three half-day workshops delivered virtually due to the COVID-19 pandemic. Between the initial webinar and the ‘live’ sessions, participants completed five short interactive online modules developed to align with FLiRO objectives. In addition, participants had to complete a short quality improvement project planning exercise. The content covered in the blended and fully virtual versions was almost identical and is shown in Fig. 1. Effort was made to maximise interactivity between participants and teachers in the virtual version, in keeping with the high interactivity of the previous in-person workshops. For all interactive elements, the session topics were applied to activities based on hypothetical RO workplace scenarios, to ensure relevance of learning and to replicate the workplace team setting.

Fig. 1.

Program outline and session topics.

Data collection and analysis

Participants were invited to complete optional pre-program and post-program surveys distributed through the Survey Monkey® platform. Responses were anonymous and completion of the survey was regarded as participant consent. The pre-program survey asked mainly simple demographic questions. Immediate post-program surveys asked 6–7 questions collecting participants’ opinions on the degree to which each major learning outcome was met, the relevance and value of learning to their daily practice and, after two versions only (one blended and one virtual), the opportunity to engage and interact with others throughout the program. Surveys were piloted by faculty not directly involved in their design. Demographic and non-leadership-specific questions were taken from the regular ESTRO School evaluation tool used for other courses. Participants were also asked about organisational aspects of the program and the quality of resources provided (results not presented in this report). 5-point Likert scales were used to measure participant ratings with 1 being the poorest rating and 5, the most favourable. Free text comments were invited to give additional feedback, in particular, regarding suggested improvements. Comments have been used to guide minor changes to format and content but formal analysis of the large number open-ended comments has not been conducted.

Responses were collated through the back end of Survey Monkey® as well as being entered into an Excel spreadsheet. Most results are reported as percentages of all participants and/or those responding to a specific question. Likert scale ratings for the extent to which learning outcomes were met were calculated as a mean (and range) across responders for each pre-articulated learning outcome.

Results

Learner demographics

A total of 170 RO professionals took part in the FLiRO program over five versions from 2018 to 2021 (32, 26, 34, 44 and 34 participants respectively). Four versions were in a blended learning format (part on-line learning and part in-person) and one was completely virtual, with all incorporating real-time interactive elements as well as self-directed online learning. Fig. 2 shows the breakdown of the RO professional type for all 170 participants. The majority of participants were radiation (or clinical) oncologists. There was no apparent pattern to the proportion of other RO professional types who applied/were accepted to FLiRO over time.

Fig. 2.

Percentage of participants by professional type (all participants, n = 170).

At the time of enrolling for FLiRO, participants were practicing in 36 countries across six continents. Notably there was only one participant from Africa and one from South America. Table 1 shows the number of participants from each country attending the course.

Table 1.

Countries of practice (or training) and number of participants from each (n = 170).

Country of practice	Number of participants	Country of practice	Number of participants
Australia	40	Poland	1
Austria	2	Portugal	1
Azerbaijan	1	Malta	1
Belarus	2	Montenegro	2
Belgium	2	New Zealand	22
Canada	31	North Macedonia	1
Chile	1	Pakistan	1
China	1	Russian Federation	2
Cyprus	1	Romania	2
Denmark	2	Saudi Arabia	1
Estonia	1	Spain	2
Germany	3	Sweden	2
India	1	Switzerland	10
Iraq	1	The Netherlands	3
Italy	4	Tunisia	1
Ireland (Republic)	2	Turkey	2
Lebanon	1	United Kingdom	9
Lithuania	1	United States of America	10

Program evaluation

All participants responded to the short pre-program survey. The post-survey response rate was 141/170 (83%) overall. As survey questions could be ‘skipped’, for certain survey questions the response rates fell; the lowest response rate for an individual question being 130/170 (76%). All data were used, regardless of whether responders answered all survey questions or not. The number of responses to the relevant question are indicated in each Figure. Fig. 3 shows responders’ averaged ratings around how well the program’s overarching pre-articulated learning outcomes were met. Mean ratings ranged from 4.30 − 4.61 on a 5-point scale (from 1 = not met at all to 5 = completely met).

Fig. 3.

Extent to which pre-articulated program learning outcomes were met.

When asked about the educational value of FLiRO, 134 of 141 (95%) participants answering the question indicated that their learning from the FLiRO program was/would be ‘very useful’ or ‘extremely useful’ in terms of their likely application of learning to their own practice in the workplace (see Fig. 4).

Fig. 4.

Self-reported relevance and value of learning to workplace practice.

Participants’ perspectives on the opportunities for engagement and interaction with other learners and the teaching faculty were examined for one of the blended versions of the program and for the fully virtual version. These results are seen in Fig. 5. Most responders felt that engagement and interactivity were high for both learning formats, with the proportions of those scoring opportunities as ‘excellent’ being comparable between these formats. As only one version of the fully online format had been conducted at the time of this study, a more detailed analysis of the comparisons between the virtual and blended programs was not conducted.

Fig. 5.

Opportunity for engagement and interaction with others (comparing blended versus virtual formats, n = 54).

The large majority of open-ended comments were favourable. Participants consistently enjoyed the learning using the personal ‘Insights’ profiling and the related activities dealing with self-awareness and improving connections with others. With regard to the fully virtual program held in 2021, there were several comments around the convenience of avoiding travel and the potential for improved access for applicants around the world, especially for those from lower income countries. Conversely, the attraction of in-person networking (not possible in 2021) was mentioned by many across all programs. For the 2021 cohort only, the problem of unfriendly time-zones for learning was stated as the most common negative feature by far.

There were very few suggestions for improvement. One participant suggested that a blended and virtual program be run annually. Another person noted that gender equality (requiring more men) and cultural diversity could be improved amongst the teaching faculty.

Discussion

The FLiRO program was designed to address the recognised educational gap in formal and systematic leadership development for RO professionals [16], [17], [18]. The preliminary evaluation outcomes based on the first five iterations of the course (involving 170 participants) as reported in this article suggest that the program is meeting its initial goals in providing the opportunity for foundational learning around elements of effective leadership. In view of the wide international reach, attracting participants working in 36 countries, and overwhelming enthusiasm for the program to be ongoing, it can be concluded that the launch of the FLiRO program has successfully begun addressing an educational gap for RO professionals of all types. Continual evaluation of program outputs and incorporation of newer educational methods and topics relevant to effective leadership will be important in maintaining its currency and purpose as FLiRO grows.

Recent program developments have included broadening of the teaching faculty to increase professional (more RTs and physicists), gender and cultural diversity/equality and incorporating a specific ‘Equality, Diversity and Inclusion (EDI) in Leadership’ session. Enlarging the teaching faculty will also support sustainability of the program and the potential for a ‘train the trainer’ model of expansion is under discussion. In 2022, a collaboration with the Royal College of Radiologists in the UK has allowed another cohort of entirely UK radiation/clinical oncology professionals to access the program.

Future challenges lie not only in trying to measure the longitudinal impacts of FLiRO, but in expanding the reach of the program to a broader audience. In particular, the gains in strengthening RO leadership capacity could be key for regions in which there is a dire need for RO service implementation and/or expansion such as in many low- and middle-income countries where leadership development opportunities may be especially scarce. The tension between maintaining a relatively small interactive ‘hands-on’ approach to learning seemingly suited to the area of leadership education and scaling up the program to upskill as many people as possible over the shortest time frame is ever present. The challenge in reaching professionals who are not confident English speakers, as for other educational resources, needs addressing.

Both within RO [23], [24] and in other health-related disciplines [25] researchers have recognised that self-awareness (an understanding of oneself, also termed ‘emotional intelligence’) is central to effective leadership, as has also been shown to be the case in corporate environments [26], [27]. The theoretical evidence around the value of increasing self-awareness, and hence a better appreciation of interpersonal differences amongst team members, aligns well with the overwhelmingly positive feedback about the use of the Insights Discovery® personalised profiles and related activities at the heart of the FLiRO program design (full data not presented in this article). The focus on ‘self’ in leadership development also corresponds to the CanMEDS and other established health leadership frameworks subdomains [28], but at present there remains limited opportunity for formal instruction around self-awareness as part of health professional training programs.

Despite the recognised importance of leadership development for health professionals, particularly doctors, and the increase of leadership training interventions, there is little in the literature evaluating programs for clinical outcomes. The long-term impact of learning on quality or safety endpoints in patient care, or other measures of workplace improvement would be the ultimate measure of success [29]. In addition, in the field of RO, ready access to high quality radiation therapy for all cancer patients, in keeping with optimal utilisation rates, represents the holy grail of leadership success. Unfortunately, the current report is not able to comment on longitudinal outcomes despite highly positive participant responses around the ability to apply learning to their workplace practice. Attempts thus far to collect longer term data from past participants (e.g., 12 months out from FLiRO) have been disappointing. This challenge is shared by others exploring the impact of leadership education, despite the demonstrated positive correlation between strong clinical leadership and established patient outcome measures including hospital mortality rates [4].

Strengths of FLiRO as reflected in the current report are that the program is well accepted, is applicable for professionals in many countries across the world and is regarded as directly applicable to the RO workplace setting. In part, the authors suggest that the perceived relevance is aided by both the teaching faculty and learner group being comprised of interdisciplinary RO professionals, replicating the real-life work setting. Findings also suggest (in support of leadership theory on this topic) that leadership skills and behaviours can be learned and improved through practice, regardless of individual personality type [30]. The fully virtual FLiRO version held in 2021 also proved that adequate interaction for learning can be successfully achieved, if less popular for many, in the online setting. As two further fully virtual programs will have been completed by the end of 2022, a more detailed comparison of virtual and blended approaches will be more feasible down the track.

Limitations of this study (in part reflecting the program in general) are that FLiRO and the evaluation were only conducted in English, potentially excluding those without a high level of competence in English and affecting generalisability of findings. Similarly, there was a paucity of participants who attended from lower- and middle-income countries for whom the format and/or content of the program may be less applicable. Some adaptations to relate to the local environment and/or cultural differences might be necessary were the program to be tailored to other settings, in addition to overcoming language barriers. Although there was a high response rate to the surveys, responder bias is still a potential confounder of results for all survey studies especially as not all questions were answered by all responders. The post-program survey tool in retrospect was not ideal in other ways. The tool was slightly modified between programs, in part due to the requirements of the hosting societies. These inconsistencies meant that some questions of potential interest were not backed up by data considered complete enough to include for analysis. As mentioned previously, optimally the longer-term benefits of learning on both the individual, the workplace and patient care could be collated as a measure of improved leadership capacity resulting from the program. Unfortunately, the post-survey tool, which was designed to be the basis of a longitudinal program evaluation, was lengthy due to the inclusion of a validated leadership development program evaluation instrument [31]. Not only were response rates low to the subsequent 12-month survey but it is likely the length of the initial survey acted as a deterrent to full completion of questions relating to early outcomes (as reported in this paper).

Clearly, it is much too ambitious to postulate that a stand-alone program conducted over just a few weeks will provide hard measurable outcomes of effective leadership. The authors’ (teachers’) goal is that the knowledge, skills and better recognition of what leadership means for FLiRO learners will translate to their ability to identify and approach leadership opportunities and challenges in the workplace with a higher chance of success. It will be important that ongoing work continues to ensure that the existing curricula for leadership development become standardly integrated into training programs and translated into proper educational opportunities, resources and ongoing mentorship/role-modelling within RO departments and training institutions [22]. Additionally, it is intended that expanding the network of FLiRO ‘alumni’ professionals and teaching faculty in an international setting will provide added value and support for leadership initiatives across all domains within our discipline.

It is anticipated that over time, the resulting new wave of educational and administrative RO leaders will play a central role in ensuring the systematic integration of leadership development as a core element of training for all RO professionals, across all training institutions and cancer services around the world.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.