The Fourth-Year Review: Different Paths to Success

Abstract

An independent mid-term review of Regional Training Centres (RTCs) to prepare health and nursing services researchers found that the centres were doing a remarkable job in achieving the objectives of the program. The RTCs were using innovative and varied organizational models to deliver high-quality education in applied health and nursing services research, and were offering these programs at multiple university sites, often across provinces. The RTCs received excellent support from the participating universities, and were attracting students willing to exceed the formal degree requirements of their universities to gain access to decision-makers in placements/residencies and institutes and workshops. The decision-makers, in turn, valued this contact as it provided access to a cadre of well prepared, potential future employees and, significantly, to the body of research that the students produced. The major challenge now for the RTCs, the universities and the funders lies in developing appropriate models for sustaining this enormously successful experiment when the 10-year funding period ends.

Key messages

• Highly innovative university programs that operate across institutional and provincial boundaries and that require the involvement of organizations beyond the universities present challenges that must be understood by all concerned, particularly the funders.

• Achieving success in such programs requires that funders, institutions and awardees collaborate in a flexible way, particularly during the early stages of the award.

• This flexible cooperation will be important as the RTCs, which are seen as successful by all partners, seek to establish sustainability at the end of the funding period.

The purpose in establishing the Regional Training Centres (RTCs) – the Western Regional Training Centre (WRTC), the Ontario Training Centre (OTC), Quebec's Centre de formation et d'expertise en recherche en administration des services infirmiers (FERASI), the Atlantic Regional Training Centre (ARTC) and one national centre, the Centre for Knowledge Transfer – was to increase the number of applied health and nursing services researchers at the graduate level and, whenever possible, to create synergy with other programs of the funders, the Canadian Health Services Research Foundation (CHSRF) and the Canadian Institutes of Health Research (CIHR) (see Conrad 2008 for a discussion of this partnership). The funders hoped that by encouraging flexibility in the RTCs' program development, new and different educational models would emerge. The success of four of the centres, each unique in its approach, attests to the fulfillment of that hope.

The funders' request for proposals stipulated some shared requirements: a single training program across each region's sites, based on local needs and resources; the enrolment of students from diverse disciplines; provision of student placements/residencies in decision-making organizations; and training in knowledge translation and exchange (KTE; see D'Amour et al. 2008) and ethics. A further requirement was an external mid-term review of each centre. One or both of the authors were members of every review.

After the evaluations, four RTCs were recommended for continued funding without condition. As the national Centre for Knowledge Transfer was unable to offer placements/residencies, the funders accepted the recommendation of the review panel that its funding not be continued.

This paper is based on a 2005 report prepared by the authors for the Board of Trustees of CHSRF. That report, in turn, was informed by the detailed reports of the individual review panels that visited each RTC. Our paper, therefore, is a snapshot of the program in its fourth year of operation. Since then, the RTCs have evolved, partly in response to the reviews, and for that reason, some of our generalizations may no longer apply to every centre. The RTCs as they presently function are covered by other papers in this special journal issue (e.g., Brachman et al. 2008).

The Program

Offering programs with uniform requirements across different universities represented a considerable challenge. Each RTC had to develop standards and requirements common to all students while respecting the degree requirements of the institutions in which the students were pursuing their graduate degrees. The core curriculum, therefore, had to incorporate the required interdisciplinary training in applied health and nursing services research and knowledge transfer (including placements with decision-maker organizations) as well as research ethics, but without overloading the students to the point that the program interfered with the degree requirements of their home universities. The RTCs successfully met this challenge. The pathway to success, however, was different for each centre. Each developed its own organizational and methodological approach:

• The WRTC designed a core curriculum to be delivered at major sites in British Columbia and Manitoba (sites in Alberta were planned at the time of the review and are now in place). This simple model allowed this centre to be established quickly and effectively.

• The ARTC established a free-standing, joint graduate program across four universities and four provinces, an unprecedented accomplishment that required each university to cede some elements of direct control over an academic program leading to a degree in its name.

• The Centre FERASI created a new specialist stream in nursing administration for existing graduate programs (doctoral and master's) at three universities, with formal written agreements specifying the institutional support, including a significant financial contribution.

• The OTC developed a Diploma in Health Services and Policy Research across six universities, requiring students to attain prescribed core competencies.

Further, each RTC established an effective mechanism for student placements or residencies in decision-maker organizations – perhaps the most valuable program component from the perspective of the community and the students.

Delivery of the curriculum in each centre engaged significant numbers of experienced and committed faculty from a range of disciplines, many outside traditional health fields. Most assumed their RTC responsibilities in addition to their usual teaching load. The faculty with whom we spoke were attracted to the RTCs by the program's importance and by the perceived high quality of the students.

Some of the CHSRF/CIHR chairs also took active roles within the RTCs. Their contributions ranged from directing one of the centres (the OTC) to participating in the teaching program. Students of the chairs were frequently trainees within the RTCs.

Just as different approaches evolved in each centre, initial implementation produced different results. For example, Web-based courses emerged, but their success at the time of the review varied among the centres. Some locations had little experience with electronic course delivery and were unprepared for the considerable investment of faculty time. Others with more experience had greater success. With increased experience, there is potential for cooperation among the RTCs, both in terms of process and course content, leading possibly to national Web-based courses – for example, on KTE or research methods.

The RTCs also developed institutes, workshops or equivalents in which students, faculty and decision-makers meet. These provided opportunities for students to interact with decision-makers and with one another over relevant topics.

The Students

The students entering the RTCs came from very diverse backgrounds in health and other disciplines, including education, sociology, social work, environmental studies and political science. Our contact with them was among the most satisfying and enjoyable aspects of the site visits. The students were articulate, confident and very committed to health services research.

Unlike the ARTC, where there was a new, free-standing degree program, the students at the other centres were undertaking a curriculum that added considerable work to the graduate degree requirements of their home universities. Despite the extra load, they sought admission to the RTCs because the placements/residencies and the institutes would offer unprecedented access to decision-makers, the possibility (at the OTC) of additional accreditation and an additional stipend for RTC study. The stipend has been especially important for practising nurses entering the Centre FERASI's program, as it has allowed them to continue their education without an unacceptable drop in income. For nurses, this may represent a significant means of enhancing recruitment and retention.

In general, the students benefited from their RTC experience by leveraging their diverse backgrounds with the unique program content. Following graduation, their interests and career plans appeared to be equally diversified between academia and the applied sector.

The Decision-makers

RTC decision-makers included policy makers from government, regional health authorities, local community advocacy groups, small service organizations, major hospitals and national committees. They were an essential part of the program, providing student placements and residencies, sitting on the RTC Advisory Boards and participating in the institutes and workshops.

Students saw the decision-makers as an especially significant resource because this contact with the applied sector helped them gain a sense of real-word accountabilities. In particular, students could observe the performance of the health system and the influence of research upon it, and gain insight into how research literacy, evidence-informed decision-making and knowledge transfer operate at that level.

For their part, decision-makers placed high value on their association with the RTCs, often committing significant human or financial resources to the relationship. Besides appreciating the opportunity to influence training and research, they viewed the centres, more pragmatically, as a means of furthering the education of existing staff and as a source of future employees. The decision-makers also valued their contact with the faculty researchers, who were a source of information and advice.

The Institutions

The development of a multi-site, interdisciplinary program required strong support from the senior administration and governance bodies of the institutions involved. Institutional support was ongoing, particularly where some form of inter-university accreditation was present, or where formal agreements were in place.

While the senior administrators were generally well informed about the RTCs, they did not appear to have recognized the full potential of their successful operations, particularly the mutually beneficial, senior-level relationships between the centres and the applied sector. In future, institutions, especially those wishing to embed themselves more firmly in their surrounding communities, will be able both to utilize the contacts that already exist between the RTCs and decision-makers and to create new partnerships inspired by the RTC model. We saw a need for improved marketing of the accomplishments and potential of the RTCs within participating institutions.

Key Issues

Here we describe some early challenges and successes of the RTC enterprise. We then suggest some aspects that will become increasingly important as the centres evolve and look towards a future beyond the CHSRF/CIHR grant.

Early difficulties

There were some stumbles on the path to success.

Reliability of Data

At the time of the review, there were no reliable data for the program as a whole related to the numbers, previous experience and post-program placement of the students involved. Each RTC was collecting information, but there was no agreement across the regions, for example, about how to define an RTC student. In some cases, any student registered in an RTC course was deemed a student, while in others, only those receiving a stipend from the RTC were considered students. It is our understanding that this significant difficulty concerning data has now been addressed. The concern was deeper than mere administrative tidiness. The RTC program, after all, was widely viewed as a bold experiment in increasing the capacity for health services research in Canada; assessing the results of that experiment therefore requires high-quality data about the students who participate in the programs.

Strategic Planning

The RTCs struggled with two related requirements of the program. They were (a) to develop some form of advisory body and (b) to undertake strategic planning for the development and long-term future of the centre, providing an accountability framework to guide progress towards the strategic objectives.

At the time of the review, all the RTCs had established Advisory Boards. Typically, these included the most relevant stakeholders as members, acted as an important bridge across universities and provided good personal support to the directors. However, the form, frequency of meeting and formal responsibilities of the boards varied, and while these bodies dealt occasionally with strategic issues, their involvement in this sphere was inconsistent.

The RTCs recognized that they needed to engage in strategic planning to address their broad, long-term issues, but the results at the time of the review were mixed. Only one centre provided a thorough analysis of the environment and possible sources of revenue once the initial funding ended. The others generated operational rather than strategic plans, and omitted some important considerations. Missing elements included the link between the strategic direction of the RTCs and the strategic plans of the collaborating universities, the challenge of enhancing the profile and relevance of the centres to potential funders now and in the future, and the impact on the RTCs of changing internal and external environments.

Implementation

Many RTC directors found the first two years of implementation to be frustrating: they experienced onerous and shifting program and financial reporting requirements, user-unfriendly databases and a sense that CHSRF/CIHR did not understand the academy. Conversely, the funders were perplexed by what they saw as professors and institutions unwilling to conform to the conditions of the awards.

In retrospect, this disconnect was understandable. The CHSRF/CIHR CADRE partnership (Capacity for Applied and Developmental Research and Evaluation), under which the RTCs were funded (Conrad 2008), represented a major thrust to attack directly the gap in applied health and nursing services research capacity in Canada. However, because the CADRE initiative itself was so innovative, there were no templates to follow. Given the pressures of the day, it is not surprising that decisions about the implementation and administrative underpinnings of the RTC program were made too quickly by both the funders and the awardees. The granting agency, by working collaboratively over time with the award holders, was able to identify the difficulties and help address them so that by the time of the review, most of the RTCs were operating smoothly.

Factors for Success

Committed Personnel

Like the other programs in CADRE, the RTCs are highly innovative and depended upon individuals in the academy who would be attracted by the vision and willing to commit to the objectives. That, in our view, is one of the primary factors for the RTC programs' success: the faculty who stepped forward to accept the challenge of establishing these centres were highly dedicated to their overall purpose. The cynical might observe that the funding was the principal attraction. On the other hand, the people who applied for these awards were taking on a good deal of additional work with few obvious rewards. Without a group of applicants who were commited to the discipline, the RTCs would have failed.

Conditions Attached to the Awards

The RTCs' funding was contingent upon their meeting the following conditions:

• an implementation commitment of 10 years, a degree of security unprecedented in the academic research environment;

• financial contributions from decision-maker partners and the institutions, resulting in a leverage of funds in many cases beyond the initial required sums;

• a rigorous selection process for the awards, involving a two-tiered, competitive, internationally peer-reviewed process to ensure excellence;

• hiring of an administrative manager for each RTC to allow the academics to concentrate on academic matters: without such an administrator, the professors would have been overwhelmed with disparate claims on their time and attention.

Program Design: A Collaborative Environment

In spite of the initial difficulties and misunderstandings described above, the collaborative approach that emerged in the design of the RTC programs is undoubtedly key to their eventual success. This experience contains, perhaps, a lesson for CHSRF/CIHR and other funders: innovative programs are more likely to succeed in an environment of mutual trust, in which both the awardees and the funders understand each other's requirements and work together to produce the conditions that ensure those requirements can be met.

The evolving academic environment

The environment in which the RTCs were conceived was very different from the environment after four years of operation. That evolution has continued at a very rapid pace and will present both challenges and opportunities for the RTCs. Some elements of the changes have included:

• greater availability of research funds for health services research, as the result of greater investment in research by former federal governments;

• expansion of the mandate of CIHR to include health research in all disciplines;

• the need to replace aging faculty, and to expand graduate programs to meet that need;

• greater interest by several universities in health services research with the establishment of new, free-standing programs and research institutes; and

• increased competition for graduate students among the disciplines.

Sustainability

The issue of financial sustainability is considered in another paper (Montelpare et al. 2008). From the reviewers' perspective, it was apparent that the RTCs achieved much of their success by exploiting the momentum derived from the initial funding and its 10-year term. That momentum may disappear once the grants end.

For any particular centre, sustainability may or may not imply continuation in its current form. Much depends on the individual environment. If that environment, internal or external, has changed or is changing, then “sustainability” may have a different meaning for that RTC.

Each of the RTCs is to some degree institutionalized, and all involve more than one institution. In the case of those that offer special accreditation, it could be argued that the institutions have made a commitment to continue to offer their program even if a particular RTC should cease to exist. While this assumption may be valid, it is also perhaps naïve. Programs depend for their success on a sufficient number of applicants, and there is no doubt that the existence of the RTCs, and particularly the stipends that they offer, attract students from a diversity of disciplines. Moreover, certain key aspects of the programs – such as coordination among the participating institutions, placements/residencies and the institutes/workshops – require at least a minimal infrastructure and moderate levels of funding.

Further, the mid-term review revealed that decision-makers value the RTCs as a source of employees, a resource for the education of their existing staff and as a fount of research information. What a significant development for the program, one that mutually benefits students, institutions and decision-maker partners!

The greatest challenge lies in maintaining the funds needed to support the current programs or some similar version of them: infrastructure, multiple sites, placements/residencies, institutes/workshops and students. Each RTC will have different opportunities: for example, in at least one centre, the provincial government may be willing to increase its contribution.

Part of the solution may lie in a more entrepreneurial approach. A few possibilities include:

• a multi-year funding agreement with decision-maker organizations for the support of students in placements;

• agreement on a retainer fee for the delivery of a specified set of services;

• charging consultant fees for delivery of research services;

• development of an educational module, perhaps with some form of accreditation, that could be delivered, for a fee, to members of decision-maker organizations.

The responsibility for addressing sustainability should be shared by the RTC award holders, the institutions and the funding bodies. The current success of the centres bodes well for finding a solution to the issue of their viability.