Graduate Attributes for Master's Programs in Health Services and Policy Research: Results of a National Consultation

Abstract

Objective:

Our objective was to identify desirable attributes to be developed through graduate training in health services and policy research (HSPR) by identifying the knowledge, skills and abilities thought to be keys to success in HSPR-related careers. We aimed for a framework clear enough to serve as a touchstone for HSPR training programs across Canada yet flexible enough to permit diversity of specialization across and within those programs.

Methods:

Our approach involved several stages of data collection and analysis: a review of literature; telephone interviews with opinion leaders; online surveys of HSPR students, recent graduates and employers; an invitational workshop; and an interactive panel at a national conference. Our final framework was arrived at through an iterative process of thematic analysis, reflection on invited feedback from consultation participants and triangulation with existing competency frameworks.

Results:

Our final result was a framework that identifies traits, knowledge and abilities of master's-level graduates who are capable of fostering health system improvement through planning, management, analysis or monitoring that is informed by credible evidence and relevant theory. These attributes are organized into three levels: generic graduate attributes, knowledge related to health and health systems and, finally, attributes related to the application of knowledge for health system improvement. The HSPR-specific attributes include not only an understanding of HSPR theories and methods but also the skills related to the practical application of knowledge in the complex environments of health system decision-making and healthcare policy.

Conclusion:

Master's-level HSPR training programs should prepare students to pose and seek answers to important questions and provide them with the skills necessary to apply their knowledge within complex decision-making environments.

What distinguishes a health services and policy research (HSPR) graduate from graduates of related fields such as epidemiology or population health? What traits, knowledge and skills should HSPR training programs be sure to help all their students attain? Most readers will have ideas about how to answer these questions; however, we suspect those answers would vary considerably. HSPR is an interdisciplinary field – one that attracts people from a wide variety of backgrounds and shares somewhat blurred boundaries with related fields. This range would explain some variety in how people might envision HSPR graduates. Another reason for variation in answers to questions about the attributes of HSPR trainees is that, despite the many indicators that HSPR is maturing as a field of its own – the emergence of funding institutes, a national association, annual conferences and a dedicated journal – HSPR in Canada presently lacks a common definition of its core competencies.

A clear definition of attributes that distinguish HSPR graduates would be helpful to educators, students and employers. It would serve the practical goal of communicating the abilities that employers can expect in HSPR graduates and that prospective students can expect to attain through HSPR training. Moreover, the process of exploring and delineating the field's unifying attributes can help improve our understanding of the role or purpose of HSPR (Pittman and Holve 2009), and in doing so, assist individuals in identifying themselves as members of the HSPR field – a potentially important outcome of advanced education (Lucia and Lepsinger 1999; Barnett et al. 2001).

Several training centres have published lists delineating learning objectives for their students (Atlantic Regional Training Centre 2008; Ontario Training Centre in Health Services and Policy Research 2008; Western Regional Training Centre for Health Services Research 2008). While these lists represent important steps for the Canadian field as a whole, each was independently developed, varies in structure and content from the others and includes elements that may not be relevant for non-academic HSPR-related careers. American researchers have conducted two national consultations to determine core competencies for US doctoral programs in health services research (HSR) (Martin 2008; Forrest et al. 2009). By design, however, these lists of competencies were tailored for predominantly academic careers in the context of the US health system and related research and policy foci.

This paper summarizes an extensive consultation with experts from across Canada that we undertook to identify desirable attributes to be developed through graduate training in HSPR. We began by seeking broad input on the “core competencies” of the field: the skills, knowledge and abilities that are keys to success in HSPR-related careers today and in the foreseeable future. We also sought input on how HSPR training could be improved to better prepare students for related careers. Our approach involved several stages of data collection and analysis, which we report in sequence here: a review of literature on competencies, graduate attributes and curriculum design; telephone interviews with renowned leaders in HSPR-related careers; online surveys of current HSPR students, recent alumni of HSPR training programs and employers of HSPR graduates; an invitational workshop; and an interactive panel at a national conference. Readers who wish to ‘skip to the chase’ can simply bypass the methods and analysis sections of this paper and review the final framework in the results section (page 75).

We wish to emphasize from the outset that we did not set out to professionalize HSPR with narrowly defined, task-specific competencies. Instead, we aimed for a framework of desirable graduate attributes that was clear enough to serve as a touchstone for HSPR training programs across Canada yet flexible enough to permit diversity of specialization across and within those programs. The final framework lists attributes that would ideally be possessed by all graduates of master's programs in HSPR, regardless of their area of specialization. Similar frameworks for subspecialties of the HSPR field – including frameworks for doctoral-level trainees – should be developed through subsequent consultation processes.

Background: Core Competencies and Graduate Attributes

Many members of the HSPR community will be familiar with “core competency” lists. This is so because many identify not only as being in the HSPR field but also as members of professions that use “competence” as a condition of membership. HSPR-related professions that have developed and promoted detailed competency lists include medicine, nursing, pharmacy, public health and health administration (Calhoun et al. 2002; Neilson et al. 2003; CNA 2005; ACGME 2007; Burke et al. 2008; Calhoun et al. 2008; Leach 2008; PHAC 2008; Laidlaw et al. 2009). A unifying theme of processes for professional competency definition and assessment – though one not always adhered to in practice – is that it is both possible and desirable to define a set of observable traits that have a causal relationship to workplace performance (McClelland and Boyatzis 1980). Professional training programs would be designed to help students attain such competencies, and accreditation processes would test for them in order to certify standards of the profession.

Advanced education beyond the boundaries of accredited professional schools has also embraced curriculum development based on specified learning objectives (Lucia and Lepsinger 1999; Markus et al. 2005; Howard 2007). Comprehensive educational programs – in some cases, entire university systems (Barrie 2004; Bath et al. 2004) – have been influenced by the view that defining a range of desirable graduate attributes is a useful mechanism for rational and accountable curriculum development. Here, the phrase graduate attributes – as opposed to core competencies – is used to connote a set of learning outcomes that extends beyond the demonstration of workplace-specific roles and abilities. The phrase helps distinguish more flexible approaches or guides to curriculum development from the more focused professional core competency models that are often praised and yet much maligned (Lucia and Lepsinger 1999; Voorhees 2001; Westera 2001; Jones et al. 2002; Jarvis and Parker 2004; Howard 2007). The use of the term “attributes” also helps to serve as a reminder that all components of training programs – from admissions through curriculum, practicum and evaluation – can be affected by defining them (Barrie 2004, 2006).

For specific disciplines or fields, such as HSPR, frameworks for defining desirable attributes for graduates often take the form of pyramids (Lucia and Lepsinger 1999; Calhoun et al. 2002; Jones et al. 2002; Barrie 2004; Chyung et al. 2006). At their foundation are generic yet enabling traits such as critical thinking, creativity, teamwork and communication skills. Some of these generic attributes may even be viewed as prerequisites for entry into a field when their attainment is a necessary foundation for the development of field-specific competencies (Barrie 2006). Secondary attributes in these frameworks generally include content area knowledge and skills, but might also include certain core values or orientations. Finally, attribute frameworks often include demonstrated abilities to apply competence in the context of a particular field. Whereas the demonstration of abilities in professional competency models may focus on narrow workplace competencies, graduate attribute frameworks may incorporate more general definitions of how abilities and knowledge are applied, permitting diversity within a field and interpretation by particular training programs or departments.

Methods: The Consultation Process

Input from leaders in HSPR-related fields

Telephone interviews

Over a 10-week period in 2008, we conducted a series of telephone interviews with individuals identified as key leaders in HSPR-related careers. To identify interviewees, we compiled a list of suggestions from the CIHR's Institute for Health Services and Policy Research, the Canadian Health Services Research Foundation, the Canadian Association for Health Services and Policy Research and directors of health services and policy research centres across Canada. We purposefully chose a sample of 67 of the 106 recommended experts from across Canada and from a diversity of backgrounds and places of work.

Forty-five people, representing all provinces except Prince Edward Island, agreed to participate in a telephone interview (67% participation rate). Twenty-eight (62%) of the interviewees were non-academics in a range of mid- to senior-level positions and careers. Interviews were conducted by the principal investigator (SM) and, whenever possible, the research assistant (KO). Interviews lasted 20 to 40 minutes and involved a maximum of 12 open-ended questions about keys to success in HSPR-related careers today and in the foreseeable future (see online Appendix). We digitally recorded the interviews and coded the professionally transcribed transcripts using NVivo software. Coding and analysis were guided by the qualitative coding strategies of Miles and Huberman (1994). Two of us (SM and KO) independently generated open coded themes from the raw transcripts and met to develop a final coding scheme. We identified 22 broad theme areas under which we grouped 53 competencies and traits.

Results from expert interviews

Table 1 lists the 10 competencies most frequently identified in telephone interviews as keys to success in HSPR-related careers. Two of the three competencies most commonly cited related to knowledge of the Canadian healthcare system. Most notably in this regard, many experts that we interviewed suggested that HSPR graduates needed to understand the politics of healthcare systems and policies in addition to basic knowledge of healthcare system structures. This level of understanding policy making related to an overarching theme of HSPR careers as a bridge between academe and the real world of health systems and policies:

TABLE 1.

Ten competencies most frequently cited during telephone interviews with key leaders in the HSPR field

	Mentions by non-academics (n=28)	Mentions by academics (n=17)	Total mentions (n=45)
Knowledge of the Canadian healthcare system	16	8	24
Understanding of research methods and study designs	13	11	24
Understanding of the politics of healthcare systems and policy processes	14	6	20
Ability to engage in meaningful research producer–user partnerships	16	4	20
Effective, non-technical written communication skills	17	2	19
Strong interpersonal and teamwork skills	16	2	18
Ability to think critically and analytically	12	6	18
General communication skills	14	3	17
Ability to conduct quantitative research	9	6	15
Sufficient understanding to be able to assess/interpret quantitative research	9	6	15

I think it is absolutely critical for people to have some grounding in politics. That is, political theory, political science and public administration. It doesn't have to be huge, but there needs to be a basic understanding of how democracies work and how – in the healthcare system – resources get allocated. How interests come to play in decisions and how decision-makers digest and use research-based evidence and/or don't use research-based evidence in what they do. (Private consultant)

Certain research skills also ranked highly in terms of frequency of mention in our telephone interviews. Research skills were typically discussed in the context of the ability to critically appraise existing work and to formulate important research questions. As one expert noted, the HSPR career in most non-academic environments is one of being an educated “user” of research, not a “producer” of research:

We do less conducting research in the health system, in the applied side. We do some [in our organization] … but it tends to be the university community that drives and conducts research. So if I was employing somebody in [our organization], I would be less expecting them to be conducting studies rather than applying research and developing policy based on research that was done. (Decision-maker in a regional health authority)

One of the most interesting themes to emerge from the interviews was an emphasis on practical skills as essential traits for success in HSPR-related careers. These skills included communication, teamwork, critical thinking and – quite bluntly – intellect:

It's absolutely essential that people are able to express complex ideas in a succinct way. (Senior executive of a public agency)

Some of the most successful people [in our organization] are those people who have good personalities – who have good people skills. Who can build bridges and be able to share the kind of expertise that they can bring to a problem in a way that is ultimately constructive and advances the larger organization's learning. (Senior government manager)

But really, at the end of the day, it's the ability to make sense of a complex and vast amount of information and being able to draw insights from it. (Government analyst)

A final theme of interest from the telephone interviews was that when asked what one thing they would change about HSPR training in Canada, 31 of the 45 leaders interviewed (69%) recommended more student interaction with the healthcare system. Work placements or co-op–based programs were mentioned by 21 of these experts (47%); assigning projects or theses to be done either in collaboration with or in response to questions generated by decision-makers was mentioned by 10 others (22%). No other category of recommendation for HSPR training was cited by more than five people.

Online surveys of students, alumni and employers

Online survey

Over three months in the fall of 2008, we surveyed HSPR students, recent graduates and employers of HSPR graduates using LimeSurvey software. Students and alumni were recruited to these voluntary surveys through focused e-mails sent to 16 training programs across Canada: 12 university department–based programs in HSPR spanning eight provinces as well as the Atlantic, Ontario and Western CHSRF/CIHR Regional Training Centres for health services and policy research and the Quebec Regional Training Centre for nursing services research (Conrad 2008). Employers were invited to participate by e-mail notices sent via contact lists developed in order to select experts for telephone interviews (excluding those who were interviewed by phone) and through broadcast e-mails sent by the Canadian Association for Health Services and Policy Research and the CIHR's Institute for Health Services and Policy Research.

Survey questions were tailored for participant groups, but all three surveys included open-ended, short-form questions similar to the telephone interviews in their focus on the knowledge, abilities and skills required for success in HSPR-related careers. Two of us (CM and KO) independently coded all survey responses using a coding scheme that best suited the format of and themes from online survey responses.

Results from student and alumni surveys

We received 72 complete student surveys, 36 complete alumni surveys and 39 complete employer surveys. Students responded from 14 universities, representing all provinces but Prince Edward Island. Over half (56%) of the students were in master's programs. Alumni responded from seven provinces and from the United States, with a majority (61%) being employed in Ontario. Just 11% of alumni respondents were in academic careers. Employer surveys were completed by participants from eight provinces, with Ontario accounting for the largest number (15 of 39 completed surveys). Fourteen employer surveys were from university-based academics and 25 came from persons working in government, health authorities, hospitals, consultancies and health associations.

Table 2 lists the competency themes most commonly cited by students, alumni and employers who completed online surveys. As with telephone interviews, online surveys revealed a pattern of competency themes that suggested an “applied scholarship” view of HSPR careers. Moreover, the range of graduate attributes identified as necessary for HSPR success were similar among students, alumni and employers, with all groups citing the same four categories of attributes with the highest frequency. These most frequently cited competency themes included general research skills; personal professional skills such as time management, flexibility, patience and curiosity/drive; comfort with interdisciplinary teamwork; and knowledge of Canada's health system. Current students appeared to differ somewhat from alumni and employers with respect to the relative importance they placed on research skills (more frequently cited by students) versus knowledge of policy processes and politics (more frequently cited by alumni and employers).

TABLE 2.

Ten most frequent competency themes in online surveys submitted by HSPR students

Students	Alumni	Employers
General research skills	General research skills	Interdisciplinary teamwork
Professional skills	Professional skills	Professional skills
Interdisciplinary teamwork	Interdisciplinary teamwork	General research skills
Canadian health system knowledge	Canadian health system knowledge	Canadian health system knowledge
Quantitative research methods	Writing skills	Quantitative research methods
Knowledge translation skills	Critical and analytical thinking	Research methods knowledge
Research methods knowledge	Research methods knowledge	Critical and analytical thinking
Policy analysis skills	Quantitative research methods	Partnership skills for KT
Qualitative research methods	Knowledge of policy processes and politics	Knowledge translation skills
Critical and analytical thinking	Partnership skills for KT	Knowledge of policy processes and politics

Alumni and employer responses also encompassed a more recognizable and consistent range of terms and concepts than student responses: a discrete “language of HSPR.” Perhaps this language is acquired over time and through exposure to the concepts of this field – access, quality, equity, cost-effectiveness, etc. – in the context of applied settings and policy environments. An additional and perhaps related finding of interest from the student survey was that eight student respondents offered unsolicited comments about their wishes to have additional opportunities to participate in “policy” or “decision-maker” settings in a “hands-on” or “practical” way; as one student stated plainly, “more time with non-researchers in their environment.”

Deliberative consultations

Invitational workshop

After tabulating data from the telephone interviews and online surveys, we held a workshop to which we invited 30 people selected based on their standing in the field, geographic representation and professional background. Twelve academics and 12 non-academics accepted the invitation to participate. Prior to the workshop, we sent these participants summaries of the leading themes identified in our telephone interviews and online surveys and asked them to look for emerging themes and gaps.

At the workshop, borrowing from the question asked by a group working towards Master of Public Health competencies in the United States (Moser et al. 2008), we asked our participants to answer the following question: “What types of knowledge and abilities are needed by every HSPR graduate for them to be effective in their careers and in lifelong learning, whatever form or focus their HSPR-related career might take?” Following a lengthy group discussion about themes and gaps in data generated to that date, participants identified four domains of HSPR competence that are common to most (if not all) HSPR-related careers: (1) theories and frameworks, (2) research methods, (3) policies and systems and (4) knowledge translation. Breakout groups discussed and reported back with recommendations concerning the nature of competency in these four domains.

With respect to theories and frameworks, workshop participants argued that being equipped with discipline-specific theories (typically gained prior to HSPR training) and theories of the HSPR field was as essential to critical thinking and creative problem-solving. They identified a few HSPR-specific theories and concepts – i.e., healthcare utilization, access and quality – and highlighted a few HSPR-related, but not HSPR-specific, theories – e.g., health economics, determinants of health and organizational behaviour.

With respect to research methods, participants concluded that HSPR graduates should be able to formulate relevant research questions and to critically appraise methods – including experimental, non-experimental, qualitative and quantitative methods – for answering such questions.

With regard to policies and systems, participants suggested that HSPR graduates should understand the history and structure of the Canadian healthcare system and to be able to place that system within a comparative international context. Workshop participants also urged that graduates should be able to work with basic ideas of political economy, such as how interests, institutions and ideas shape policy processes.

Finally, with regard to knowledge translation, participants argued that HSPR graduates need to be able to engage effectively with various audiences and to understand the barriers and facilitators to the use of evidence in practice and policy. Workshop participants highlighted the importance of communication skills, including plain language writing, teamwork, presentations and relationship building.

Conference panel

Following the invitational workshop, we hosted a panel at the annual meeting of the Canadian Association of Health Services and Policy Research in May 2009. Over 80 participants attended the panel (making it the largest parallel session held during that time slot). The panel consisted of short talks by the study authors, followed by comments from four invited experts and an interactive discussion with attendees. Detailed field notes were taken by the principal investigator (SM) during the panel session and incorporated into the latter stages of theme definitions and framework development.

The conference panel discussion produced several notable recommendations regarding a framework for core competencies in HSPR. Panel participants and audience members noted that it is important to distinguish HSPR from related fields – such as population health, health administration, public health and clinical epidemiology – and to be clear about the role of the “P” (for policy) in HSPR. At the same time, participants also argued that while trying to be clear in defining this field, we should deliberately strive to keep the framework “interpretable” so that it is inclusive of our community's diversity and flexible enough for programs and students to use as a base for building their own concentrations.

A repeated concern among participants in the panel session was that there is a risk that we might set the bar too high for HSPR trainees. Participants argued that a framework for graduate attributes – and its interpretation by training programs – needs to be pragmatic about what can be learned in degree programs and what will ultimately require on-the-job training over the course of a career. Others suggested that simply introducing students to a breadth of concepts may suffice to help prepare them to confront the concepts again later in their careers. A key theme in relation to this point was that “not every competency equals a course” – in other words, don't over-program students and don't try to teach everything in the classroom.

Finally, panel participants and audience members recommended that we not place too great an emphasis on training for an academic track. Most master's-level graduates (and many graduates of doctoral programs) enter careers in non-academic settings that require a different balance of skills related to the collection, appraisal and use of evidence versus skills related to evidence production. As one participant noted, HSPR professionals in non-academic careers often have to review and synthesize knowledge bases quickly, so that they can respond to requests from their employers in “hours or days” rather than the “weeks, months or years” time scale of academe. Others noted that the flexibility and practicality of skill sets are important because HSPR-related careers often require the ability to become reasonably “expert” in several topics over the course of a given year.

Feedback on draft frameworks

After reviewing the data from all prior stages of the analysis, the principal investigator (SM) developed a draft framework of attributes that should be attained by graduates of master's-level HSPR training programs in Canada. This framework was based on triangulating input from consultation with the existing HSR competency frameworks from the United States and the learning objectives of Canadian training centres. We circulated that draft by e-mail to the 51 individuals who participated in telephone interviews, the invitational workshop or both. Nineteen of these individuals (37%) responded with feedback and suggestions on the framework during August and September 2009.

By the time we had disseminated the draft framework for comments, the focus was clearly placed on the largely non-academic career paths that a majority of master's-level graduates take. All individuals who provided feedback felt the focus on master's-level graduates was appropriate and that the framework could be useful for training programs across Canada. Nevertheless, most (17 of 19) had suggestions for improvement – ranging from minor rewording to re-organizing the order in which specific attributes were presented. In this process, we were reminded again to ensure that the framework is motivated by consideration of the career paths of master's-level HSPR graduates.

Results: A Framework of Graduate Attributes

To arrive at the final list of desirable graduate attributes for those entering the field of HSPR, it was necessary to define the field and, in particular, the ultimate goal of HSPR education programs in terms of the impact(s) on graduate careers. We posit that HSPR is a problem-oriented field into which people enter from a wide range of disciplinary backgrounds to work together to find ways that healthcare can best be organized, financed and delivered. This includes work conducted to measure, assess and monitor health services, which is captured by the AcademyHealth definition of the field of health services research (HSR, no “P”):

Health services research is the multidisciplinary field of scientific investigation that studies how social factors, financing systems, organizational structures and processes, health technologies and personal behaviors affect access to health care, the quality and cost of health care and ultimately our health and well-being. Its research domains are individuals, families, organizations, institutions, communities and populations. (Lohr and Steinwachs 2002)

On the basis of our consultations, we would add that HSPR (with the “P”) also includes an evidence-informed focus on a public purpose: to improve the health and well-being of patients and populations by advancing health system accessibility, quality, equity, efficiency and safety. (It is worth noting that members of the American health services research field have also argued for such an outcomes focus in their definitions [Phillips 2006].) The “P” in HSPR may be viewed as a symbol of this community's focus on effecting change and therefore on the need to understand the policy processes – at local, regional, provincial, national and supra-national levels – that influence health system decision-making and outcomes.

Figure 1 illustrates our framework of attributes of HSPR graduates who would be capable of fostering health system improvement through policy planning, analysis, implementation and monitoring that is informed by credible evidence and relevant theory. It begins with a set of foundational attributes and works upward towards the specific application of those attributes within health systems.

FIGURE 1.

Graduate attributes for master's programs in health services and policy research (HSPR)

Foundational attributes of a lifelong learner

At the base of our framework are foundational graduate attributes: clear communicator, critical thinker, problem-solver, ethical and socially responsible, interdisciplinary and effective worker. The overarching theme of these foundational graduate attributes is lifelong learning, including the curiosity and drive to tackle new problems and the skills needed to do so in a creative and socially responsible way.

As delineated in Table 3, these foundational attributes are clearly “generic” in the sense that they describe characteristics that are not specific to one discipline or context. Most, for example, have been identified as desired outcomes for higher education more generally (University of South Australia 2009). They are nevertheless among the attributes most commonly cited by experts in the field as required for success in HSPR-related careers. We therefore refer to the generic attributes in our framework as “foundational attributes” in order to emphasize the fact that the acquisition of more context-specific attributes is, at least in part, dependent on first developing these generic traits (Barrie 2006).

TABLE 3.

Foundational attributes of a lifelong learner

Lifelong learner Prepared for lifelong learning in pursuit of personal development and career achievement

Clear communicator Able to communicate important aspects of theory, research findings and context clearly and effectively, in ways appropriate to various audiences; able to give, receive and incorporate feedback relevant to research and professional work

Critical thinker Able to critically appraise theory and evidence and to use independent judgment to synthesize information and place it in a broader context

Problem-solver Able to conceptualize problems and use theory, evidence, context and reasoning to identify a range of possible solutions and make recommendations from among them

Ethical and socially responsible Aware of and committed to ethical action and social responsibility in all research and professional activities, including respect for context and diversity – e.g., along lines of ethnicity, sex and gender, geography, education, income, occupation and others

Interdisciplinary Have capacity for and orientation towards working with and integrating the knowledge of people with different academic training, professional roles and socio-cultural backgrounds

Effective worker Able to work effectively, independently and collaboratively within teams, including the ability to plan and manage projects through to completion on time and on budget

The importance of these foundational attributes is underscored by the way that leading HSPR experts responded to questions about which important skills, knowledge or abilities were most commonly lacking among recent graduates. Communication skills were often cited in this context:

Increasingly, I'm finding that people who graduate can't write. Can't write with power…. They can write a 30-page research paper and footnote it to the New England Standard, but they can't write a briefing note. (Senior executive of a public agency)

You can be God's gift to policy research, but if you can't communicate that to the folks who are actually making the decisions … you're extremely deficient. (Former ADM of a ministry of health)

The ability to write clearly and concisely – which also requires critical thinking and creative problem-solving abilities – is thus among the fundamental attributes required for success in careers related to HSPR.

Breadth and depth of knowledge related to health and healthcare systems

Above the generic attributes in our framework are areas of knowledge related to health and healthcare systems: knowledge of health systems, the determinants of health, health research methods, health economic theory, evaluation methods and organizational theory. Delineated in Table 4, these attributes are clearly context-related. The bases for many courses in the departments and schools in which master's programs in HSPR are offered, these areas of knowledge are shared (to varying extents) with related fields such as healthcare leadership, applied epidemiology and public health.

TABLE 4.

Breadth and depth of knowledge related to health and healthcare systems

Disciplinary depth Possess detailed knowledge and skills from a specific discipline or field related to health services and policy research (e.g., epidemiology, medicine, nursing, sociology, economics, political science or management)

Health systems Able to describe the main features of the Canadian healthcare system and to place the Canadian system in a comparative international context

Determinants of health Able to define and work with concepts of health and identify the relative importance of broad determinants of health at the individual, group, community and population level

Health research methods Able to explain how health and disease are measured and how relationships between determinants (e.g., environmental, behavioural or treatment) and health are established

Health economic theory Able to explain how health and healthcare differ from ordinary unique economic goods and describe core concepts in health economics (e.g., cost-effectiveness, health insurance, moral hazard, etc.)

Evaluation Able to identify appropriate ways in which health services can be evaluated using tools of program evaluation, health technology assessment and/or health economics

Organizational theory Able to describe a variety of theories concerning how people interact within and between organizations and to place such theories in the context of healthcare systems

It is important to recognize that no master's-level graduate could be an expert in all these context-related subject areas. Master's-level graduates should simply be conversant with these areas of content knowledge; that is, they should be sufficiently familiar with each subject that they would be prepared to work with related theories or research evidence as needed. One former assistant deputy minister of a ministry of health described this breadth requirement as “the ability to span the discipline” and to work on a variety of projects. Several decision-makers suggested that a balance is achieved when people know enough to be able to seek out and make partnerships with experts in particular niche areas.

Along with a breadth of subject area conversance, a master's-level HSPR graduate should also possess somewhat deeper knowledge and skills from one or more specific disciplines and on one or more HSPR content areas. Disciplinary depth may come from such fields as medicine, nursing, sociology, economics, political science or statistics. Content depth – in areas such as primary healthcare, mental health, health human resources, etc. – could be developed through major projects, term papers, work placements or even theses. The disciplinary depth requirement, in particular, is a precursor attribute that provides a framework for explaining and exploring a range of phenomena related to the subject area (Barrie 2004).

This combination of broad subject knowledge, disciplinary depth and interdisciplinary comfort (one of the foundational skills) is a hallmark of an HSPR professional.

Application of knowledge for health system improvement

The very peak of the framework is the application of knowledge for health system improvement. HSPR-specific attributes related to health systems insight and acumen are therefore the top cluster of graduate attributes for HSPR trainees. These include the drive to pose important questions about health services and policy in pursuit of health system improvement; the ability to find credible and reliable answers to those questions; and the skills necessary to apply the resulting knowledge within complex decision-making environments. Delineated in Table 5, the specific attributes in this cluster are health policy insight, knowledge of HSPR theory and HSPR methodology, and the ability to generate, synthesize and exchange HSPR knowledge.

TABLE 5.

Attributes related to the application of knowledge for health systems improvement

Health systems insight and acumen Inclined to pose and seek answers to important questions about health services and policy, informed by decision-making needs, existing evidence and relevant theory; aware of the many considerations that influence decision-making in health systems; able to know when and how to support evidence-informed change

HSPR knowledge exchange Aware of the needs of and constraints on different participants; able to communicate effectively with researchers, health professionals, policy makers and communities for the purpose of planning, conducting and translating health services and policy research

HSPR evidence synthesis Able to identify sources of information and data on health services and policy issues and to acquire, assess and synthesize relevant theory and evidence for application to particular contexts

HSPR evidence creation Able to contribute to original research to answer important health services and policy questions, including ability to help frame questions, select methods and interpret results

Health policy insight Able to describe decision-making processes in the healthcare system and to identify historical, political, economic and legal factors that influence health policies and their implementation at a national, provincial and local level

HSPR methodology Able to describe the approaches, strengths and weaknesses of a variety of research designs used to study health services and policy issues; able to identify methods (including mixed methods) appropriate for generating valid and reliable information on specific research questions

HSPR theory Able to define and work with core concepts of health services research (e.g., need, access, quality, equity and safety); able to identify, select from and work with a range of theories (e.g., social, political, behavioural and economic) regarding the organization, financing, delivery, utilization and outcomes of health services

Although people come into HSPR from a variety of disciplines and are therefore equipped with a range of knowledge and theoretical frameworks, the field of HSPR has its own theories, concepts and methods. These have evolved over time, often adapted from frameworks and tools from specific disciplines and applied to health systems. Because they are important to explaining and exploring a range of phenomena related to the field, the core HSPR concepts, theories and methods should be understood by all master's-level graduates of HSPR programs. These include understanding of such concepts as accessibility, quality, equity, efficiency and safety; sufficient theoretical knowledge to be able to identify appropriate questions and hypotheses regarding such concepts; and sufficient understanding of both quantitative and qualitative methods to be able to identify strengths and limitations of different approaches to addressing each question.

HSPR graduates should also develop an understanding of the “real world” of decision-making and an ability to advance evidence-informed change within its complexity and politics:

Basically, to work in non-academic jobs, you have to be comfortable with uncertainty, arbitrariness, the ebb and flow of policy moments, organizational dynamics, etc. … those who succeed in these environments have a certain feel for what is important, what language to use when engaging decision-makers, some sensitivity to the rhythms of democracy and empathy with the dilemmas confronting governments. (Private consultant)

This attribute, so frankly emphasized in all stages of our consultation process, poses perhaps the greatest challenge for university-based HSPR training programs. It requires an explicit acknowledgement that evidence is not the only driver of policy and practice; it also requires mechanisms for teaching practical skill sets that are seldom used within academe.

Discussion

The framework presented above includes a long list of desirable attributes for graduates of master's-level HSPR programs. As training programs strive to help students develop these attributes, it will be important to recall that not every attribute requires a course to attain it. Indeed, some of the most important competencies delineated above are ones that can be developed only through interactions with the system and work experience. Others, including a range of subject knowledge, may be obtained in survey courses that cover a variety of content.

Some of the attributes identified through our consultations as key to success in HSPR-related careers may be seen as prerequisites for graduate training programs themselves. Communication skills and various traits of “bright” individuals – critical thinking, problem-solving and ability to work productively, for example – were the attributes most commonly identified as required for success in HSPR-related careers. Of course, these skills can (and should) be enhanced or tailored for specific applications through various learning opportunities. But an HSPR training program is not the place for remedial action to be taken on foundational attributes that are seriously deficient.

Finally, it would be a mistake to suggest that our consultation did not uncover sources of tension among the many shared beliefs. Perhaps the most troubling tension was the fact that a few of our peers did not appear comfortable with a vision of their students in non-academic careers. Even in this applied field, some scholars appear to have a lingering belief that a terminal master's degree is a failed doctorate. Some even put forward arguments against any framework that would encourage non-thesis master's degrees, citing evidence that students who complete thesis-based master's programs take less time to complete their doctoral dissertations. While true in some respects, such arguments assume that the raison d'être for master's programs is to prepare students for a doctorate.

While a master's program for many students is a testing ground for their interest and ability to seek further academic training, most master's students graduate to a myriad of careers no less worthy than academe. We believe that HSPR educators should not only accept this reality – and are glad that most already do – but that the HSPR community should view success in non-academic careers as a principal metric of excellence for a master's-level training program (Conrad et al. 1993; Demers and Desai 2002). This view does not require us to “dumb down” HSPR training. Indeed, the final framework from our consultation places knowledge of illuminating theories and investigating methodologies at the core of master's-level HSPR programs. What might be encouraged, however, is greater consideration to attracting and nurturing well-rounded candidates and striving whenever possible to expose master's students to practical health system workplaces, as is done in the regional training centres (Conrad 2008; DiCenso et al. 2008). If this is done correctly, the academic stars of our HSPR community will flourish no less – and perhaps to an even greater extent – under such a paradigm.

Conclusion

The framework for graduate attributes in HSPR that has emerged from our consultation process is our best, first attempt at identifying the knowledge, skills, traits and abilities that are respectful of our field's diversity yet instrumental to the roles and responsibilities that most master's-level graduates need to be prepared for. The framework identifies common ground for all master's-level trainees while providing sufficient flexibility for training programs – and for individual students – to specialize while covering most (if not all) of that common ground. The building blocks (as shown in Figure 1) related to critical thinking, values, content knowledge and research methods are essential for success in HSPR-related careers. However, it is the skills related to the practical application of knowledge within the complex environments of health system decision-making and healthcare policy that generate value for communities from the work of HSPR graduates. The better prepared students are for this reality – through graduate training that includes increased practical exposures – the greater the value created by the HSPR field as a whole.

We hope this framework and its overarching focus will be a useful guide to educators and those responsible for the design of HSPR training programs – from admissions through curriculum, practicum and evaluation. It is certainly not an easy task to align learning opportunities with a broad set of learning objectives while dealing with the time limitations of graduate students, practical constraints of learning institutions and subject interests of faculty educators. Different programs will (indeed, should) make different choices about where to focus in the development of student knowledge, skills and abilities. An overarching framework developed through consultation and input from across the field – such as the one provided here – can serve as a compass to help keep capacity development on course. We therefore wish to emphasize that the framework presented here should be considered an evolving tool for Canada's HSPR community. As the community debates, shapes and enhances this framework – and similar ones for HSPR subspecialties – we believe such dialogue will not only improve HSPR training programs but also provide important insights about the nature and purpose of the field more generally.

Appendix: Telephone Interview Guide

Working definition: A “core competency” is an ability, skill or knowledge base that is essential for success in careers related to health services research and/or health policy.

Here are the questions we would like to discuss with you.

1. Your background: Please tell us a bit about your career path. And, if relevant, could you briefly describe your experience working with and/or hiring graduates of programs in health services and policy research.

2. Career tracks: Based on your experience and observations, how might you describe the main career tracks for health services and policy research graduates?

3. General abilities: What general abilities do you believe to be important for success in careers related to health services research and/or health policy?

4. Knowledge base: Are there areas of knowledge that you view to be important for success in careers related to health services research and/or health policy?

5. Specific skills: Are there specific skills that you view to be important for careers related to health services research and/or health policy?

6. Future competencies: Do you believe that the core competencies of health services and policy researchers will change in the next 10 years? If so, how?

7. Competency levels: Do you feel that, upon graduation, students generally possess the abilities, skills and knowledge bases necessary for careers related to health services research and/or health policy? What are the competencies that are most often lacking?

8. Unnecessary training: Do you feel that some skills taught in health services and policy research programs are unnecessary or over-emphasized? If so, which ones?

9. In-career training: Do you feel that health services and policy research graduates generally require further training, perhaps through on-the-job training or extra courses? Could universities play a role in helping to meet any of these skills?

10. Training successes: Overall, what most impresses you about graduates from health services and policy research programs in Canada?

11. First thing to change: If there was one thing that you could change about current HSPR graduate training programs, what would it be?

12. Any other comments: Do you have any other comments about core competencies that you feel would help schools across Canada build stronger HSPR graduate curricula?