Inadequate Performance Measures Affecting Practices, Organizations and Outcomes of Ontario's Family Health Teams

Abstract

Background: Emphasis on quantity as the main performance measure may be posing challenges for Family Health Team (FHT) practices and organizational structures. This study asked: What healthcare practices and organizational structures are encouraged by the FHT model?

Methods: An exploratory qualitative design guided by discourse analysis was used. This paper presents findings from in-depth semi-structured interviews conducted with seven policy informants and 29 FHT leaders.

Results: Participants report that performance measures value quantity and are not inclusive of the broad scope of attributes that comprise primary healthcare. Performance measures do not appear to be accurately capturing the demand for healthcare services, or the actual amount of services being provided by FHTs. Results suggest that unintended consequences of performance measures may be posing challenges to access and health outcomes.

Conclusion: It is recommended that performance measures be developed and used to measure, support and encourage FHTs to achieve the goals of PHC.

Family Health Teams (FHTs) are one model of primary healthcare (PHC) in Ontario, implemented in 2005. FHTs aim to improve access to comprehensive and collaborative patient-centred care (Aggarwal 2009; MOHLTC 2004, 2010). This article presents findings from a study that asked: What healthcare practices and organizational structures are encouraged by the FHT model?

Patient enrolment, or “rostering,” is a core component of the FHT model (Fleming n.d.; MOHLTC 2005) and refers to a process in which patients register with an organization, team or provider (Aggarwal 2011; MOHLTC 2005). Patient enrolment is used to help determine funding and compensation in FHTs (Fleming n.d.; Health Force Ontario 2014), and aids accountability, quality improvement and performance measurement (Hutchinson 2008).

Performance measures are tools that influence healthcare practices, priorities, organizational structures, processes and health outcomes (Campbell et al. 2003; Félix-Bortolotti 2009; Haggerty et al. 2007; Sandy et al. 2009). Performance measures refer to parameters whereby services and programs are measured (Aggarwal 2011). Research on performance measures from a Canadian PHC context is relatively new (Johnston et al. 2008).

Methods

An exploratory qualitative design was used, guided by Gee's (2011) approach to discourse analysis that sees meaning emerging from a combination of saying (informing), doing (action) and being (identity). One of the reasons why this approach to discourse analysis was selected for this study is because it parallels Donabedian's (1966) influential framework (Table 1).

TABLE 1.

Discourse analysis: saying, doing, being

Discourse	Discourse definition	Study focus
Saying (informing)	An oral or written communication; utterance	Identified aim
Doing (action)	Socially situated activity or practice	Healthcare practices
Being (identity)	Socially situated identity	Organizational structures

Semi-structured interviews were conducted with seven key policy informants (PIs) and 29 FHT leaders. PIs were Ministry of Health and Long Term Care (MOHLTC) policy makers and decision-makers, or consultants who helped develop the FHT policy. FHT leaders were those active in leadership positions, namely, 10 physicians, 15 executive directors and four clinical leaders. Purposive and snowball sampling were used to identify PIs. Stratified purposive sampling (Miles and Huberman 1994) was used to guide sampling of FHT leaders around geography and the wave (or year) that FHT application was approved.

Snowball sampling informed the sample because four FHT leaders initiated suggestion of additional participants. Interviews were conducted using a semi-structured interview guide and occurred in-person or by telephone. Modification was made to the interview guide as interviews progressed so that emerging topics were explored (Miles and Huberman 1994) (Table 2).

TABLE 2.

Interview guide

Policy informant interview guide

Tell me about your role in shaping the Ontario Family Health Team policy. What do you consider the key policies guiding the Ontario Family Health Teams? What helped to guide you in the development of the Ontario Family Health Team policy? What were/are the goals of the Ontario Family Health policy? What would you say are the greatest strengths of the Ontario Family Health policy? What would you say are the limitations of the Ontario Family Health policy?

FHT leaders

Tell me about the background of this Family Health Team. In what ways has provincial policy assisted your Family Health Team? In what ways has provincial policy been challenging to your Family Health Team? What does this Family Health Team hope to accomplish?

Individual interviews were conducted with seven PIs and 20 FHT leader interviews; also, four interviews with FHT leaders were conducted as group interviews (one group of three participants, three groups of two participants) at request of the participants involved in those four small groups. All interviews were audio-recorded and transcribed (see Table 3 at www.longwoods.com/content/23929).

Constant comparison was used to capture major themes and break the data into manageable parts (Cresswell 2007). Gee's (2011) discourse analysis framework then guided the second phase of analysis. Each participant has been assigned a code that will be used in the presentation of data in the form of quotations: PIs have been assigned the code “PI” followed by a number. FHT leaders have been assigned the code “L” followed by a number.

Results

This paper presents findings that demonstrate that inadequate performance measures are being used to evaluate the success of FHTs. Findings suggest that performance measures largely informed by patient enrolment numbers present challenges to healthcare practices, FHT organizations and health outcomes.

Saying: valuing quantity

According to the data, the primary measure used to determine the success of FHTs is the number of individual patients enrolled with FHT physicians: “The increased emphasis on access and unattached patients as we call them, has meant that we've become more and more disciplined and expect more discipline from the [FHTs] in terms of meeting enrolment targets” (PI4). FHT leaders described an emphasis on using quantity as a key measure of success: “There are formal outcomes that the Ministry's looking for. Those are really around visit numbers and roster numbers” (L4). Another FHT leader states, “What we're told is we're measuring [FHT] success by … how many people we've rostered” (L23).

Participants described challenges related to this emphasis on quantity: “One of the biggest challenges is … where we all want to go and what the Ministry's currently right now asking us to submit – and that's just volume-driven-type quantity. We want quality, they want quantity” (L8). Another FHT leader stated, “There's got to be a better way that we can measure our success” (L23.1). These data suggest that other attributes of PHC that also shape FHTs are neglected: “All of the other things which make up the value proposition of primary care toits population that it serves are absent; all we have is this microscope on the wrong numbers” (L10.1). Another FHT leader voices concern with measures being used to determine success: “The way that they measure success is not necessarily how I would measure success. They measure success by how many rostered patients have you enrolled …. For us, we would rather look more at patient outcomes …. We're more interested in things that actually make a difference in the practice” (L8). This FHT leader went on to state, “I think that's where we'll havea bit of a disconnect between what is success. I really truly don't believe what they're measuring is success at all” (L8). The following FHT leader demonstrates how there is incongruity between measures being used to determine FHT success and quality of healthcare services:

On paper, they can show that lots more people can say that they have a healthcare provider today than they did before the Family Health Team. But, I think a lot of people would say and still say that they have difficulty accessing their primary care provider … that goal has not been met and part of the reason is the … drawbacks to the rostering system … if you have any sort of business sense you soon realize that I don't have to be in my office … but still have this income stream coming in …. I don't know if physicians are spending as much time in the office as they were beforehand when they were fee-for-service. I would say that there is incentives to sign people up and roster them. (L20)

Doing: volume influencing practice

Based on the data analysis, inadequate measures are influencing healthcare practices in two ways. First, specific healthcare practices are being implemented in response to demands of patient volume. Second, healthcare practices are encouraged to include programs that promote greater numbers of encounters. Although patients are enrolled to individual physicians, the emphasis on patient volume affects practices of other healthcare providers within the FHT. For example, one PI described how patient volume has influenced the practice of dieticians: “Dieticians have traditionally provided one-on-one counseling and they are finding that the volume of patients needing their care has been so overwhelming that they're starting to do group work” (PI2). Although there may be benefits to group work, the PI states that group work is a way to keep up with demands.

Another FHT leader described challenges of emphasizing success based on quantity of encounters in relation to a particular disease:

When you're told to report your number of transactions by chronic disease, by allied health professional, essentially what that is saying is take these allied health professionals and have them do programs. And if you want to spend a lot of money, that's how you do it because then every diabetic becomes a client of the program …. You can go endlessly to all the sort of programmatic transaction based things that programs can do. But what you end up doing is hardly impacting at all on the actual patient-oriented outcomes that are important to the population. (L10.1)

The FHT leader continued on by stating that these measures of success are not reflective of the experiences of physicians and makes reference to an uproar on a physician's listserv:

There was a firestorm and it was a backlash reaction to what they were using as indicators. There was a sense that they didn't really reflect what practitioners understood and sensed was the reason why they were practicing primary care … I rarely ever see a diabetic where it's an access or clinical challenge in my practice. Those that don't achieve outcomes usually don't for particular reasons. (L10.1)

FHT leaders also described how inadequate performance measures detract from person-centred care. For example, one FHT leader stated: “Patients just don't present as one problem …. Patients aren't a disease” (L12.1). Another FHT leader agreed: “How do we manage patients as patients, not as disease-specific?” (L21). Participants raised concerns that current performance measures are inadequate for supporting person-centred care. One FHT leader indicated that even multiple diseases pose a challenge for these measures: “How do we care for patients with multiple co-morbidities as opposed to just focusing on one disease, specific disease?” (L21). One PI recognized that current measures are presenting challenges: “Person-centred care diminished with the accountability framework” (PI2). Another PI agreed:

Working right now … is voluminous and it's about just ploughing through the numbers, which makes it hard to look up and out. There's a tendency to be looking down … it's about two things – managing within … budgets and having good accountability …. I would argue that client care gets talked about secondly, thirdly. (PI7)

This PI continued to describe how volume detracts from person-centred care because of the resulting time and care demands on physicians: “Docs don't have time within whatever their funding model is … and with the kind of patient loads that they have to address” (PI7). An additional concern identified in the data is the influence of inadequate measures on the organizational structure of FHTs.

Being: organizational impact

Participants expressed concerns about the organizational impacts of inadequate performance measures. One concern is in the allotment of funding for interdisciplinary health providers. According to one FHT leader, the MOHLTC's method of assessing success does not adequately reflect the amount of care actually being provided in FHTs, which may put funding at risk. In this case, the FHT leader described how patient enrolment is not an accurate assessment because some patients choose not to enrol:

It has an implication for policy, because those people will still use our services, we don't prevent them from doing so, but their numbers are not included in the count that the Ministry uses to judge our success. So, if those people generate visits for our dietician and yet the Ministry would say, “Oh well, you don't have a very big roster. Your roster's not large enough to support a full-time dietician. We're going to cut your funding.” (L4)

According to the above FHT leader, the measure being used to determine success is not accurately representing the real demands of healthcare and may impact funding for interdisciplinary health providers. Participants described how the patient enrolment is not an accurate representation of healthcare services, particularly in Northern communities. According to FHT leaders, patient enrolment is not working for First Nations' people. One FHT leader stated, “Aboriginal people don't want to roster, as a generalization, not all” (L2). Another FHT leader agreed, “The First Nations' component is difficult … they don't like to roster” (L3). When asked the reason why patient enrolment is not working for First Nations' people, a FHT leader stated, “It's a trust issue, that's part of it. I think it's a trust issue. I think it's a cultural piece” (L2). Another FHT leader was unsure of the reasons why First Nations' people were more adverse to patient enrolment, “Many of them are not rostered and they will not roster and I'm not sure why. So, they just sort of come when they need to come and I think they are a little hesitant about rostering” (L11). One FHT leader further expands:

The First Nations' component is difficult. First of all, they don't like to roster. Secondly, they go for care sort of traditionally all over … part of the problem with that is that if you are rostered patients and you seek care elsewhere, your physicianis financially penalized …. So, physicians have been reluctant to roster First Nations' people. (L3)

Participants indicated that healthcare services are being provided to patients who are not enrolled. Interdisciplinary healthcare provider funding based on an inaccurate measure of healthcare demands may impact some FHTs.

Health outcomes: volume emphasis encouraging acuity

An additional concern raised by participants is that some practices may be more focused on meeting the expectations of numbers and less about quality care. For example, using patient enrolment numbers as a key measure of success only provides an assessment of effort regarding quantity of individuals who are enrolled; evaluation of patient experiences, adequacy of healthcare practices and health outcomes are excluded. This concern is elaborated by the following FHT leader: “I think that there are practices providing poor services and getting paid a lot of money because they rostered the world and don't service them” (L10.1).

Inadequate measures may even result with individuals who require care being excluded from FHTs. According to the following FHT leader, increasing patient volumes means being selective about which individuals are accepted as patients: “If we wanted to manipulate the system, if it's all about roster, then we would cherry pick which patients we got because then we'd want single males … healthy … with no issues, because then our physicians could take on 2,000 patients each” (L21). According to this FHT leader, single healthy males require the least amount of healthcare services. This FHT leader suggests that healthy individuals requiring fewer healthcare services are desirable for patient enrolment; less demands for services means that more individuals can be enrolled, thus increasing patient volume. Individuals with simple health issues do not require extensive services because their care needs are minimal. According to the following FHT leader, exclusion of those most requiring healthcare services may lead to acute issues:

What that means in medicine … is oftentimes you get people in the worst case of the course of whatever illness or issue that they have, because they've waited until the last minute in that they don't have actual access for preventative or earlier intervention or use of services that would have actually mitigated the issue in the first place. So we end up becoming quite responsive on the acute end. (L23.2)

This participant describes how achieving measures of success at time of this study is ineffective in the promotion of healthy outcomes.

Discussion

Participants described how using patient enrolment and contact numbers as a main performance measure for FHTs provides limited information and that the broad scope of attributes that inform quality PHC is absent. The emphasis on quantity as a measure of FHT success is presenting challenges to those who strive to meet the intended goals of FHTs. Quantification of results can provide useful information such as the numbers of individuals who use a particular program. This study suggests that there is an absence of additional information on quality and, as a result, care of the individual has become lost in performance measures (Johnston et al. 2008).

A challenge of performance measures, like described by participants in this study, is thata focus on achieving measureable targets may pose challenges to the quality of patient care (Johnston et al. 2008). Healthcare practices are being implemented in response to volume that is emerging as a result of the measures rather than best practices. Not only is the emphasis on quantity not encouraging congruency of FHTs with PHC attributes, the measure of quantity of patients rostered does not always result in accurate numbers. Patient enrolment numbers refer to those patients enrolled by a physician, but tell nothing of the quality of service being provided by the physician and exclude the other health professionals who are part of thecare team. Given the broad scope and relational nature of PHC, quantifying outcomes only provides part of the picture. As well, the emphasis on quantity as an evaluation tool poses challenges for social workers and other allied health professionals in FHTs to demonstrate their effectiveness and the extent of their contributions. With FHTs being a newer modelof PHC, and performance measures suited for a Canadian PHC context still in infancy (Johnston et al. 2008), there is a risk of unintended consequences occurring, such as the ones described in this study (Powell et al. 2011).

Powell et al. (2011) describe how unintended consequences of performance measures can influence patient-centred care, as well as provider morale. Participants in this study described some healthcare practices driven by a need to increase numbers, and the need to keep up with the volume of patients, rather than measures of the effectiveness of interventions. This further heightens a concern for interdisciplinary healthcare providers. Despite increased access to various healthcare professionals in FHTs, it remains concerning that healthcare services provided by the interdisciplinary providers like dietary and social work might be influenced and shaped in response to the demand of volume. This may not be the most effective approach if the aim is to impact health outcomes.

Although there is no evidence in the data, the emphasis on inadequate measures involving number of rostered patients and number of patient encounters makes it worthwhile to hypothesize one additional impact: encouraging inclusion of interdisciplinary health professionals whose practice results in increased numbers. Such allied health professionals may be considered more valuable when determining which healthcare professionals to hire, particularly for FHT leaders who may not have extensive knowledge about the roles of the various health professionals. If performance measures emphasize quantity, then the interdisciplinary health professionals who would be most desirable additions to the FHT team are those who would increase number of patients and number of patient interactions. An alternative view might be to include health professionals based on ability to impact health outcomes. Inadequate performance measures that focus solely on patient volumes can impact healthcare practices in a way that detracts from timely attention to healthcare needs.

A significant concern arising from the use of inadequate performance measures is the resulting encouragement of acuity in health outcomes. Current performance measures are inadequate because they encourage the exclusion of individuals who may require higher demands of healthcare services. Kalucy and colleagues (2009) warn that patient enrolment may decrease equity if providers are encouraged not to enrol high-need patients.

It must be acknowledged that measuring outcomes in an FHT setting is very complex because of varying factors – no two FHTs are alike. For example, differences exist between FHTs in terms of composition, both in number of providers and the type of health professionals involved; diverse patient populations; differing local resources; and even geographical variations. Given the diversity between FHTs, it is understandable that enrolment numbers provide data that are timely and consistent (Aggarwal 2011) across all FHTs. Some aspects of PHC are easier and less costly to measure than others. For example, patient enrolment numbers are easier to measure than other qualities such as trust, patient-centred care and cultural sensitivity (Johnston et al. 2008). Studies have shown that patient enrolment in PHC results with lower health system costs (Kralj and Kantarevic 2012). Further, a group of experts advocate for Ontario to continue implementation of patient enrolment models (Aggarwal 2011). There is value to patient enrolment. However, findings of this study suggest that the emphasis on patient enrolment with exclusion of other attributes of PHC is problematic.

Participants in this study were asked to comment on ways that provincial policy has challenged their FHT. Results presented in this article highlight the challenges that participants described. A limitation of this study is that participants were not probed for their opinions on alternatives to these existing challenges. However, this is an area of priority for stakeholders across Canada who aim to meet the needs of policy makers, PHC providers and organizations (CIHI and HQO 2013).

Conclusion

FHTs are experiencing challenges from inadequate performance measures emphasizing volume over other attributes and goals that FHTs are working towards, such as quality person-centred care. The demand for health services created by the volume of patients and patient encounters has presented challenges for physicians and interprofessional healthcare providers. Performance measures do not appear to be accurately capturing the demand for healthcare services, or the actual amount of services being provided by FHTs. Although a large number of patients have enrolled with FHT physicians, some patients choose not to enrol and continue to receive healthcare services from FHTs. The non-enrolled patients receiving healthcare services from FHTs may not be recognized in the formula used to determine funding for interdisciplinary providers. It is recommended that performance measures be developed and used to measure, support and encourage FHTs to achieve the goals of PHC. It is recommended that performance measures be developed for FHTs' complex and diverse environment. Based on these findings, FHTs that are successful in the implementation of quality person-centred care appear to be doing so despite the emphasis on quantity over quality.