Pre–post analysis of the impact of British Columbia nurse practitioner primary care clinics on patient health and care experience

Abstract

Objective

This study aims to evaluate the impact of a primary care nurse practitioner (NP)-led clinic model piloted in British Columbia (Canada) on patients’ health and care experience.

Design

The study relies on a quasi-experimental longitudinal design based on a pre-and-post survey of patients receiving care in NP-led clinics. The prerostering survey (T0) was focused on patients’ health status and care experiences preceding being rostered to the NP clinic. One year later, patients were asked to complete a similar survey (T1) focused on the care experiences with the NP clinic.

Setting

To solve recurring problems related to poor primary care accessibility, British Columbia opened four pilot NP-led clinics in 2020. Each clinic has the equivalent of approximately six full-time NPs, four other clinicians plus support staff. Clinics are located in four cities ranging from urban to suburban.

Participants

Recruitment was conducted by the clinic’s clerical staff or by their care provider. A total of 437 usable T0 surveys and 254 matched and usable T1 surveys were collected.

Primary outcome measures

The survey instrument was focused on five core dimensions of patients’ primary care experience (accessibility, continuity, comprehensiveness, responsiveness and outcomes of care) as well as on the SF-12 Short-form Health Survey.

Results

Scores for all dimensions of patients’ primary care experience increased significantly: accessibility (T0=5.9, T1=7.9, p<0.001), continuity (T0=5.5, T1=8.8, p<0.001), comprehensiveness (T0=5.6, T1=8.4, p<0.001), responsiveness (T0=7.2, T1=9.5, p<0.001), outcomes of care (T0=5.0, T1=8.3, p<0.001). SF-12 Physical health T-scores also rose significantly (T0=44.8, T1=47.6, p<0.001) but no changes we found in the mental health T scores (T0=45.8, T1=46.3 p=0.709).

Conclusions

Our results suggest that the NP-led primary care model studied here likely constitutes an effective approach to improve primary care accessibility and quality.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• This study evaluates the impact of a primary care nurse practitioner-led clinic model piloted in British Columbia (Canada) on patients’ health and care experience.

• The study rests on a pre-and-post survey without a control group therefore the differences observed could be caused by external factors.

• Data collection took place between 2020 and 2022, during the COVID-19 pandemic.

• Only four nurse-practitioner primary care clinics exist and participation in the survey was voluntary and uncompensated limiting the number of respondents.

Introduction

The lack of timely access to primary care services has been a significant problem across Canada for decades. Nearly one in seven Canadians had reported not having a regular healthcare provider in 2019¹ and, in 2020, 40% reported having an avoidable emergency department visit because of the lack of primary care options.² Available indicators, such as the proportion of the Canadian population without a primary care provider, also suggest the situation has significantly worsened since the COVID-19 pandemic.^{3 4}

Increased reliance on non-physicians such as registered nurses (RNs) and nurse practitioners (NPs) has long been documented as a way to improve primary care access.^5–10 In Canada, NPs are RNs but with a broader scope of practice including the ability to autonomously diagnose, prescribe and monitor treatments, and refer patients to medical specialists, as well as having a specialised master-level or doctorate-level training.¹¹

Several models have been used to integrate NPs into primary care delivery. The model with the highest disruption potential¹² to the current primary care landscape is called the NP-led clinic.¹³ These clinics have NPs as the central professional on an interdisciplinary team providing the full scope of primary care services to a panel of patients.^{5 14} The NP-led clinic model has consistently been found to be effective and efficient in meeting the primary care needs of a general population.8–10 15–20

Despite the strength and abundance of evidence supporting the NP-led model, its implementation in Canada has been limited and patchy. In 2020, in response to the widespread need for primary care and the successful advocacy from its NP association,²¹ British Columbia (BC), 1 of Canada’s 10 provinces, opened 4 pilot NP-led clinics. The BC model was named NP primary care clinic (NP-PCC).

The aim of this study was to evaluate the impact of NP-PCCs on patient-reported health and experience of the first cohorts of rostered patients.

Methods

Background

The pilot NP-PCCs are located in four cities ranging from core urban to suburban. Three of the NP-PCC clinics are located on Vancouver Island and one on the BC mainland. All four clinics opened in 2020 (between June and October). From the onset, the clinics were tasked with providing comprehensive primary care services to the general population from formally defined catchment areas who were without a primary care provider. The rostering goal for all clinics was set at 6800 patients, or 1000 patients per NP, after 3 years of operation by the BC Ministry of Health (MoH). Patients are formally rostered to the clinic as well as informally attached to a specific NP within the clinic. The scope of practice of primary care NPs in BC is large and allows them to autonomously provide the full spectrum of primary care services to all ages from birth to death. Medical specialists are consulted when a patient’s healthcare needs to fall outside of the NP scope of practice. Due to very high demand, all four clinics used patient waitlists or random draws as a part of their attachment processes. The clinics are funded to have the equivalent of six full-time primary care NPs, one social worker, one mental health counsellor or clinician, two RNs, one clinic manager and four medical office assistants at the end of the 3 years implementation period. Clinics are funded by the BC MoH and have a self-appointed board of directors and managerial autonomy.²²

Study design

The study relies on a quasi-experimental, longitudinal design based on a pre-and-post survey of patients receiving care in one of the four NP-PCCs. The prerostering survey (time 0 or T0) asked respondents to describe their care experience during the 2 years preceding being rostered to an NP-PCC as well as their current health. Approximately 1 year later, patients were asked to complete a second (time 1 or T1) similar survey, this time focused on their care experience as an NP-PCC patient. The design was adapted from previous work done by the same team.^{14 23 24}

Patient and public involvement

The study is focused on patients’ health and care experience as reported by patients. The instruments used were developed in previous projects and no patient or public involvement took place this time. Results will be disseminated to patients through the clinics.

Participant recruitment

For the T0 recruitment, all patients newly rostered to one of the clinics were given an information sheet about the project by the clinic’s clerical staff or by their care provider. The information sheet included a link to a consent form and online survey instrument hosted on the University of Victoria Survey Monkey platform. The survey asked patients to provide their government-issued personal health number (PHN). The PHNs of patients who completed the T0 survey were shared with the clinics’ clerical staff who made a note in the clinics’ electronic health record system. About a year later during a subsequent visit to the clinic, those same patients were provided with an invitation to complete the T1 survey. Email invitations were also sent by the clinics to patients identified as having completed the T0 survey. Data collection periods varied slightly from site to site (see online supplemental appendix figure 1) but, overall, the T0 period ran from November 2020 to January 2022 and the T1 period ran from December 2021 to December 2022.

Instruments

The patient reported experience from being rostered in an NP-PCC were measured according to five core dimensions of primary care experience: accessibility, continuity, comprehensiveness, responsiveness and outcomes of care. Those dimensions have been extensively discussed and conceptualised in the literature.^25–27 The definition of accessibility rests on Donabedian’s seminal work,²⁸ as the fit between the structures of production, on one hand, and society’s needs and their geographic distribution, on the other. It is operationalised here as the patient’s perception that they can access healthcare services without undue constraints. Continuity is defined, based on the work of Haggerty et al,²⁹ as the fact that a patient is treated by the same professional or the same team over time (relational continuity) and that different services are harmoniously integrated with each other (management continuity). Comprehensiveness encompasses two dimensions that make up the scope of patient management: considering all of a patient’s needs and providing a complete basket of services.^{14 30} Responsiveness is defined here as the convergence between the patients’ expectations regarding non-technical elements of the care and what the clinic offers in practice.²⁷ Finally, what is described as outcomes of care relates to the patients’ perception that the care received had a positive impact on their health.³¹

The five dimensions described above were measured using a survey instrument adapted from the one developed by Pineault et al^{31 32} which was in turn an expanded version of two well-validated pre-existing instruments: the Primary Care Assessment Survey³³ and the Primary Care Assessment Tool.³⁴ Details on the phrasing of the survey questions and computation of scores can be found elsewhere.^{24 32} Each score is measured on a scale ranging from 0 to 10. For each score, missing values were processed according to the rules established for that instrument.^{24 32} In addition to the patient-focused indicators described above, we also measured respondents’ general health status using the SF-12 Short-form Health Survey.³⁵ The SF-12 instrument provides a physical health score and a mental health score, both measured on a scale ranging from 0 to 100 and calibrated at 50 for the average US adult population. The surveys relied on a branching logic (the questions a given respondent will see depend on their previous answers) but overall, the T0 survey included 89 questions and the T1 survey 59 questions.

Sample and procedures

At the end of the data collection period, patients’ PHNs were used to individually match T0 and T1 survey answers. Survey answers that could not be matched (missing or divergent PHN) were removed from the database. In the same way, missing data either at T0 or T1 for any given score led to the removal of those responses in the computation of mean scores and differences. Scores were computed in Microsoft Excel.

There were 437 usable T0 surveys and 254 matched and usable T1 surveys (see figure 1). The representativeness of the survey respondents as compared with the population of the clinics’ catchment areas as well as to the clinics’ rosters was analysed at the end of T0 data collection and results have been published elsewhere.³⁶ For all sociodemographic variables tested, the clinics’ patients were similar to the population of the catchment areas except for an over-representation of children at the two clinics with an NP whose practice was focused on paediatric care. The comparison of the clinics’ patients with T0 respondents showed no major differences. At the end of the data collection period, we also used descriptive and inferential statistics on sociodemographic information to assess the risk of attrition bias. For categorical data, we used χ² tests based on the actual T1 number of respondents per category and expected T1 numbers based on T0 proportions. Age was assessed both as a categorical variable based on 10-year age groups and by a t-test of the mean. Those tests were done in MS Excel.

Figure 1.

Recruitment periods, number of participants and completion rates per sites.

Primary outcomes were analysed using paired t-tests of the mean (T1–T0) on each of the five scores related to the core dimensions of patients’ primary care experiences as well as for the mental and physical health scores. Computations were done in IBM SPSS (V.26).

Results

The overall response rate of the T1 survey from T0 respondents was 58% (individual clinics rate varied from 44% to 72%). The χ² tests conducted showed no significant indication of an attrition bias in the core sociodemographic variables captured by the survey (see online supplemental appendix table 1)

The main finding from the longitudinal analysis of patients’ experience of care is that there was a large increase in the scores for all dimensions measured (see figure 2). Patient perceptions of accessibility, continuity, comprehensiveness, responsiveness and that the care received made a difference on their health were significantly higher compared with the period preceding their rostering in the clinic. Probabilities from the paired t-test of the mean are below 0.001 for all five dimensions (see online supplemental appendix table 2).

Figure 2.

Patient experience of care scores (error bars=95% CI).

There was some intersite variability in prescores and postscores but the overall trend of significant improvements remains (see figure 3). All dimensions of patients’ primary care experiences vastly improved at all four sites. The pre–post differences for individual clinics are again significant (p<0.01) for all dimensions (see online supplemental appendix table 2).

Figure 3.

Patient experience of care scores per clinic (error bars=95% CI).

The composite experience of care scores also aligns with individual survey questions focused on similar constructs. For example, the proportion of respondents reporting unmet needs (based on the question, ‘In the last year, did you feel the need to see a doctor or another health professional for a health problem, but didn’t see one’) sharply decreased between T0 and T1. The overall proportion of patients reporting unmet needs at T0 was 60% (ranging from 45% to 67%). At T1 that proportion was down to 19% (ranging from 4% to 33%). As the question is not specific to primary care, challenges in access to specialist physicians or other services in each area should be considered in the interpretation of those scores.

Regarding respondents’ health condition as measured by the SF-12 instruments, our results show an overall improvement in the physical health scores (rising from 44.8 at T0 to 47.6 at T1 p<0.001) and no change in the mental health score (from 45.8 at T0 to 46.3 at T1 p=0.709) (see figure 4 and online supplemental appendix table 3).

Figure 4.

Differences in SF-12 scores (T1–T0) error bars=95% CI. SF-12, Short-form Health Survey.

When analysed on a site-by-site basis, the trend of a significant improvement in the SF-12 physical health score persists. The data show a significant difference (at alpha 0.05) in pre-and-post physical health scores at three of the four clinics. Given the low number of responses (df ranging from 32 to 69), this trend is interesting.

When the matched T1–T0 differences in physical health scores are plotted in relation to the T1–T0 differences in mental health scores, it creates a scatter plot with four quadrants. The upper right quadrant contains respondents who reported both improved physical and mental health. The lower left quadrant contains respondents who reported both decreased physical and mental health. The two other quadrants correspond to people reporting improvements in one area and a decrease in the other. Overall, the figure shows that the improvement in the average physical health score is a product of both more people on the right on the graph (62% of respondents reported an improvement in their physical health) and the fact their average score difference is higher in absolute value (+7.6) as compared with respondents with a negative physical health score difference (−5.1). No such pattern exists for the mental health score differences (top vs bottom of figure 5)

Figure 5.

T1–T0 differences in SF-12 scores scatterplot. SF-12, Short-form Health Survey.

Discussion

The main finding from the longitudinal analysis of patients’ reported health and experience of care is that there was a large increase in the scores for all dimensions measured after rostering in an NP-PCC. The improvements observed in patients’ reported health and experiences regarding primary care can be in part explained by the low prerostering (T0) baseline. Other studies that relied on the same instruments^{24 31} measured higher baseline or control-arm scores. However, the postrostering scores (T1) measured here are similar to what was found for well-established medical clinics in those studies. Both the magnitude of the improvements measured here and the comparison of scores from the NP-PCC model as compared with the ones of large well-established interdisciplinary clinics found in other studies²⁴ suggest that NP-PCCs are an effective model of primary care delivery.

Given the short duration of the longitudinal follow-up conducted here, we were surprised to observe a significant improvement in the physical health as measured by the SF-12 instrument. Informal discussions with NPs working at one of the clinics suggest that the NP-PCCs rostered a large proportion of patients who had been left with unmet need for long periods of time. Our hypothesis is that providing accessible and comprehensive care to those patients might have been enough to cause the observed improvements in physical health. It could also be that, given recovery from mental health condition is usually longer, our study period was too short to catch improvements at that level.

More generally, it might be stressed that the NP-PCC model was, from the onset, based on the assumption those clinics would focus on rostering patients who do not have a regular provider. The survey data confirm most patients did fit that profile and that they faced low accessibility to care as well as generally low quality of care before joining the NP-PCC.³⁶ The proportion of survey respondents reporting unmet needs at T0 (60%) is many times higher than what is observed in other populations. In 2022, 7.7% of people in BC and 7.2% in Canada declared they did not get care when they had needs³⁷ while the European Union average for medical care was at 4.8%.³⁸

Limitations

This study has some limitations. First, it rests on a quasi-experimental design based on a pre-and-post survey without a control group. As such there is a risk that external factors explain the differences observed here. The most notable external factor is that the data collection took place between 2020 and 2022, during the COVID-19 pandemic. To control for some biases, the prerostering survey questions were edited to ask respondents about the care they received during the ‘last 2 years’ therefore including one prepandemic year. This phrasing is likely to have overinflated T0 scores as compared with what they would have been if those patients had to seek care outside of the NP-PCCs at that time. It might also be worth mentioning that all available indicators suggest that the primary care delivery capacity in BC sharply declined throughout the data collection period.³⁹ Both the phrasing of the T0 survey and the context during data collection are likely to have had a negative impact on T1–T0 score differences making the magnitude of the differences observed here even more interesting.

Second, privacy and confidentiality considerations prevented the matching of T0 respondents with clinic’s patient rosters and made it impossible to compute a response rate for the ‘pre’ component of the survey. A descriptive and comparative analysis of T0 respondents published elsewhere³⁶ suggest they were generally representative of the total clinic roster population. However, a sampling bias cannot be excluded given the methods used here. The survey instrument was only available in English which could cause a bias. However, the proportion of respondents who declared speaking another language at home or being born abroad was similar to the general population of the catchment area. We want to mention that the limited number of respondents is explained mostly by the fact only four NP-PCC clinics exist and that participation in the survey was voluntary and uncompensated.

Conclusion

Our findings show that offering people who were relying on low continuity, low comprehensiveness services, such as medical walk-in clinics, access to an NP-led alternative dramatically improves their care experience. All five dimensions of care measured are positively affected, and our results suggest that being followed in an NP-PCC could lead to gains in patients’ overall physical health. The patients’ primary care experience scores measured in our study favourably compare to the ones measured in well-established interdisciplinary and medical clinics.²⁴

The international literature on NPs autonomously working as main providers of primary care suggests that it is a viable model.¹⁰ As such, our findings are well aligned with the large body of evidence showing that NP-led primary care is equivalent or better than average available medical care.8 10 15 40 By showing interesting outcomes of the NP-PCC model in BC, this study also suggests that NP-led models work in many different jurisdictions and contexts.

At a more local level, the results analysed here also suggest that BC’s NP-PCC model does produce the results that were hoped for when the model was launched²² and constitutes an effective approach to improve primary care accessibility and quality. More research on the efficiency of such models could play a role in future policy decisions to scale them up.

Data availability statement

No data are available. The nature of the data and conditions set by the University of Victoria Human Research Ethics Board prevent the sharing of the raw data.

Ethics statements

Patient consent for publication

Consent obtained directly from patient(s).

Ethics approval

This study involves human participants and the study was approved by the University of Victoria Human Research Ethics Board (#20-0324). Participants gave informed consent to participate in the study before taking part.