Comparing the Attainment of the Patient's Medical Home Model across Regions in Three Canadian Provinces: A Cross-Sectional Study

Abstract

Background:

The aim of this work was to show the feasibility of providing a comprehensive portrait of regional primary care performance.

Methods:

The TRANSFORMATION study used a mixed-methods concurrent study design where we analyzed survey data and case studies. Data were collected in British Columbia, Ontario and Nova Scotia. Patient's Medical Home (PMH) pillar scores were created by calculating mean clinic-level scores across regions. Scores and qualitative themes were compared.

Results:

Participation included 86 practices (n = 1,929 patients; n = 117 clinicians). Regions had differential attainment towards PMH orientation with respect to infrastructure; community adaptiveness and accountability; and patient and family partnered care. The lowest PMH attainment for all regions were observed in connected care; accessible care; measurement, continuous quality improvement and research; and training, education and continuing professional development.

Conclusions:

Comprehensive performance reporting that draws on multiple data sources in primary care is possible. Regional portraits highlighting many of the key pillars of a PMH approach to primary care show that despite differences in policy contexts, achieving a PMH remains elusive.

Introduction

A primary healthcare (PHC)–oriented health system is the most cost-effective and equitable way for health systems to address population health outcomes (WHO 2018). Strong primary care service delivery – a core part of a PHC-oriented system – is the backbone of a high-performing system. Since the early 2000s, primary care reform has been high on the health policy agenda across Canada (Aggarwal and Hutchison 2012). Yet there remains a lack of a coordinated and comprehensive approach to collect, analyze and report data on the performance of primary care at either the individual practice level or the regional/system level (Haj-Ali and Hutchison 2017). Key opportunities at the practice or regional level that can identify areas for improvement or track the effect of improvement initiatives are being missed. For provincial policy makers, whose responsibility is to ensure access to high-quality primary care services for its residents, the paucity of information available on this sector's performance makes it challenging to monitor and evaluate the effectiveness of policy changes and investments (Haj-Ali and Hutchison 2017).

Across most high-income countries, there is consensus that independent, impartial assessment of performance is an essential part of quality improvement (Levesque and Sutherland 2020). Public reporting can be used as a lever for change, although the potential for negative, unintended consequences such as gaming (Campanella et al. 2016) or a preoccupation with a small number of published, often easily measurable, indicators always exists (Levesque and Sutherland 2020). However, public reporting of health system performance can promote accountability, highlight variation, identify areas for improvement and be used to support change. Regional case studies of performance reporting (Smith et al. 2012; Young 2012) and evidence from the hospital sector (Tu et al. 2009) indicate that public reporting can influence decision makers' and clinicians' quality-improvement agendas. Public reporting may improve performance (Faber et al. 2009; Hibbard et al. 2012; Smith et al. 2012; The Commonwealth Fund 2011) because it has the potential to “facilitate public participation and increase accountability in healthcare” (Ellins and McIver 2009: 41), impact societal and professional values and direct attention to issues not currently on the policy agenda (Oxman et al. 2009). It may also facilitate collaboration among stakeholders with a common agenda (van Walraven et al. 2010).

There remains limited comparative data available on primary care performance in Canada. What is available is mostly at a provincial level that uses health administrative data from the International Health Policy Surveys (The Commonwealth Fund 2011) or aggregated data from the Canadian Medical Association (2021). Health administrative data can contribute to performance measurement of primary care in a relatively inexpensive way. But only using these data cannot adequately address core primary care dimensions such as health promotion, interaction with social sectors or communication (Green et al. 2012; Hutchison et al. 2020). Hutchison et al (2020) suggested that data are currently available only for 13% of practice-based measures and 41% of system-level measures as identified by the Primary Care Performance Measurement Framework for Ontario. Another limitation is that health administrative data usually include only some activities by billing physicians and routinely omit contributions of other healthcare team members.

Clinicians, healthcare quality-improvement organizations and insurers increasingly recognize the need for systematic, ongoing feedback on primary care performance (Roberts et al. 2014). Clinicians prefer to reflect on their own performance data but also have comparative data (at the individual practice and/or the regional/provincial level) available (Robert Wood Johnson Foundation 2013). Reporting needs to be accurately, fairly and meaningfully measured and must be comprehensive, systematic and rigorous (Langton et al. 2016; Levesque and Sutherland 2020), especially given clinicians' particular sensitivity to comparative data and strong debates that can occur in the media about data.

The objective of this study is to measure and report on the science of comprehensive performance measures, taking advantage of multiple sources of data in primary care across regions in three Canadian provinces. This study is timely because Canadian practice-based (Hogg et al. 2008) and system-level (Watson et al. 2009) performance measurement frameworks provide guidance on what is needed to measure primary care performance. It is important to regularly include measures of primary care performance to inform policy development, service planning, management and quality improvement (Hutchison et al. 2020).

Method

Design

The TRANSFORMATION study was a mixed-methods concurrent study design where quantitative analysis was supplemented with qualitative case studies. The multi-method data collection strategy was carried out in three Canadian geographic regions. The research team consisted of researchers, decision makers, clinicians, patients and regional and international advisory committees. We conducted surveys in English and French in primary care practices, collecting organization-, provider- and patient-level data. Case studies and deliberative dialogues (day-long discussions) were conducted in each region. For the purposes of this work, we used the practice-based surveys and case studies. Canadian practice-based and system-level performance measurement frameworks provide guidance on what is needed to measure primary care performance. Together, the surveys and case studies cover many of the dimensions of primary care needed to measure this sector's performance.

Sample

The sample consisted of primary care clinicians responsible for their own patient panel working in Fraser East, BC; Eastern Ontario Health Unit, ON; and Central Zone, NS. Survey and case study data were collected between 2014 and 2016. There were a total of 86 practices (n = 1,929 patients; n = 117 clinicians) that participated (Table 2).

Table 2.

Practice-based surveys undertaken in each region and the number completed

Region	Patient (n)		Clinician (n)		Organizational (n)		TCI (n)
	Surveyed	Complete/partially complete	Surveyed	Complete/partially complete	Surveyed	Complete/partially complete
Fraser East, BC	506	504	35	34	22	22	20
Eastern Ontario, ON	548	547	37	36	26	26	21
Central Zone, NS	878	878	47	47	39	38	22
Total	1,932	1,929	119	117	87	86	63

Here n represents whether a TCI score could be calculated at the practice level.

Survey development

Three surveys (patient-, provider- and organization-based) were developed and administered as part of practice-based data collection. All practice staff (e.g., office manager, nurses, nurse practitioners, etc.) also completed the Team Climate Inventory (TCI), a validated tool to measure team functioning (Anderson and West 1998; Beaulieu et al. 2013).

The TRANSFORMATION patient survey was built on the foundation of the Canadian Patient Experiences Survey on Inpatient Care by the Canadian Institute for Health Information (CIHI) (CIHI 2013) and was developed by Wong and Haggerty (Wong and Haggerty 2013). The TRANSFORMATION study used questions from the CIHI patient survey and previously validated and publicly available items and scales (Campbell et al. 2009; Care Quality Commission n.d.; Ford-Gilboe et al. 2020; Fortin et al. 2017; Haggerty and Levesque 2017; Herdman et al. 2011; Hibbard et al. 2004; Lorig et al. 2001; Stewart et al. 2007). We identified key questions from these sources after consulting with Canadian and international primary care experts (Campbell et al. 2009; Kringos et al. 2013) to obtain a fuller picture of patient experience in this sector. The resulting questionnaire (n = 116 questions) captured patient-reported experiences including access, relationship-based care, health promotion, self-management support, coordination orientation, safe healthcare system contribution and equity orientation.

The TRANSFORMATION study's organizational and provider surveys were also built on the foundation of previous work (Levesque et al. 2010) and the team's previous contribution to the development of CIHI PHC surveys (CIHI 2013). The organizational survey had six sections (n = 77 questions): general practice information; organizational vision and values; organizational resources (human, economic and technical resources); organizational structures; service provision; and clinical practice and organizational context. The provider survey had five sections (n = 24 questions): practice demographics; quality and safety; clinical accountability; organization of the practice including its management, governance and organizational adaptiveness; and healthcare service delivery, which included satisfaction with their practice, coordination of care and collaboration with others.

Practice-based survey data collection

As detailed elsewhere (Wong et al. 2018), in consenting practices, one organizational lead completed the organizational survey, up to five clinicians completed a provider survey and all team members completed a TCI. Practice recruitment approaches included regional study advisory stakeholder committees, engagement with local organizations, presence at physician-attended events, peer-to-peer practice recruitment and demonstration of study relevance to physicians. The participation rates of practices were 38%, 41% and 32%, respectively, in Fraser East, Eastern Ontario and Central Zone. The completion rates of the provider and TCI surveys were 100% in both Fraser East and Eastern Ontario and 97% in Central Zone. We recruited a consecutive sample of attending patients (a minimum of 20 per practice) to complete the patient survey. Patients were eligible if they were (1) aged 18 years and over, (2) had been with their current provider for at least one year; and 3) were able to complete the survey in either English or French.

Case studies

We used a multiple comparative embedded case study design (Yin 2013) as detailed elsewhere (Martin-Misener et al. 2019). Specific jurisdictional information for primary care renewal initiatives were identified using document review, interviews and focus groups. The document review provided information on the implementation of primary care policies and innovations, involvement of stakeholders and results of policy evaluations from 2003 to 2014. Purposively selected key informants (e.g., lead decision makers, heads of regulatory colleges or associations) and clinician participants (e.g., family physicians, registered nurses, pharmacists) were recruited from a list of potential participants created with our regional advisory stakeholders. Patients were recruited from a convenience sample of patient participants who consented to be contacted after completing the patient experience survey. We used case study data to provide additional context for the quantitative results.

In-person or telephonic in-depth interviews were semi-structured and lasted about 60 minutes. In-person focus groups (n = 5–10 patients or clinicians) lasted two hours. Each focus group/interview was audio recorded and transcribed along with any field notes.

All data collection procedures for the practice-based surveys and case studies were approved by the ethics review boards of the University of British Columbia, University of Ottawa and the Ottawa Health Science Network, and the Dalhousie University and the Nova Scotia Health Authority.

Data analysis

To report the data, we had initially used the 2011 Patient's Medical Home (PMH) framework (CFPC 2011). For the final synthesis, we used the updated 2019 College of Family Physicians of Canada's (CFPC's) revised PMH model (CFPC 2019) as an organizing framework because it can provide primary care practices with a common understanding of attributes that should be attained in the context of ongoing jurisdictional primary care reform. The CFPC defines a PMH as “a family practice defined by its patients as the place they feel most comfortable presenting and discussing their personal and family health and medical concerns” (CFPC 2019: 2). Positive impacts associated with PMH delivery models include lower healthcare costs (Strumpf et al. 2017) and improved quality of care (Rosenthal et al. 2016). There are three themes associated with the PMH (CFPC 2019) – foundations, functions and ongoing development – represented by 10 pillars (Table 1).

Table 1.

PMH pillars and quantitative data sources of the TRANSFORMATION study

PMH pillars	Data sources (and dimensions)
Foundations
1. Administration and Funding Practices need staff and financial support, advocacy, governance, leadership and management in order to function as part of the community and deliver exceptional care.	Organizational survey (service provision and clinical practices, economic resources, organizational context and organizational structures) Clinician survey (management and practice governance)
2. Appropriate Infrastructure Physical space, staffing, electronic records and other digital supports, equipment and virtual networks facilitate the delivery of timely, accessible and comprehensive care.	Organizational survey (technical resources, organizational structures, organizational context, service provision and clinical practices) Clinician survey (information technology)
3. Connected Care Practice integration with other care settings and services – a process enabled by integrating health information technology.	Patient experience survey (coordination, orientation and uncategorized questions regarding support) Organizational survey (service provision and clinical practices and organizational context) Clinician survey (coordination of care and collaboration)
Functions
4. Accessible Care By adopting advanced and timely access, virtual access and team-based approaches, accessible care ensures that patients can be seen quickly.	Organizational survey (service provision and clinical practices and organizational context)> Patient experience survey (accessibility orientation)
5. Community Adaptiveness and Social Accountability A PMH is accountable to its community and meets their needs through interventions at the patient, practice, community and policy level.	Patient experience survey (uncategorized questions regarding difficulty receiving healthcare due to costs) Organizational survey (organizational context)
6. Comprehensive Team-Based Care with Family Physician Leadership A broad range of services is offered by an interprofessional team. The patient does not always see their family physician but interactions with all team members are communicated efficiently within a PMH. The team might not be co-located but the patient is always seen by a professional with relevant skills who can connect with a physician (ideally the patient's own personal physician) as necessary.	Organizational survey (service provision and clinical practices and organizational context) Patient experience survey (preventive health and uncategorized questions regarding experiences with the healthcare team) Clinician survey (health human resources' roles and responsibilities and organizational adaptiveness)
7. Continuity of Care Patients live healthier, fuller lives when they receive care from a responsible provider who journeys with them and knows how their health changes over time.	Organizational survey (service provision and clinical practices) Clinician survey (coordination of care and collaboration) Patient experience survey (uncategorized questions regarding relationship with healthcare professionals)
8. Patient and Family Partnered Care Family practices respond to the unique needs of patients and their families within the context of their environment.	Organizational survey (service provision and clinical practices) Patient experience survey (relationship-based care, empowerment and confidence, safe healthcare system coordination, self-management support, uncategorized questions regarding trust in healthcare staff)
Ongoing development
9. Measurement, Continuous Quality Improvement and Research Family practices strive for progress through performance measurement and continuous quality improvement. Patient safety is always a focus, and new ideas are brought to the fore through patient engagement in quality improvement and research activities.	Organizational survey (organizational vision and values; organizational structures; service provision and clinical practices; organizational context; and uncategorized questions regarding medication safety and diagnostic test results) Patient experience survey (safe healthcare system contribution) Clinician survey (quality and safety processes, accountability and organizational adaptiveness)
10. Education, Training and Continuous Professional Development Emphasis on training and education ensures that the knowledge and expertise of family physicians can be shared with the broader healthcare community and also over time by creating learning organizations where both students and fully practising family physicians can stay at the forefront of the best practices.	Organizational survey (organizational structures and organizational context) Clinician survey (provider satisfaction)

Source: CFPC 2019 (PMH pillars).

Our main outcomes of interest were the creation of PMH pillar scores (Table 2). Analysis took place in two phases: (1) mapping and item reduction and (2) creation of PMH pillar scores. We used the case study data to provide context to guide interpretation of the score.

All survey items from the patient, organizational, provider and TCI surveys that could be mapped to the PMH pillars were used to create the most comprehensive representation of each pillar.

Pillar Scores

Scale scores (0–100) within each survey were created where a higher score meant more alignment with the underlying construct. We then calculated the clinical averages of all the scores from the patient, provider, organizational and TCI surveys that mapped to a specific PMH pillar. To obtain the final regional-level pillar scores, we calculated the mean of all the clinic-level scores in each region after checking for normally distributed data.

Datasets containing quantitative data were analyzed using descriptive statistics. We used one-way analysis of variance or Kruskal–Wallis tests to examine if there were statistical differences across regional areas. All numerical analyses were carried out using SAS software, Version 9.4 (https://www.sas.com/en_ca/home.html).

Text data (document review) and transcripts (focus groups and interviews) were organized into codes. Team members from each province developed the coding structure using inductive and deductive processes (Crabtree and Miller 1999). The final coding structure was discussed with and approved by all team members. Two team members per study region coded the data. The research team discussed coding and emerging themes at monthly telephonic and three face-to-face meetings. Data were analyzed within each region and across regions (Yin 2013). All text analyses were carried out using the data management system NVivo 11 (https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home).

Results

The PMH unadjusted performance scores are reported on a 0 to 100 scale for each PMH pillar in Figure 1. Despite apparent regional differences in the attainment of PHC pillars, most standard deviations overlap with the exceptions of appropriate infrastructure; community adaptiveness and social accountability; and patient and family partnered care, where testing confirms statistically significant regional difference (p < 0.05). Fraser East had the highest scores with respect to appropriate infrastructure but the lowest score for community adaptiveness and social accountability of the regions. Eastern Ontario had the highest scores for patient and family partnered care. Comprehensive team-based care scores were higher in Fraser East and Eastern Ontario compared to Central Zone (p < 0.05). The lowest PMH attainment for all the regions were observed in connected care; accessible care; measurement, continuous quality improvement and research; and training, education and continuing professional development.

Figure 1.

Overall pillar scores

*Score is significantly different across provinces (p < 0.05).

BC = Fraser East, British Columbia; ON = Eastern Ontario Health Unit, Ontario; NS = Central Zone, Nova Scotia.

The overall pillar scores mask underlying differences between regions in performance on sub-dimensions of PMH pillars. We report details for the Comprehensive Team-Based Care with Family Physician Leadership pillar in Figure 2 as an example of how to read each PMH pillar, with the nine other PMH pillars reported in Figure A1 (Appendix 1, available online here). In each panel, the grey area indicates statistically significant differences between the scores across the regions. In the Comprehensive Team-Based Care pillar (Figure 2), Eastern Ontario (orange dots) scored significantly higher than the other two regions for four patient-reported experiences (health promotion, efficiency of skills used, managing health concerns and team role clarity). The sub-dimensional analysis further reveals that Fraser East (light green dots) provided a significantly larger number of services compared to the other regions, as measured by one organizational scale (availability of a range of services), providing nuances to the pillar score. Sub-dimensional analysis also revealed statistically significant lower roles for staff comprising registered nurses (reported in the organizational survey) and lower scores that were reported in the patient survey for team role clarity in Fraser East compared to the other regions. More detail on the other pillars can be found in Appendix 1 available online here.

Figure 2.

PMH pillar 6: Comprehensive Team-Based Care with Family Physician Leadership

Five items with p-value > 0.4 are not shown.

Cl = Clinic survey; Pr = Provider survey; Pt = Patient survey; TCI = Team Climate Index.

The case study work provided specific information about areas of primary care innovation: interprofessional team-based approaches, provider skill mix with the introduction of new providers or expansion of existing provider roles, physician groups and networks, physician remuneration models and incentives, the use of information technology and beginning performance measurement and having a reporting infrastructure (Martin-Misener et al. 2019). Ontario introduced interprofessional team-based care as well as new or expanded use of existing providers by defining model-specific mandates and governance requirements. There has been much focus on integration of nurse practitioners in primary care, with Ontario investing the greatest effort to do so over the longest period of time. The most extensive work with regard to physician groups and networks occurred in BC, with the implementation of 35 different divisions of family practice. Remuneration to primary care physicians was mainly through fee-for-service, except in Ontario, where there was greater use of capitation-based and salaried models. All regions had similar use of electronic medical records of between 46% and 52%. Ontario had implemented provincial strategies for performance measurement reporting to all physicians, whereas British Columbia and Nova Scotia's efforts were focused on individuals' practice improvement.

Patient respondents were heterogenous in their demographic characteristics (Table 3). Central Zone patients were more likely to be female, be younger, have higher educational attainment and not be retired compared to patients in Eastern Ontario and Fraser East (p < 0.05). There was more ethnic, Indigenous and immigrant diversity among those who participated in Fraser East compared to the other two regions (p < 0.05). French was spoken at home by those living in Eastern Ontario more frequently than in the other two regions (p < 0.05). Across all sites, about one third reported their financial situation as poor/very tight/tight and reported having either depression, arthritis or hypertension, whereas about 50% reported having three or more chronic conditions. Table 4 shows provider characteristics. Over half (53%) of clinician respondents were female. Clinicians were on average 51 years old, practising for about 21 years and working full time.

Table 3.

Characteristics of a patient survey sample

Demographics	Fraser East, BC (n = 504)	Eastern Ontario Health Unit, ON (n = 547)	Central Zone, NS (n = 878)	Total (n = 1,929)
Percentage of overall sample	26	28	46	100
Gender  Percentage female**	66	60	71	66
Age  Mean (SD)**	56 (18)	55 (16)	52 (16)	54 (17)
Percentage ethnicity** § and percentage Indigenous*
European descent	76	81	80	79
Asian	9	.	2	3
Indigenous	6	3	4	4
Other	4	2	4	3
Percentage born outside Canada**	24	8	7	12
Years living in Canada§  Mean (SD)	35 (18)	37 (19)	32 (23)	35 (20)
Percentage language spoken at home (check all that apply)
English**	95	67	99	89
French**	1	45	4	15
Other**	13	2	2	5
Percentage marital status**
Married/co-habitating	70	71	61	66
Percentage education**
Less than high school	14	12	7	10
High school	28	30	19	25
Some university or college	43	39	40	41
Completed undergraduate degree	11	14	22	17
Graduate degree	4	5	12	8
Percentage employment status**
Full time	30	42	40	38
Part time	11	8	11	10
Not employed outside the home	16	11	15	14
Long-term sickness or disability	9	9	9	9
Retired	34	30	24	29
Percentage yearly household income*§
<$5,000–$9,999	7	4	7	6
$10,000–$29,999	21	21	18	20
$30,000–$49,999	21	24	21	22
$50,000–$79,999	26	24	23	24
$80,000–$99,999	13	12	10	11
>$100,000	12	16	20	17
Percentage best describing your financial situation
Poor/very tight	13	12	15	14
Tight	17	16	16	17
Modestly comfortable	36	34	38	36
Comfortable/very comfortable	34	37	31	34
Percentage chronic conditions
Depression*	33	30	39	35
Arthritis	31	28	31	30
High blood pressure or hypertension	33	34	30	32
Zero chronic conditions	13	16	17	15
One chronic condition	19	18	20	19
Two chronic conditions	18	15	14	16
Three or more chronic conditions	51	51	49	50
Percentage health status*
Excellent/very good	29	34	31	31
Good	40	39	36	38
Fair/poor	32	27	34	31

^*p < 0.05.

^**p <0.001.

^§19% were missing “years in Canada”, 12% were missing for “income” and 11% were missing for “ethnicity” variables. The rest were all <7% missing. Use of healthcare measures were calculated for the subset of patients who consented to linkage and were able to be linked: 487 in Fraser East, BC; 533 in Eastern Ontario Health Unit, ON; and 821 in Central Zone, NS. Chi-squared tests were performed to test for differences in categorical variables between the provinces, and one-way analysis of variance tests were used to test for difference in continuous variables. When expected cell counts were less than five, Fisher's exact test of independence was performed in place of a chi-squared test.

Table 4.

Provider characteristics

Characteristics	Fraser East, BC (n = 34)	Eastern Ontario Health Unit, ON (n = 36)	Central Zone, NS (n = 47)	Total (n = 117)
Percentage female	42	53	62	53
Age  Mean (SD)	50 (10)	50 (9)	51 (10)	51 (10)
Work hours spent seeing patients Mean (SD)	29 (12)	30 (8)	30 (10)	30 (10)
Work hours spent not seeing patients  Mean (SD)	11 (11)	11 (9)	8 (6)	10 (9)
Weeks worked in the last year  Mean (SD)	46 (2)	46 (4)	46 (5)	46 (4)
Fee-for-service insured**	74 (29)	21 (30)	59 (38)	52 (39)
Salary*	14 (29)	33 (44)	13 (30)	19 (35)
Capitation**	0 (.)	37 (37)	0 (.)	11 (26)
Sessional/per diem/hourly*	4 (6)	0.1 (0.9)	5 (9)	3 (7)
Other^*	8 (10)	8 (13)	21 (30)	14 (22)
Percentage 1+ deliveries	31	25	s
Percentage only pre/post-natal care	53	75	s
Percentage no maternity care	16	0	s

^*p <0.05.

^**p <0.001; s = suppressed (fewer than five cases).

^{^}Other income category includes service contracts, fee-for-service uninsured (private pay services), incentives and bonuses and others. Chi-squared tests were performed to test for differences in categorical variables between the provinces, and one-way analysis of variance tests were used to test for difference in continuous variables. When expected cell counts were less than five, Fisher's exact test of independence was performed in place of a chi-squared test.

Discussion

To our knowledge, this work provides the first comprehensive portrait of regional primary care performance in Canada. Multiple sources of data were used to provide an overview of achievement of PMH orientation and relative strengths and weaknesses across the many pillars of the PMH framework within a region and across regions. Our work is one of the first attempts to provide a portrait on any regional variation and make comparisons across these areas toward the attainment of the PMH pillars. It identifies that multiple data sources are needed to move beyond opportunistic and piecemeal approaches to indicator selection for primary care evaluation. This work highlights the potential for evaluating and reporting on the multidimensionality of primary care simultaneously to show how this complex adaptive system might evolve over time.

The portraits create a picture of relative primary care performance at a point in time. We found significant regional differences in policy contexts and variation in regional achievement toward a PMH model in primary care across several pillars. Particularly troubling is the overall low performance on the Connected Care and Accessibility pillars across jurisdictions despite these being considered core attributes of PHC (Haggerty et al. 2007; Starfield 1998). Achievement of pillar goals varies across jurisdictions for community adaptiveness and social accountability, appropriate infrastructure, comprehensive team-based care and patient- and family-centered care. Relative to each other, Central Zone appears to lag behind Fraser East and Eastern Ontario for most pillar scores.

While our work cannot establish the cause of differences, it sets the stage for longitudinal assessments of primary care performance and the impact of different policies across fairly similar provincial health systems and populations. Importantly, it offers a model for assessment of the relative performance of different PHC performance domains within and across regions. This is needed as progress in PHC seeks to improve all pillars and detect variation, which may be reflective of shifting priorities.

Katz et al. (2017) used the 2013/14 Canadian Quality and Costs of Primary Care data, which also included patient, clinician and organizational data to examine alignment with goals set in reference to the PMH model. This study provides valuable baseline measurement on the standard of primary care across Canada. Our study conducted several years later also found that Ontario practices continue to score significantly higher than their British Columbia or Nova Scotia counterparts in the areas of comprehensive team-based care and patient- and family-partnered care. Albeit the different analytic techniques across the two studies, it seems clear that the policy focus on expansion of team-based care models and increased use of nurses and nurse practitioners has helped Ontario offer more team-based care.

Limitations

This work should be interpreted with caution as no causation can be inferred. There is likely selection bias in clinicians and patient participants; although, this is analogous to previous primary care surveys in Canada (Hogg et al. 2008; Katz et al. 2017). We collected limited patient data per clinician. The patient, provider and organizational surveys draw on previously validated survey instruments. We note that extensive survey validation was not completed for this study. This work was meant to show the feasibility of reporting at a regional level, where we use means of all clinics in the regions to indicate overall performance. Other data sources that could enhance performance measurement include administrative and electronic medical record (EMR) data. The PMH framework has limitations, such as the absence of a pillar addressing healthcare equity.

Despite limitations, we provide compelling evidence about the feasibility of collecting data from multiple sources to inform a more comprehensive portrait of primary care performance. Linking these data to EMR and health administrative data within provinces and territories is feasible and likely to become easier with increased adoption of information technology. These data form the foundation for performance measurement and reporting, a fundamental part of a learning health system (Smith et al. 2010). Developing a primary care information system that could be used as a foundation for a learning health system is within reach. Funding, jurisdictional, political and logistical barriers to implementing the system are, however, formidable.

Comprehensive measurement and reporting of primary care is challenging, given the complexity inherent in primary care and the systems which consist of individuals that have freedom to act in ways that are not always predictable but whose actions are interconnected (Bureau of Health Information 2014; Donabedian 1988). Development of any primary care information system requires engaged stakeholders, funding and leadership to tackle important resource-related questions. Support to practices will be required to maintain data security and patient privacy while allowing data collection for longitudinal evaluations. The most likely funders of the data collection infrastructure are the federal, provincial and territorial governments because they are the stewards of the healthcare system. Building this system requires trust that data are reliable and valid and not being used for disciplinary purposes or to determine remuneration. Leadership, engagement and reporting on primary care performance could best be done by an arm's length organization whose mandate is to support the accountability of the healthcare system. One example is the Bureau of Health Information in New South Wales (NSW), where they report on the performance of NSW's publicly funded health system (Bureau of Health Information 2018). Similar types of organizations across Canada's provincial/territorial landscape exist, such as Health Quality Councils (Johnston and Hogel 2016).

A primary care information system that collects data from multiple sources can be used for reporting performance toward national PMH priorities. These multi-source data from jurisdictional contexts can transparently highlight differences for the purpose of improvement and learning (Aschengrau and Seage 2009). High-level aggregation, similar to work completed by the CIHI (2021) in addition to more detailed information, can be useful to many, including provincial and health authority health system planners and clinicians for quality improvement. Our work provides a way forward to address what remains a limited primary care performance measurement information system in Canada.

Funding

This research was funded by the Canadian Institutes of Health Research (grant number TTF-128265) and the Michael Smith Foundation for Health Research (grant number PT-CPH-00001-134).