Abstract
Background
Voluntary clinician engagement in audit and feedback (A&F) initiatives is often suboptimal. We compared a range of characteristics between physicians who voluntarily engaged and those who did not.
Methods
We conducted a retrospective cohort study using administrative databases and a survey. The cohort included Ontario primary care physicians practising between April 2017 and March 2020. The A&F group voluntarily enrolled in an A&F intervention; control physicians did not enrol. We compared demographic, practice and patient characteristics between groups. Four validated questionnaires measured beliefs about feedback and improvement. Logistic regression modelled adjusted odds of A&F engagement.
Results
Of 10 654 active primary care physicians in Ontario, 1400 voluntarily engaged in A&F compared with 9254 controls. Physicians had higher odds of engaging with A&F if they had a larger roster size (OR (95% CI) 2.75 (2.15 to 3.51), p<0.001) and worked in home care (1.75 (1.55 to 1.97), p<0.001), and lower odds of engaging if they had higher continuity of care (0.66 (0.49 to 0.90), p=0.008), practised for more years (0.53 (0.44 to 0.62), p<0.001) and enrolled in a fee-for-service model (0.74 (0.65 to 0.85), p<0.001). Physicians engaged with A&F had fewer racialised or immigrant patients (25% vs 30% in marginalisation quintile 5, standardised mean difference=0.10), while other demographic characteristics were similar. Among the survey respondents (36 A&F physicians and 90 controls), higher principal support (2.28 (1.29 to 4.48), p=0.008), lower perceived need for change (discrepancy) (0.39 (0.20 to 0.70), p=0.003) and lower engagement in QI (0.43 (0.18 to 0.88), p=0.032) predicted A&F engagement.
Conclusion
To expand A&F’s reach and impact, future efforts should (1) tailor outreach to physicians with larger, more diverse practice populations, (2) seek to understand why physicians with higher perceived need for change engage less, (3) understand which physicians are most likely to benefit and (4) co-design how to make A&F more relevant and actionable for those serving marginalised communities.
Keywords: audit and feedback; educational outreach, academic detialing; general practice; healthcare quality improvement; primary care
WHAT IS ALREADY KNOWN ON THIS TOPIC
Audit and feedback (A&F) is an effective quality improvement strategy to improve physician practice, but voluntary engagement is often suboptimal with limited knowledge on who engages and who does not.
WHAT THIS STUDY ADDS
Physicians had higher odds of engaging with A&F if they had a larger roster size or worked in home care, but lower odds of engaging if they had higher continuity of care, practised for longer or enrolled in a fee-for-service model.
A&F physicians saw fewer racialised or immigrant patients than controls.
Psychological constructs such as higher principal support and lower perceived need for change (discrepancy) predicted A&F engagement.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Identifying individuals who are more likely to engage in A&F will inform the development of tailored, evidence-based co-interventions to be delivered alongside A&F to maximise its reach and effectiveness.
Introduction
Audit and feedback (A&F) is the most commonly used quality improvement (QI) strategy,1–4 yet its full potential remains unrealised. The intended purpose is to support healthcare professionals in identifying gaps or opportunities for improvement in their practice, so that they may make the appropriate adjustments to improve patient outcomes.5 While A&F can be effective when well-designed and well-implemented, most large-scale initiatives experience low voluntary engagement6—undermining their potential to drive meaningful change.7 Little attention has been paid to the characteristics of those receiving feedback, leaving a critical gap in understanding who engages with A&F and how they differ from those who do not.
Suboptimal uptake of A&F initiatives raises the question—what assumptions are we making about A&F recipients? The decision to engage with A&F may be driven by individual differences such as receptivity to change, personal beliefs and QI capabilities—the skills needed to interpret practice-level data and implement changes.8 Without a deeper understanding of these recipient variables, efforts to improve A&F will continue to fall short. Literature on individual and contextual factors associated with participation has been sparse and largely qualitative in nature.9 10
There is a need to systematically understand recipient variables, starting with who engages with A&F and the characteristics of the patients they care for. The objective of this study was to understand the practice characteristics, patient characteristics and beliefs of primary care physicians who do and do not engage with a voluntary A&F intervention in Ontario, Canada. Our use of provincial health administrative databases will allow us to understand variability within a large sample of clinicians that includes diversity in primary care models, patient demographics and geographic density, among others. Findings from this study will provide insights into whether and for whom tailored, evidence-based co-interventions may be delivered alongside A&F to maximise its reach and effectiveness.
Methods
A&F intervention
The MyPractice Primary Care Report, a collaborative effort spearheaded by Ontario Health (a Crown agency of the Government of Ontario that oversees the delivery of healthcare in Ontario) and ICES (formerly the Institute for Clinical Evaluative Sciences, a non-profit organisation funded by the Ontario Ministry of Health for the purpose of housing routinely collected health administrative data), is an A&F initiative launched in April 2017 focused on primary care.11 Physicians can voluntarily register to receive their A&F report by completing an online form. After providing consent, confidential, individualised reports were pushed to physicians, providing them with practice, regional and provincial-level data, allowing them to compare their care patterns and patient outcomes with their peers. Topics included opioid prescribing, cancer screening, diabetes management and patients’ use of other health services such as emergency departments. MyPractice reports highlighted potential areas for improvement along with ideas on how physicians may action their feedback. Reports were refreshed twice per year and reflected data up to 9 months prior. A sample report is available in online supplementalappendix 1. Studies have shown the effectiveness of MyPractice in improving physician practice.6
Study design and population
We conducted a retrospective cohort study, whereby physicians were categorised as part of either the A&F group if they registered for the A&F programme (MyPractice Primary Care Report) and were actively engaged with it during the study period (defined as having downloaded the report at least once during the study period). The control group included all actively practising primary care physicians who either did not register for A&F during the study period or registered but never accessed the report. For the A&F group, the physician’s index date was their date of A&F registration, while for the control group, their index date was a randomly assigned date based on the distribution of index dates for the A&F group. Reporting follows the Strengthening the Reporting of Observational Studies in Epidemiology guidelines.
We included all primary care physicians who were actively practising in Ontario from 1 April 2017 to 31 March 2020. This period was selected to maximise the time frame while excluding data from the COVID-19 pandemic, during which significant disruptions to the primary care sector and health system could introduce confounding variables and potentially contaminate the analysis. The following criteria were applied for inclusion: (1) physician exists in Primary Care Population Database dataset, (2) worked a minimum of 44 unique service days within 1 year of their index date, (3) had billings in at least seven activity areas, (4) had an own-patient ratio of 0.20 or higher, defined as the number of primary care visits to the physician in the past 2 years prior to index date by rostered patients divided by the total number of primary care visits in the past 2 years prior to the index date by all patients to the physician (this criteria was intended to exclude physicians with a walk-in practice) and (5) were not deemed to be emergency physicians based on billing activity.
We also identified all patients who were rostered or virtually rostered to the physicians in this study in the year prior to the physician’s index date. Virtual rostering refers to a method of attributing patients to a primary care physician based on their healthcare utilisation patterns rather than formal patient enrolment.
We recruited a subset of the physicians in the study to complete a survey on psychological recipient factors based on individual beliefs about the intervention. Participants were recruited through three channels of distribution: (1) mailing lists of key primary care organisations (eg, Association of Family Health Teams of Ontario, Centre for Effective Practice, Ontario Medical Association, Ontario College of Family Physicians and University of Toronto Practice-Based Research Network), (2) via Ontario Health’s distribution list of registered recipients of the MyPractice Primary Care Practice Report (n=2728) and (3) social media platforms of the study lead (LD) and key primary care organisations. Interested participants received a link to the online survey containing the questionnaire. To increase the response rate, two reminders were sent out at 2-week intervals and participants were offered a $C50 gift card for completing the survey.
Data sources
The list of ICES administrative databases used in this study is available in online supplemental table 1. All practice and patient characteristics derived from these databases were linked with a registry of physicians who had voluntarily registered to receive an A&F report to create the two cohorts. These data were then linked to survey data collected for the purposes of this study. Databases were linked using unique encoded identifiers and analysed at ICES. ICES is an independent, non-profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyse healthcare and demographic data, without requiring consent, for health system evaluation and improvement. The use of the data for this project is authorised under section 45 of Ontario’s Personal Health Information Protection Act.
For the survey, scales were selected with strong psychometric properties that evaluate constructs of interest identified from the literature that were both related to feedback and QI and can be effectively targeted by behaviour change interventions (see online supplemental appendix 2 for full questionnaire). Psychological constructs included: (1) the Feedback Orientation Scale (FOS), which measured individual receptivity to feedback based on subdomains of accountability (degree to which one feels responsible for acting on feedback), feedback self-efficacy (confidence in ability to understand and act on feedback) and feedback utility (belief that the feedback is useful).12 The social awareness subdomain of the FOS was omitted as A&F does not provide a means for recipients to understand how others view them; (2) the Organisational Change Recipients’ Belief Scale (OCRBS), which measured organisational and individual aspects of readiness to change, including valence (perceived benefit of the change), discrepancy (perceived need for change) and principal support (perceived support from leadership)13; (3) the Psychological Safety and Team Learning Scale (PSTLS), which measures psychological safety, engagement in QI work and team learning behaviours14 and (4) the Regulatory Focus Questionnaire (RFQ), which consists of two subscales that measure an individual’s orientation towards their goals using a prevention focus (eg, avoid loss) versus a promotion focus (eg, pursue gains).15
Baseline characteristics
The following characteristics were extracted on the physician’s index date: age, sex, rurality (measured by the Rurality Index for Ontario score, with higher score indicating greater degree of rurality), years of practice, roster size in past year, mean physician-level continuity from past 2 years (defined as the mean Usual Provider of Care index of all rostered patients for each physician, which was then averaged at the physician-level; a greater value indicates the provision of more continuous care with their patients), having worked in an emergency department, having worked in home care and country of graduation. Primary care physicians in Ontario are typically enrolled in one of several funding and organisational models available, with each model blending fee-for-service (FFS), capitation and team-based funding in unique ways16 17 (see online supplemental table 2 for more details). Therefore, we also included enrolment model, categorised as either an FFS model (including comprehensive care models, family health groups or physician not enrolled in a formal model) and non-FFS models (including capitation models, family health teams and those coded as other enrolment group). We also included group size, defined as number of physicians in the same assigned group, for those enrolled in a formal model.
The following patient characteristics were extracted on their physician’s index date: age, sex, immigrant status, rurality, neighbourhood income quintile, provider of care continuity (defined as the number of primary care visits to the rostered physician divided by the total number of visits to all providers in the 2 years prior to index date), number of core primary care visits to the rostering physician in the past 2 years (core visits defined in online supplemental table 3), age and labour force marginalisation quintile, households and dwellings marginalisation quintile, material resources marginalisation quintile, racialised and newcomer populations marginalisation quintile, resource utilisation band (with 0 indicating non-user and 5 indicating very high morbidity) and Aggregate Diagnosis Groups (ADG) score (an indicator of comorbidity).
Practice and patient characteristics were selected to ensure a comprehensive descriptive profile of both A&F physicians and control physicians and the patients they serve.
Statistical analysis
All characteristics and scores were reported as means and SD for continuous variables and proportions for binary variables. In order to statistically compare the A&F and control groups, we calculated the standardised mean differences (SMD) for each variable, with the interpretation that values of <0.1 are considered negligible difference and values ≥0.1 show some degree of imbalance (with larger SMD indicating a larger imbalance between groups).18 To account for potential confounding between our characteristics of interest, we performed logistic regression modelling in both the full sample and the survey subsample using physician-level data only, with the outcome being engagement in A&F. We selected physician characteristics to adjust for based on the following justifications: (1) variables with significant SMDs as indicated by the univariate analyses while excluding those that were likely collinear with another variable (such as age and years of practice) and (2) variables that did not have a significant SMD but were associated with the outcome in theoretical models from prior research (eg, OCRBS discrepancy).19 All physician demographic and practice characteristics included in the two regression models were extracted from administrative data, and given the completeness of these data, there were no missing values. Any missing responses to the survey questions related to the psychological constructs were treated as ‘prefer not to answer’ given that the missing response may reflect hesitance, low trust or other underlying traits for disengagement, as opposed to being truly missing. Linearity assumptions and influential observations were examined. Assumptions of independence of errors were examined through a non-parametric dispersion of residuals fitted versus simulated and multicollinearity among predictors was assessed using variance of inflation factors. Bootstrap validation (n=200 replicates) was performed to assess model stability. All analyses were conducted using SAS V.8.3.
Results
Physician baseline characteristics
Figure 1 outlines the cohort creation process through which we identified 10 654 eligible physicians, of which 9254 did not register for A&F or registered but were inactive (control group) while 1400 registered and were active (A&F group) (table 1). The following characteristics were similar between groups (SMD <0.1): sex, rurality, worked in emergency department and country of graduation. Age was lower among the A&F group than controls (mean 48.50 vs 51.00 years, SMD=0.20), as was years of practice (mean 21.50 vs 24.16 years, SMD=0.20). Roster size was higher among A&F physicians than controls (mean 1313.81 vs 1149.00 patients, SMD=0.21). Mean physician-level continuity was higher in the A&F group than control (36.5% vs 41.7% had a low index, SMD=0.11). More A&F physicians worked in home care compared with controls (66.1% vs 53.5%, SMD=0.26). The enrolment model also differed between groups, with more A&F physicians enrolled in FFS than controls (65.5% vs 48.8%, SMD=0.34). A&F physicians generally had a smaller group size than controls (44.6% vs 38.0% had a group size of <6, SMD=0.14; 31.2% vs 41.5% had a group size of 16+, SMD=0.22).
Figure 1. Cohort creation for the study population. PCPOP, Primary Care Population Database.
Table 1. Physician baseline characteristics.
| Characteristic | Overall (n=10 654) | Control (n=9254) | A&F (n=1400) | SMD |
|---|---|---|---|---|
| Age, mean (SD) | 50.67 (12.96) | 51.00 (13.04) | 48.50 (12.17) | 0.20* |
| Male sex, n (%) | 5551 (52.1%) | 4848 (52.4%) | 703 (50.2%) | 0.04 |
| Practice years, mean (SD) | 23.81 (13.52) | 24.16 (13.60) | 21.50 (12.76) | 0.20* |
| Practice years, n (%) | ||||
| <10 | 2273 (21.3%) | 1917 (20.7%) | 356 (25.4%) | 0.11* |
| 10–19 | 2036 (19.1%) | 1742 (18.8%) | 294 (21.0%) | 0.06 |
| 20–29 | 2476 (23.2%) | 2160 (23.3%) | 316 (22.6%) | 0.02 |
| 30+ | 3869 (36.3%) | 3435 (37.1%) | 434 (31.0%) | 0.13* |
| Country of graduation, n (%) | ||||
| Canada | 6275 (58.9%) | 5429 (58.7%) | 846 (60.4%) | 0.04 |
| International | 2291 (21.5%) | 2024 (21.9%) | 267 (19.1%) | 0.07 |
| Missing | 2088 (19.6%) | 1801 (19.5%) | 287 (20.5%) | 0.03 |
| RIO score (rurality), n (%) | ||||
| 0–9 | 7477 (70.2%) | 6516 (70.4%) | 961 (68.6%) | 0.04 |
| 10–39 | 1575 (14.8%) | 1355 (14.6%) | 220 (15.7%) | 0.03 |
| 40+ | 721 (6.8%) | 629 (6.8%) | 92 (6.6%) | 0.01 |
| Missing | 881 (8.3%) | 754 (8.1%) | 127 (9.1%) | 0.03 |
| Roster size, mean (SD) | 1170.66 (810.58) | 1149.00 (814.33) | 1313.81 (770.50) | 0.21* |
| Roster size, n (%) | ||||
| <500 | 2377 (22.3%) | 2204 (23.8%) | 173 (12.4%) | 0.30* |
| 500–999 | 2276 (21.4%) | 1956 (21.1%) | 320 (22.9%) | 0.04 |
| 1000–1499 | 2756 (25.9%) | 2338 (25.3%) | 418 (29.9%) | 0.10* |
| 1500–1999 | 1797 (16.9%) | 1535 (16.6%) | 262 (18.7%) | 0.06 |
| 2000+ | 1448 (13.6%) | 1221 (13.2%) | 227 (16.2%) | 0.09 |
| Mean physician-level continuity, n (%) | ||||
| Low (0.0 to 0.5) | 4374 (41.1%) | 3863 (41.7%) | 511 (36.5%) | 0.11* |
| Moderate (0.5 to 0.8) | 4928 (46.3%) | 4202 (45.4%) | 726 (51.9%) | 0.13* |
| High (0.8 or higher) | 1351 (12.7%) | 1188 (12.8%) | 163 (11.6%) | 0.04 |
| Missing | 1 (0.0%) | 1 (0.0%) | 0 (0.0%) | 0.02 |
| Mean physician-level continuity (quintile), n (%) | ||||
| 0 to <0.2 | 1272 (11.9%) | 1179 (12.7%) | 93 (6.6%) | 0.21* |
| 0.2 to <0.4 | 1988 (18.7%) | 1724 (18.6%) | 264 (18.9%) | 0.006 |
| 0.4 to <0.6 | 2540 (23.8%) | 2178 (23.5%) | 362 (25.9%) | 0.005 |
| 0.6 to <0.8 | 3503 (32.9%) | 2985 (32.3%) | 518 (37.0%) | 0.10* |
| 0.8 to 1.0 | 1351 (12.7%) | 1188 (12.8%) | 163 (11.6%) | 0.04 |
| Worked in ED, n (%) | 1255 (11.8%) | 1082 (11.7%) | 173 (12.4%) | 0.02 |
| Worked in home care, n (%) | 5876 (55.2%) | 4950 (53.5%) | 926 (66.1%) | 0.26* |
| Enrolment category, n (%) | ||||
| Fee-for-service (CCM, FHG, NOG) | 5429 (51.0%) | 4512 (48.8%) | 917 (65.5%) | 0.34* |
| Not fee-for-service (FHT, OGP, CAP) | 5225 (49.0%) | 4742 (51.2%) | 483 (34.5%) | |
| Enrolment model, n (%) | ||||
| CAP, n (%) | 2693 (25.3%) | 2256 (24.4%) | 437 (31.2%) | 0.15* |
| CCM, n (%) | 320 (3.0%) | 286 (3.1%) | 34 (2.4%) | 0.04 |
| FHG, n (%) | 2391 (22.4%) | 2097 (22.7%) | 294 (21.0%) | 0.04 |
| FHT, n (%) | 2645 (24.8%) | 2184 (23.6%) | 461 (32.9%) | 0.21* |
| NOG, n (%) | 2514 (23.6%) | 2359 (25.5%) | 155 (11.1%) | 0.38* |
| OGP, n (%) | 91 (0.9%) | 72 (0.8%) | 19 (1.4%) | 0.06 |
| Group size, n (%) | ||||
| <6, n (%) | 4140 (38.9%) | 3515 (38.0%) | 625 (44.6%) | 0.14* |
| 6–10, n (%) | 1465 (13.8%) | 1247 (13.5%) | 218 (15.6%) | 0.06 |
| 11–15, n (%) | 772 (7.2%) | 652 (7.0%) | 120 (8.6%) | 0.06 |
| 16+, n (%) | 4277 (40.1%) | 3840 (41.5%) | 437 (31.2%) | 0.22* |
SMD is ≥ 0.1.
A&F, audit and feedback; CAP, capitation; CCM, comprehensive care model; ER, emergency department; FHG, family health group; FHT, family health team; NOG, physician not enrolled in any model; OGP, other enrolment group; RIO, Rurality Index for Ontario; SMD, standardised mean difference.
Patient baseline characteristics
10 632 864 patients were rostered to physicians in the control group compared with 1 839 328 patients for the A&F group (table 2). All patient characteristics were similar between groups (SMD <0.1), except for racialised and newcomer populations' marginalisation, where the A&F group had fewer patients in the highest quintile compared with the control group (25% vs 30% in quintile 5, SMD=0.10).
Table 2. Patient baseline characteristics.
| Characteristic | Overall (n=12 472 192) | Control (n=10 632 864) | A&F (n=1 839 328) | SMD |
|---|---|---|---|---|
| Age, mean (SD) | 41.60 (22.91) | 41.50 (22.92) | 42.20 (22.89) | 0.03 |
| Male sex, n (%) | 5 992 216 (48%) | 5 111 152 (48%) | 881 064 (48%) | 0 |
| Immigrant, n (%) | 719 026 (6%) | 628 592 (6%) | 90 434 (5%) | 0.04 |
| Rural, n (%) | 862 074 (7%) | 725 280 (7%) | 136 794 (7%) | 0.02 |
| Income quintile in $C, n (%) | ||||
| 1 | 2 361 161 (19%) | 2 028 349 (19%) | 332 812 (18%) | 0.03 |
| 2 | 2 433 992 (20%) | 2 075 844 (20%) | 358 148 (19%) | 0 |
| 3 | 2 537 603 (20%) | 2 161 554 (20%) | 376 049 (20%) | 0 |
| 4 | 2 554 219 (20%) | 2 177 245 (20%) | 376 974 (20%) | 0 |
| 5 | 2 585 217 (21%) | 2 189 872 (21%) | 395 345 (21%) | 0.02 |
| Provider of care continuity, mean (SD) | 0.64 (0.38) | 0.64 (0.38) | 0.64 (0.38) | 0 |
| Provider of care continuity (quintile), n (%) | ||||
| 0.0 to <0.2 | 1 197 704 (10%) | 1 023 927 (10%) | 173 777 (9%) | 0.006 |
| 0.2 to <0.4 | 945 138 (8%) | 807 111 (8%) | 138 027 (8%) | 0.003 |
| 0.4 to <0.6 | 1 572 108 (13%) | 1 342 752 (13%) | 229 356 (12%) | 0.005 |
| 0.6 to <0.8 | 1 801 640 (14%) | 1 533 348 (14%) | 268 292 (15%) | 0.005 |
| 0.8 to 1.0 | 5 865 914 (47%) | 4 996 724 (47%) | 869 190 (47%) | 0.005 |
| Missing | 1 089 688 (9%) | 929 002 (9%) | 160 686 (9%) | 0 |
| Core primary care visits to rostering physician, mean (SD) | 4.46 (5.58) | 4.47 (5.61) | 4.43 (5.40) | 0.01 |
| Age and labour force marginalisation quintile, n (%) | ||||
| 1 | 3 425 373 (27%) | 2 945 153 (28%) | 480 220 (26%) | 0.04 |
| 2 | 2 516 942 (20%) | 2 152 097 (20%) | 364 845 (20%) | 0.01 |
| 3 | 2 167 727 (17%) | 1 840 371 (17%) | 327 356 (18%) | 0.01 |
| 4 | 2 069 237 (17%) | 1 753 092 (16%) | 316 145 (17%) | 0.02 |
| 5 | 2 183 505 (18%) | 1 847 797 (17%) | 335 708 (18%) | 0.02 |
| Missing | 109 408 (1%) | 94 354 (1%) | 15 054 (1%) | 0.01 |
| Households and dwellings marginalisation quintile, n (%) | ||||
| 1 | 2 848 507 (23%) | 2 434 042 (23%) | 414 465 (23%) | 0.01 |
| 2 | 2 381 681 (19%) | 2 021 137 (19%) | 360 544 (20%) | 0.02 |
| 3 | 2 243 824 (18%) | 1 904 251 (18%) | 339 573 (18%) | 0.01 |
| 4 | 2 203 660 (18%) | 1 874 958 (18%) | 328 702 (18%) | 0.01 |
| 5 | 2 685 112 (22%) | 2 304 122 (22%) | 380 990 (21%) | 0.02 |
| Missing | 109 408 (1%) | 94 354 (1%) | 15 054 (1%) | 0.01 |
| Material resources marginalisation quintile, n (%) | ||||
| 1 | 2 794 642 (22%) | 2 381 156 (22%) | 413 486 (22%) | 0 |
| 2 | 2 652 715 (21%) | 2 261 695 (21%) | 391 020 (21%) | 0 |
| 3 | 2 407 743 (19%) | 2 050 107 (19%) | 357 636 (19%) | 0 |
| 4 | 2 251 624 (18%) | 1 915 215 (18%) | 336 409 (18%) | 0.01 |
| 5 | 2 256 060 (18%) | 1 930 337 (18%) | 325 723 (18%) | 0.01 |
| Missing | 109 408 (1%) | 94 354 (1%) | 15 054 (1%) | 0.01 |
| Racialised and newcomer populations marginalisation quintile, n (%) | ||||
| 1 | 1 883 280 (15%) | 1 580 750 (15%) | 302 530 (16%) | 0.04 |
| 2 | 2 046 804 (16%) | 1 711 720 (16%) | 335 084 (18%) | 0.06 |
| 3 | 2 206 099 (18%) | 1 862 829 (18%) | 343 270 (19%) | 0.03 |
| 4 | 2 625 837 (21%) | 2 241 921 (21%) | 383 916 (21%) | 0.01 |
| 5 | 3 600 764 (29%) | 3 141 290 (30%) | 459 474 (25%) | 0.10 |
| Missing | 109 408 (1%) | 94 354 (1%) | 15 054 (1%) | 0.01 |
| Resource utilisation band, n (%) | ||||
| 0—non-user | 760 322 (6%) | 649 887 (6%) | 110 435 (6%) | 0.01 |
| 1—healthy user | 774 569 (6%) | 662 579 (6%) | 111 990 (6%) | 0.01 |
| 2—low morbidity | 2 545 131 (20%) | 2 173 904 (20%) | 371 227 (20%) | 0.01 |
| 3—moderate morbidity | 6 125 697 (49%) | 5 222 634 (49%) | 903 063 (49%) | 0 |
| 4—high morbidity | 1 661 904 (13%) | 1 412 312 (13%) | 249 592 (14%) | 0.01 |
| 5—very high morbidity | 604 569 (5%) | 511 548 (5%) | 93 021 (5%) | 0.01 |
| Number of ADGs | ||||
| 0 | 763 387 (6%) | 652 442 (6%) | 110 945 (6%) | 0 |
| 1 | 5 910 084 (47%) | 5 040 028 (47%) | 870 056 (47%) | 0 |
| 2 | 4 714 115 (38%) | 4 018 210 (38%) | 695 905 (38%) | 0 |
| 3+ | 1 084 606 (9%) | 922 184 (9%) | 162 422 (9%) | 0.01 |
SMD is ≥0.1.
ADG, Aggregate Diagnosis Groups; A&F, audit and feedback; SMD, standardised mean difference.
Psychological constructs survey
Table 3 reports the demographic characteristics of the subset of physicians who responded to the psychological constructs survey along with their responses to the questionnaire. A total of 126 physicians completed the survey, and of these, 90 were not registered for A&F (control group) while 36 were registered and active in the A&F programme. All measured demographic characteristics were different between the two groups (SMD ≥0.1) except for group size and Canadian medical graduate status.
Table 3. Demographic characteristics and psychological constructs for physicians who completed the survey.
| Overall (n=126) | Not registered for A&F (n=90) | Active in A&F (n=36) | SMD | |
|---|---|---|---|---|
| Demographic characteristics | ||||
| Male, n (%) | 54 (42.9%) | 42 (46.7%) | 12 (33.3%) | 0.28* |
| Years in practice, n (%) | ||||
| <10 | 40 (31.7%) | 26 (28.9%) | 14 (38.9%) | 0.21* |
| 10–19 | 30 (23.8%) | 24 (26.7%) | 6 (16.7%) | 0.25* |
| 20–30 | 30 (23.8%) | 21 (23.3%) | 9 (25.0%) | 0.04 |
| 30+ | 26 (20.6%) | 19 (21.1%) | 7 (19.4%) | 0.04 |
| Mean physician-level continuity (quintile), n (%) | ||||
| 0 to <0.2 | 16 (12.7%) | 13 (14.4%) | 3 (8.3%) | 0.19* |
| 0.2 to <0.4 | 31 (24.6%) | 22 (24.4%) | 9 (25.0%) | 0.01 |
| 0.4 to <0.6 | 20 (15.9%) | 13 (14.4%) | 7 (19.4%) | 0.13* |
| 0.6 to <0.8 | 47 (37.3%) | 31 (34.4%) | 16 (44.4%) | 0.21* |
| 0.8 to 1.0 | 12 (9.5%) | 11 (12.2%) | 1 (2.8%) | 0.36* |
| Enrolment model | ||||
| CAP, n (%) | 36 (28.6%) | 20 (22.2%) | 16 (44.4%) | 0.49* |
| CCM, n (%) | 1 (0.8%) | 0 (0.0%) | 1 (2.8%) | 0.24* |
| FHG, n (%) | 23 (18.3%) | 20 (22.2%) | 3 (8.3%) | 0.39* |
| FHT, n (%) | 43 (34.1%) | 32 (35.6%) | 11 (30.6%) | 0.11* |
| NOG, n (%) | 23 (18.3%) | 18 (20.0%) | 5 (13.9%) | 0.16* |
| RIO score, mean (SD) | 2.25 (2.71) | 2.09 (2.62) | 2.64 (2.94) | 0.20* |
| Group size | ||||
| <6 | 52 (41.3%) | 38 (42.2%) | 14 (38.9%) | 0.07 |
| 6–10 | 19 (15.1%) | 13 (14.4%) | 6 (16.7%) | 0.06 |
| 11–15 | 10 (7.9%) | 7 (7.8%) | 3 (8.3%) | 0.02 |
| 16+ | 45 (35.7%) | 32 (35.6%) | 13 (36.1%) | 0.01 |
| Canadian medical graduate†, n (%) | 88 (84.6%) | 63 (85.1%) | 25 (83.3%) | 0.05 |
| FOS | ||||
| Utility, mean (SD) | 18.98 (3.14) | 18.88 (3.19) | 19.25 (3.05) | 0.12* |
| Accountability, mean (SD) | 19.12 (2.66) | 19.12 (2.78) | 19.11 (2.38) | 0.004 |
| Self-efficacy, mean (SD) | 18.33 (2.91) | 18.42 (2.80) | 18.11 (3.21) | 0.10* |
| FOS total, mean (SD) | 56.44 (6.98) | 56.42 (7.01) | 56.47 (6.99) | 0.01 |
| OCRBS | ||||
| Valence, mean (SD) | 14.29 (2.36) | 14.18 (2.46) | 14.58 (2.10) | 0.18* |
| Principal support, mean (SD) | 19.80 (4.27) | 19.34 (4.21) | 20.97 (4.27) | 0.38* |
| Discrepancy, mean (SD) | 15.95 (2.85) | 15.91 (2.98) | 16.06 (2.55) | 0.05 |
| OCRBS total, mean (SD) | 49.76 (6.92) | 49.26 (7.13) | 51.03 (6.30) | 0.26* |
| PSTLS | ||||
| Psychological safety, mean (SD) | 13.48 (2.74) | 13.39 (2.92) | 13.72 (2.24) | 0.13* |
| Engagement in QI work, mean (SD) | 12.40 (2.98) | 12.27 (3.18) | 12.74 (2.37) | 0.17* |
| Team learning behaviours, mean (SD) | 26.03 (5.09) | 26.07 (5.44) | 25.94 (4.17) | 0.03 |
| PSTLS total, mean (SD) | 52.72 (10.79) | 52.57 (11.49) | 53.11 (8.95) | 0.05 |
| RFQ | ||||
| Regulatory focus, mean (SD) | 22.52 (3.57) | 22.54 (3.44) | 22.47 (3.92) | 0.02 |
| Prevention focus, mean (SD) | 19.40 (3.22) | 19.38 (3.27) | 19.44 (3.12) | 0.02 |
| RFQ total, mean (SD) | 41.76 (5.79) | 41.70 (6.10) | 41.92 (5.02) | 0.04 |
SMD is ≥0.1.
Medical school graduation information is missing for 22 physicians.
A&F, audit and feedback; CAP, capitation; CCM, comprehensive care model; FHG, family health group; FHT, family health team; FOS, Feedback Orientation Scale; NOG, physician not enrolled in any model; OCRBS, Organisational Change Recipients’ Belief Scale; PSTLS, Psychological Safety and Team Learning Scale; RFQ, Regulatory Focus Questionnaire; RIO, Rurality Index for Ontario; SMD, standardised mean difference.
For the FOS, the A&F group scored higher in utility (mean 19.25 vs 18.88, SMD=0.12) but lower in self-efficacy (mean 18.11 vs 18.42, SMD=0.10) than controls. For the OCRBS, A&F respondents scored higher on valence (mean 14.58 vs 14.18, SMD=0.18), principal support (mean 20.97 vs 19.34, SMD=0.38) and total OCRBS score (mean 51.03 vs 49.26, SMD=0.26). For the PSTLS, psychological safety was higher in the A&F group (mean 13.72 vs 13.39, SMD=0.13), as was engagement in QI work (mean 12.74 vs 12.27, SMD=0.17). There were no differences between groups in any of the items on the RFQ.
Regression model results
After adjusting for covariates in the full sample (table 4), we found that physicians were more likely to engage in A&F if they had a larger roster size (2000+ vs <499: OR (95% CI) 2.75 (2.15 to 3.51), p<0.001) or worked in home care (OR (95% CI) 1.75 (1.55 to 1.97), p<0.001). Physicians were less likely to engage in A&F if they had a higher mean physician-level continuity (Q5 vs Q1: OR (95% CI) 0.66 (0.49 to 0.90), p=0.008), practised for more years (30+ vs <10 years: OR (95% CI) 0.53 (0.44 to 0.62), p<0.001) or enrolled in an FFS model (OR (95% CI) 0.74 (0.65 to 0.85), p<0.001). Physician sex was not a significant predictor for A&F engagement. Diagnostics showed an appropriate model, with no significant violations of assumptions.
Table 4. Regression model results for full sample.
| Characteristic | OR (95% CI) | P value |
|---|---|---|
| Roster size (baseline is <499) | ||
| 500–999 | 1.76 (1.42 to 2.18)* | 0.002* |
| 1000–1499 | 2.18 (1.75 to 2.73)* | <0.001* |
| 1500–1999 | 2.06 (1.62 to 2.63)* | <0.001* |
| 2000+ | 2.75 (2.15 to 3.51)* | <0.001* |
| Mean physician-level continuity, quintile (baseline is quintile 1) | ||
| Quintile 2 | 1.12 (0.87 to 1.43) | 0.376 |
| Quintile 3 | 0.91 (0.70 to 1.18) | 0.482 |
| Quintile 4 | 0.89 (0.69 to 1.17) | 0.425 |
| Quintile 5 | 0.66 (0.49 to 0.90)* | 0.008* |
| Practice years (baseline is <10 years) | ||
| 10–19 | 0.78 (0.65 to 0.92)* | 0.004* |
| 20–29 | 0.72 (0.60 to 0.86)* | <0.001* |
| 30+ | 0.53 (0.44 to 0.62)* | <0.001* |
| Male physician (baseline is female) | 0.92 (0.82 to 1.03) | 0.186 |
| Fee-for-service enrolment model (baseline is not fee-for-service enrolment) | 0.74 (0.65 to 0.85)* | <0.001* |
| Worked in home care (baseline is did not work in home care) | 1.75 (1.55 to 1.97)* | <0.001* |
Statistically significant.
Table 5 shows the adjusted results for the survey subsample only. Physicians were more likely to engage in A&F if they scored higher on OCRBS principal support (OR (95% CI) 2.28 (1.29 to 4.48), p=0.008). Physicians were less likely to engage in A&F if they practised for more years (OR (95% CI) 0.18 (0.03 to 0.79), p=0.032) or enrolled in an FFS model (OR (95% CI) 0.04 (0.01 to 0.26), p=0.002). Physicians were also less likely to engage if they scored higher on OCRBS discrepancy (OR (95% CI) 0.39 (0.20 to 0.70), p=0.003) and PSTLS engagement in QI (OR (95% CI) 0.43 (0.18 to 0.88), p=0.032). Roster size, sex, FOS self-efficacy, FOS utility, OCRBS valence and PSTLS psychological safety were not significant predictors of A&F engagement in the survey subsample.
Table 5. Regression model results for survey subsample only.
| Variable | OR (95% CI) | P value |
|---|---|---|
| Roster size: 1000+ (baseline is <1000) | 0.70 (0.13 to 3.44) | 0.665 |
| Practice years: 10+ years (baseline is <10 years) | 0.18 (0.03 to 0.79) | 0.032* |
| Male physician (baseline is female) | 0.86 (0.24 to 2.97) | 0.814 |
| Fee-for-service enrolment model (baseline is not fee-for-service enrolment) | 0.04 (0.01 to 0.26) | 0.002* |
| FOS self-efficacy (grand mean scaled) | 0.99 (0.57 to 1.73) | 0.997 |
| FOS utility (grand mean scaled) | 1.07 (0.55 to 2.11) | 0.833 |
| OCRBS valence (grand mean scaled) | 0.96 (0.55 to 1.67) | 0.891 |
| OCRBS principal support (grand mean scaled) | 2.28 (1.29 to 4.48) | 0.008* |
| OCRBS discrepancy (grand mean scaled) | 0.39 (0.20 to 0.70) | 0.003* |
| PSTLS psychological safety (grand mean scaled) | 1.68 (0.87 to 3.47) | 0.137 |
| PSTLS engagement in QI (grand mean scaled) | 0.43 (0.18 to 0.88) | 0.032* |
Statistically significant.
FOS, Feedback Orientation Scale; OCRBS, Organisational Change Recipients’ Belief Scale; PSTLS, Psychological Safety and Team Learning Scale.QI, quality improvement;
Discussion
This population-based retrospective cohort study of physicians in Ontario, Canada found physicians were more likely to engage with A&F if they were earlier in their career, had a larger roster size, lower continuity of care, were not enrolled in an FFS model and provided home care. They had proportionately fewer patients in their practice in the highest quintile of racial marginalisation. Physicians voluntarily engaged with A&F were more likely to score higher on principal support, but lower on perceived need for change (discrepancy) and engagement in QI work. Together, these insights suggest that voluntary engagement with A&F is not evenly distributed across the physician population, but rather reflects a particular profile, indicating that A&F may not provide a universal opportunity.
To our knowledge, this is the first study to characterise differences between physicians who voluntarily engaged with A&F and those who did not, while also uniquely incorporating linked administrative data to understand implications for practice. We found that demographic profiles were similar between groups, apart from age; physicians engaged with A&F tended to be younger and have fewer years of practice than their colleagues who did not engage. This may be a result of less experienced physicians being more inclined to adjust their clinical approach compared with more established providers. A systematic review found that older physicians who have been practising longer were more likely to provide lower-quality care, including lower adherence to evidence-based standards of care and worse patient outcomes, citing a lack of up-to-date knowledge and lower receptivity to new standards of care as possible drivers.20 Furthermore, age is negatively correlated with feedback-seeking behaviour21; therefore, it is possible that older physicians feel more confident in their abilities and perceive less need to improve their performance. It may also be that older physicians are more likely to perceive A&F as judgemental, negatively influencing their desire to engage,22 while younger physicians may be more receptive and equipped to act on A&F due to the emphasis on QI in medical training.23
We found that physicians who engaged with A&F were more likely to have a larger roster size and lower continuity of care, and less likely to be enrolled in a FFS model, after adjusting for covariates. This combination of characteristics can be readily explained, as capitation or blended capitation models may incentivise larger patient rosters,24 and large roster sizes often precipitate lower continuity of care.25 These results are surprising and may reflect that physicians with larger patient rosters were more motivated to engage with A&F, aiming to manage their workload and optimise care delivery and efficiency, particularly since the quality of their care affects a greater number of patients. High patient loads may also exist within clinics with infrastructure (such as staffing, technology) that can support quality monitoring and integration of feedback into workflows,26 thereby increasing their organisational readiness for A&F—which aligns with our finding that A&F physicians scored higher on their overall OCRBS score than controls. Furthermore, capitation models (as opposed to FFS) incentivise preventative care and chronic condition management,24 aligning with the priorities of many A&F programmes and potentially encouraging participation.
Physicians who voluntarily engaged with A&F served a lower percentage of visible minorities or recent immigrants. This pattern aligns with our finding that non-FFS (eg, capitation) models predicted A&F engagement, consistent with prior work that capitation physicians serve more advantaged populations and fewer newcomers.27 Physicians caring for ethnically marginalised populations often manage more demanding practices, characterised by higher clinical complexity and additional challenges such as language barriers and the need for interpreter services,28 29 which may limit their time and ability to participate in QI interventions.30 Although our data cannot assess whether racial concordance influences these patterns, similar rates of international medical graduates across groups suggest physician training location does not explain the difference. Given the disproportionately worse outcomes experienced by marginalised patients,31 32 targeted strategies are needed to ensure A&F reaches physicians whose patients may benefit most. Stratifying feedback by levels of patient marginalisation could further help physicians identify and address care gaps in their populations.
Individual beliefs and capabilities influence engagement in feedback interventions.8 Physicians engaged with A&F reported greater readiness to change, a characteristic predictive of success of healthcare interventions.33 However, among the three measured readiness components—valence, discrepancy and principal support—principal support was the only positive predictor of A&F engagement after covariate adjustment. Effective leadership has been cited as an essential facilitator in clinician engagement and implementation of QI interventions through mechanisms of enhanced learning, participation, empowerment and a culture of improvement.34 35 Our findings suggest leadership support may drive A&F engagement.
Surprisingly, a higher score for discrepancy was associated with lower odds of engagement. Although greater perceived need for change would typically be expected to motivate participation, high discrepancy may have the opposite effect if the performance gap feels too large, undermining motivation to act.36 Large gaps can also conflict with self-perceptions of competence and professional identity, creating cognitive dissonance that leads some to reject feedback to avoid scrutiny, even if they recognise its value.37 Future work should examine whether physicians with high discrepancy self-select out of A&F, which would indicate a need for targeted recruitment strategies. The finding that lower perceived engagement in QI was associated with higher likelihood of voluntary A&F sign-up—an unexpected pattern that warrants further study to determine whether it reflects a readiness gap (recognition of a need to engage but insufficient supports) or a motivation to improve (using A&F as an entry point). Clarifying this distinction will be important for tailoring outreach and support strategies.
We recognise the possibility that the design of the intervention may have also influenced physician engagement; however, a qualitative study looking at the perspectives of family physicians who used MyPractice primary care reports revealed that aesthetic design changes to the intervention did not play a significant role in its effectiveness. Rather, practice changes were primarily driven by factors such as the alignment of recipients’ goals with the focus of the report, perceived validity and accuracy of the data and actionability of the feedback.38 It was beyond the scope of our study to evaluate the intervention content and design.
Limitations of this study include the use of administrative databases, which are necessary for population-level studies but are unable to provide clinical granularity, particularly when evaluating comorbidities and healthcare utilisation among patients. We were also limited in ascertaining some physician demographic traits such as ethnicity due to lack of available data. Furthermore, only a fraction of physicians responded to the psychological constructs survey, limiting our ability to make generalisable conclusions regarding attitudes and beliefs about feedback. Therefore, the survey analysis should be considered exploratory only, and should be followed up with future studies that feature a larger sample size. Lastly, our study relied on quantitative data, which has the potential to oversimplify the narrative of why some physicians chose to engage with A&F while others did not. For example, it is possible some physicians were not even aware of A&F and its purpose. Future studies should supplement these findings with qualitative data to gather insight beyond what the psychological constructs measured, and to evaluate the downstream effect of A&F enrolment on clinical outcomes while leveraging matching techniques to ensure comparability between groups.
Conclusion
This population-based cohort study identifies a distinct profile of physicians who voluntarily engage with A&F, including those with larger rosters, fewer years in practice, lower continuity, non-FFS models and fewer ethnically marginalised patients. Engagement was also shaped by psychological factors, with higher perceived leadership support and lower discrepancy predicting sign-up. These patterns suggest that A&F is not equitably accessed and may be least likely to reach physicians whose patients could benefit most. Strengthening A&F’s impact will require (1) targeted outreach to under-represented practice types; (2) deeper investigation into why those perceiving greater need for change engage less and (3) co-designing A&F approaches that better support physicians caring for marginalised populations.
Supplementary material
Footnotes
Funding: This study was funded by the Canadian Institutes of Health Research (PJT-178046) and was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC).
Provenance and peer review: Not commissioned; externally peer reviewed.
Patient consent for publication: Not applicable.
Data availability free text: The dataset from this study is held securely in coded form at ICES. Although legal data-sharing agreements between ICES and data providers (eg, healthcare organisations and government) prohibit ICES from making the dataset publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS (email: das@ices.on.ca). The full dataset creation plan and underlying analytic code are available from the authors on request, with the understanding that the computer programs may rely on coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification. Parts of this material are based on data and/or information compiled and provided by CIHI and the Ontario Ministry of Health. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. This document used data adapted from the Statistics Canada Postal Code OM Conversion File, which is based on data licensed from Canada Post Corporation, and/or data adapted from the Ontario Ministry of Health Postal Code Conversion File, which contains data copied under license from Canada Post Corporation and Statistics Canada.
Ethics approval: Ethics approval was obtained from the Trillium Health Partners Research Ethics Board under REB #1073.
Correction notice: This article has been corrected since it was first published online. The funding statement has been updated to acknowledge support from the ICES. In addition to this, the data availability statement has been updated to acknowledge that parts of the dataset are based on data and/or information compiled and provided by CIHI and the Ontario Ministry of Health.
Data availability statement
Data may be obtained from a third party and are not publicly available.
References
- 1.Reszel J, Dunn SI, Sprague AE, et al. Use of a maternal newborn audit and feedback system in Ontario: a collective case study. BMJ Qual Saf. 2019;28:635–44. doi: 10.1136/bmjqs-2018-008354. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Mulhall CL, Lam JMC, Rich PS, et al. Enhancing Quality Care in Ontario Long-Term Care Homes Through Audit and Feedback for Physicians. J Am Med Dir Assoc. 2020;21:420–5. doi: 10.1016/j.jamda.2019.11.017. [DOI] [PubMed] [Google Scholar]
- 3.Xu AXT, Brown K, Schwartz KL, et al. Audit and Feedback Interventions for Antibiotic Prescribing in Primary Care: A Systematic Review and Meta-analysis. Clin Infect Dis. 2025;80:253–62. doi: 10.1093/cid/ciae604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Singer A, Kosowan L, Abrams EM, et al. Implementing an audit and feedback cycle to improve adherence to the Choosing Wisely Canada recommendations: clustered randomized trail. BMC Prim Care . 2022;23:302. doi: 10.1186/s12875-022-01912-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Jamtvedt G, Flottorp S, Ivers N. In: Improving healthcare quality in Europe. Busse R, Klazinga N, Panteli D, et al., editors. 2019. Audit and feedback as a quality strategy. [Google Scholar]
- 6.Ivers NM, Taljaard M, Giannakeas V, et al. Effectiveness of confidential reports to physicians on their prescribing of antipsychotic medications in nursing homes. Implement Sci Commun. 2020;1:30. doi: 10.1186/s43058-020-00013-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;2012:CD000259. doi: 10.1002/14651858.CD000259.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Desveaux L, Ivers NM, Devotta K, et al. Unpacking the intention to action gap: a qualitative study understanding how physicians engage with audit and feedback. Implement Sci. 2021;16:19. doi: 10.1186/s13012-021-01088-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wagner DJ, Durbin J, Barnsley J, et al. Beyond quality improvement: exploring why primary care teams engage in a voluntary audit and feedback program. BMC Health Serv Res. 2017;17:803. doi: 10.1186/s12913-017-2765-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Shuldiner J, Lacroix M, Saragosa M, et al. Process evaluation of two large randomized controlled trials to understand factors influencing family physicians’ use of antibiotic audit and feedback reports. Implement Sci. 2024;19:65. doi: 10.1186/s13012-024-01393-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Health Quality Ontario Practice reports. 2025. https://www.hqontario.ca/Quality-Improvement/Practice-Reports Available.
- 12.Linderbaum BA, Levy PE. The Development and Validation of the Feedback Orientation Scale (FOS) J Manage. 2010;36:1372–405. doi: 10.1177/0149206310373145. [DOI] [Google Scholar]
- 13.Armenakis AA, Bernerth JB, Pitts JP, et al. Organizational Change Recipients’ Beliefs Scale: Development of an Assessment Instrument. J Appl Behav Sci. 2007;43:481–505. doi: 10.1177/0021886307303654. [DOI] [Google Scholar]
- 14.Edmondson A. Psychological Safety and Learning Behavior in Work Teams. Adm Sci Q. 1999;44:350–83. doi: 10.2307/2666999. [DOI] [Google Scholar]
- 15.Higgins ET, Friedman RS, Harlow RE, et al. Achievement orientations from subjective histories of success: Promotion pride versus prevention pride. Euro J Social Psych. 2001;31:3–23. doi: 10.1002/ejsp.27. [DOI] [Google Scholar]
- 16.Ontario Ministry of Health Primary care payment models in Ontario: King’s Printer for Ontario. 2024. https://www.ontario.ca/page/primary-care-payment-models-ontario Available.
- 17.Ontario Physicians Recruitment Alliance Summary of payment models. 2025. https://opra.ca/resources/physician-compensation Available.
- 18.Zhang Z, Kim HJ, Lonjon G, et al. Balance diagnostics after propensity score matching. Ann Transl Med. 2019;7:16. doi: 10.21037/atm.2018.12.10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Desveaux L, Wang R, Minotti SC, et al. The engagement equation: a model for understanding what drives physician engagement with data-driven clinical performance feedback. In Review. 2025 doi: 10.21203/rs.3.rs-6769454/v1. Preprint. [DOI] [PMC free article] [PubMed]
- 20.Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med. 2005;142:260–73. doi: 10.7326/0003-4819-142-4-200502150-00008. [DOI] [PubMed] [Google Scholar]
- 21.Anseel F, Beatty AS, Shen W, et al. How Are We Doing After 30 Years? A Meta-Analytic Review of the Antecedents and Outcomes of Feedback-Seeking Behavior. J Manage. 2015;41:318–48. doi: 10.1177/0149206313484521. [DOI] [Google Scholar]
- 22.Wang M, Burlacu G, Truxillo D, et al. Age differences in feedback reactions: The roles of employee feedback orientation on social awareness and utility. J Appl Psychol. 2015;100:1296–308. doi: 10.1037/a0038334. [DOI] [PubMed] [Google Scholar]
- 23.Bailey K, Raval S, Jomy J, et al. Implementation of experiential and equity-focused quality improvement and patient safety training into undergraduate medical education: a call to action. 2023
- 24.Gosden T, Forland F, Kristiansen IS, et al. Capitation, salary, fee-for-service and mixed systems of payment: effects on the behaviour of primary care physicians. Cochrane Database Syst Rev. 2000 doi: 10.1002/14651858.CD002215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Francis MD, Zahnd WE, Varney A, et al. Effect of number of clinics and panel size on patient continuity for medical residents. J Grad Med Educ. 2009;1:310–5. doi: 10.4300/JGME-D-09-00017.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Mesman R, Westert GP, Berden B, et al. Why do high-volume hospitals achieve better outcomes? A systematic review about intermediate factors in volume-outcome relationships. Health Policy. 2015;119:1055–67. doi: 10.1016/j.healthpol.2015.04.005. [DOI] [PubMed] [Google Scholar]
- 27.Rayner J. Toronto: Institute for Clinical Evaluative Sciences; 2012. Comparison of primary care models in ontario by demographics, case mix and emergency department use, 2008/09 to 2009/10. [Google Scholar]
- 28.Suphanchaimat R, Kantamaturapoj K, Putthasri W, et al. Challenges in the provision of healthcare services for migrants: a systematic review through providers’ lens. BMC Health Serv Res. 2015;15:390. doi: 10.1186/s12913-015-1065-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Walker PF, Barnett ED, Stauffer W. Wolters Kluwer; 2024. Medical care of adult refugees, immigrants, and migrants to the united states.https://www.uptodate.com/contents/medical-care-of-adult-refugees-immigrants-and-migrants-to-the-united-states Available. [Google Scholar]
- 30.Payne VL, Hysong SJ. Model depicting aspects of audit and feedback that impact physicians’ acceptance of clinical performance feedback. BMC Health Serv Res. 2016;16:260. doi: 10.1186/s12913-016-1486-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.De Maio FG, Kemp E. The deterioration of health status among immigrants to Canada. Glob Public Health. 2010;5:462–78. doi: 10.1080/17441690902942480. [DOI] [PubMed] [Google Scholar]
- 32.Lane G, Farag M, White J, et al. Chronic health disparities among refugee and immigrant children in Canada. Appl Physiol Nutr Metab. 2018;43:1043–58. doi: 10.1139/apnm-2017-0407. [DOI] [PubMed] [Google Scholar]
- 33.Augustsson H, Richter A, Hasson H, et al. The Need for Dual Openness to Change: A Longitudinal Study Evaluating the Impact of Employees’ Openness to Organizational Change Content and Process on Intervention Outcomes. J Appl Behav Sci. 2017;53:349–68. doi: 10.1177/0021886317691930. [DOI] [Google Scholar]
- 34.Ginsburg L, Easterbrook A, Geerts A, et al. ‘We listened and supported and depended on each other’: a qualitative study of how leadership influences implementation of QI interventions. BMJ Qual Saf. 2025;34:146–56. doi: 10.1136/bmjqs-2024-017795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Elizalde J, Lumibao J, Lizarondo L. Barriers and facilitators to health professionals’ engagement in quality improvement initiatives: a mixed-methods systematic review. Int J Qual Health Care. 2024;36:mzae041. doi: 10.1093/intqhc/mzae041. [DOI] [PubMed] [Google Scholar]
- 36.Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982;92:111–35. doi: 10.1037/0033-2909.92.1.111. [DOI] [PubMed] [Google Scholar]
- 37.Gude WT, Brown B, van der Veer SN, et al. Clinical performance comparators in audit and feedback: a review of theory and evidence. Implement Sci. 2019;14:39. doi: 10.1186/s13012-019-0887-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Rouleau G, Reis C, Ivers N, et al. Characterizing the Gaps Between Best-Practice Implementation Strategies and Real-world Implementation: Qualitative Study Among Family Physicians Who Engaged With Audit and Feedback Reports. JMIR Hum Factors. 2023;10:e38736. doi: 10.2196/38736. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data may be obtained from a third party and are not publicly available.