Impact of a primary healthcare quality improvement program on diabetes in Canada: evaluation of the Quality Improvement and Innovation Partnership (QIIP)

Abstract

Objective

Primary healthcare (PHC) quality improvement (QI) initiatives are designed to improve patient care and health outcomes. We evaluated the Quality Improvement and Innovation Partnership (QIIP), an Ontario-wide PHC QI program on access to care, diabetes management and colorectal cancer screening. This manuscript highlights the impact of QIIP on diabetes outcomes and associated vascular risk factors.

Research design and methods

A cluster matched-control, retrospective prechart and postchart audit was conducted. One physician per QIIP-PHC team (N=34) and control (N=34) were recruited for the audit. Eligible charts were reviewed for prespecified type 2 diabetes mellitus clinical process and outcome data at baseline, during (intervention range: 15–17.5 months) and post. Primary outcome measures were the A1c of patients above study target and proportion of patients with an annual foot exam. Secondary outcome measures included glycemic, hypertension and lipid outcomes and management, screening for diabetes-related complications, healthcare utilization, and diabetes counseling, education and self-management goal setting.

Results

More patients in the QIIP group achieved statistically improved lipid testing, eye examinations, peripheral neuropathy exams, and documented body mass index. No statistical differences in A1c, low-density lipoprotein or systolic/diastolic blood pressure values were noted, with no significant differences in medication prescription, specialist referrals, or chart-reported diabetes counseling, education or self-management goals. Patients of QIIP physicians had significantly more PHC visits.

Conclusion

The QIIP-learning collaborative program evaluation using stratified random selection of participants and the inclusion of a control group makes this one of the most rigorous and promising efforts to date evaluating the impact of a QI program in PHC. The chart audit component of this evaluation highlighted that while QIIP improved some secondary diabetes measures, no improvements in clinical outcomes were noted. This study highlights the importance of formalized evaluation of QI initiatives to provide an evidence base to inform future program planning and scale-up.

Significance of this study.

What is already known about this subject?

• Primary healthcare quality improvement (QI) initiatives targeting type 2 diabetes mellitus (T2DM) are designed to improve diabetes care and associated health outcomes.

What are the new findings?

• The Quality Improvement and Innovation Partnership, a province-wide QI initiative in Ontario, Canada, improved some secondary T2DM measures (lipid testing, eye examinations, peripheral neuropathy exams, and documented body mass index); however, no improvements in T2DM clinical outcomes (A1c, low-density lipoprotein or systolic/diastolic blood pressure values) were noted.

How might these results change the focus of research or clinical practice?

• With resources in primary healthcare already strained by the rising economic and public health burden of T2DM, this research highlights the importance of rigorous and formalized evaluation of primary healthcare QI initiatives to provide an evidence base to inform future program planning and scale-up.

Introduction

The prevalence of diabetes is rising at an alarming rate.^{1 2} Given that 75% of patients in Ontario with diabetes receive care exclusively from their family physician (FP),³ the primary healthcare (PHC) system is being placed under immense pressure to perform well and meet clinical targets and patient needs. As more Canadians are diagnosed and receiving diabetes care, the cost of diabetes is also proving to be a significant burden on the economy.^{1 2} These combined stressors are placing strain on the PHC system, all of which are only expected to increase, hence, the need to develop innovative strategies for facilitating diabetes prevention and management in PHC.

In an effort to reform and improve PHC delivery, the Ontario Ministry of Health and Long Term Care (MOHLTC) established Family Health Teams (FHTs) in 2005.^4–6 These new models were envisioned as a multidisciplinary, team-based, patient-centered, and proactive healthcare delivery system focused on health promotion, disease prevention and chronic disease management. To help FHTs navigate this transition, the provincial Quality Improvement and Innovation Partnership (QIIP), now amalgamated with Health Quality Ontario (HQO), was established in 2005 by the MOHLTC. The overall goal of QIIP was to enable PHC teams to enhance the care experience and outcomes of the populations they serve through continuous quality improvement (QI) in effective, efficient, accessible, comprehensive, patient-centered and team-based PHC.

QIIP program description

From 2008 to 2010, QIIP used the Institute for Healthcare Improvement’s Breakthrough Series (IHI-BTS) Model⁷ to implement a QI learning collaborative (LC) program with teams of FPs, allied health professionals, and administrative staff from FHTs and community health centers (CHCs) across Ontario. IHI-BTS uses an adult learning model that typically brings together teams to learn from each other and experts in topic areas targeted for improvements.⁷ The overall purpose of QIIP was to educate, train and enable PHC teams to improve patient care and outcomes in three areas: (1) chronic disease management—program target: type 2 diabetes mellitus (T2DM) management; (2) disease prevention—program target: colorectal cancer (CRC) screening; and (3) office access and efficiency—program targets: ‘advanced access’ to healthcare and team functioning.

The QIIP-LC program was delivered over three waves; each wave was standardized in content and consisted of prework, three 2-day learning sessions (separated by action periods) and one summative congress. This ensured all participants received the same program. In total, each wave of the program lasted between 15.5 and 17 months. Learning sessions provided opportunities for teams to learn QI methodology such as Plan-Do-Study-Act cycles⁸ and Ontario’s Chronic Disease Prevention and Management framework.⁹ Learning sessions also provided a rich opportunity to network and learn from each other. The QIIP-LC program included support from QI coaches (QIC) whose role was to facilitate and mentor participants throughout the program and support electronic medical record optimization.¹⁰ A series of communication strategies or channels included a virtual office, list serve and monthly teleconferences between PHC teams and their QICs. Participation in QIIP was voluntary and open to PHC organizations across Ontario. More details about the QIIP-LC program characteristics are available in the program logic model (online supplementary file S1) and published in Harris et al.¹¹

Research design and methods

An external evaluation of the QIIP-LC program was conducted using a rigorous mixed-method, multimeasure design. The evaluation included: (1) development of a logic model (see online supplementary file S1) and assessment oriented process evaluation; (2) a retrospective, cluster, matched-control, prechart and postchart audit on the management of T2DM and rate of CRC screening; (3) a controlled post-only survey of practices participating in the chart audit on advanced access to healthcare; (4) semistructured, post-only, in-depth telephone interviews; (5) post-only web-based participant survey; and (6) health administrative data analysis with the Institute for Clinical Evaluative Sciences.¹¹

Primary outcome results¹¹ and the impact of QIIP on team functioning¹² are published elsewhere. This paper presents the results of the chart audit related to the impact of the QIIP-LC program on T2DM management, including: (1) glycemic outcomes and management; (2) vascular protection (hypertension and dyslipidemia) and (3) screening for DM-related complications. This paper also presents the chart audit findings related to healthcare utilization (healthcare professional (HCP) visits, specialist referral for diabetes care) and diabetes counseling, education and self-management goal setting.

Evaluation of the QIIP-LC was approved by the Research Ethics Board at Western University and Queen’s University. A waiver of patient consent for the chart audit was granted under the Ontario Personal Health Information Protection Act from each Ethics Review Board.

Design and participants

A retrospective, cluster, matched-control prechart and postchart audit was conducted for this component of the evaluation. PHC teams were randomly selected from a sampling frame based on the proportional distribution of participating teams: (1) model of care (academic, FHT and CHC); (2) wave of LC; (3) geographical region (Local Health Integration Network (LHIN) boundaries); and (4) practice setting (rural/urban). LHINS are geographical areas responsible for the regional administration of public healthcare services in the province of Ontario, Canada. Created 1 April 2007, they are mandated with planning, integrating, and distributing provincial funding as well as engaging with their local communities.¹³

One randomly selected physician per team (N=34) and matched control (N=34) were recruited. Control physicians were identified based on the sampling framework of their matched QIIP-LC physician using a pragmatic priority approach selecting controls. When identifying controls matched to a QIIP-LC physician within an FHT, ideally controls were ranked according to distance (prioritizing control physicians geographically further away to reduce contamination). For CHCs, controls were identified from alternate CHC practices, as most CHCs in Ontario operate in one practice location. In the event that an appropriate control physician did not exist, physicians from alternate FHT practices or community practices were recruited.

Physicians were included in the evaluation if they were in active clinical practice at least 1 year prior to the program start and had a minimum of 20 patients with T2DM in their practice. Physicians meeting eligibility criteria generated a list of patients with T2DM using ICD-9 250 billing code (International Classification of Diseases, Ninth Revision). Deceased patients and those residing in a nursing homes were excluded.

External, trained auditors were assigned to a physician. Auditors were not informed whether physicians were part of the QIIP-LC program or control group. Standard data abstraction forms were used. Auditors randomly selected patients from the list and screened for eligibility. Eligible patients were: (1) ≥18 years of age; (2) diagnosed with T2DM; and (3) diagnosed prior to the start of the audit timeframe. Eligible charts were reviewed for prespecified T2DM clinical process and outcome data 12 months prior to the LC (baseline), during the LC (intervention range: 15–17.5 months), and 12 months after the LC (post). Auditors were instructed to review charts until 12 eligible patients per physician were reached. Not all physicians had 12 eligible patients to contribute; therefore, the first dataset totaling 809 patients was used for analysis on all diabetes outcome measures. For analysis of glycemic outcomes, a second dataset was constructed of patients above study target A1c (≥7.3%). Not all physicians had five patients (from the original 12) above study target A1c; therefore, auditing continued until a minimum of 5 at baseline was met. The second dataset had a total of 310 patients. Data collection dates for control physicians were based on the time frame of their matched QIIP physician.

Primary outcome measures

Two primary outcome measures were calculated to evaluate the effectiveness of the QIIP-LC program on T2DM management. The primary clinical outcome was the A1c of patients above study target, while the primary process outcome was the proportion of patients with an annual foot exam. Investigators defined A1c to be above target for this study as 7.3% rather than the Canadian Diabetes Association Clinical Practice Guideline (CPG) of 7%¹⁴ in order to identify patients in whom glycemic treatment intensification was more likely.

Secondary outcome measures

Secondary clinical measures included glycemic outcomes and management (A1c value, A1c at CPG target, oral hypoglycemic agent (OHA)/insulin medication prescription), hypertension outcomes and management (systolic/diastolic blood pressure (BP) values, BP at CPG target and antihypertensive medication prescription) and lipid outcomes and management (low-density lipoprotein (LDL) value, LDL at CPG target and statin medication prescription). Intensification of glycemic, hypertension and lipid management were also included as secondary clinical outcome measures. Intensified glycemic management was characterized by adding an OHA, increasing the total daily dose of an OHA, adding insulin and/or increasing the total daily dose of an insulin. Intensified hypertension management was characterized by adding an antihypertensive and/or increasing the total daily dose of an antihypertensive. Intensified lipid management was characterized by adding a statin, increasing the total daily dose of a statin and/or switching statin medication to atorvastatin or rosuvastatin (higher potency statins).

Secondary process measures were the completion rate of screening for other diabetes-related complications (BP, lipid profile, albumin:creatinine ratio, glomerular filtration rate, serum creatinine, foot exam, eye exam, peripheral neuropathy exam, ECG exam, waist circumference documented and body mass index (BMI) documented). Visits to the PHC team were documented by date and provider type to determine the total number of visits to all HCPs. Documentation of diabetes counseling and education (exercise, weight, diet, smoking cessation, hypoglycemic events and adjustment of treatment plans) and self-management goals were collected by date. Specialist referrals were documented by date and type.

Sample size

Sample size calculations were conducted taking into account the sample size requirements for each primary outcome and adjusting for clustering and loss to follow-up. The final number of physicians required per group was 33.¹¹

Analysis

Analysis was performed on all outcomes using SAS V.9.2.¹⁵ Three time periods were constructed for each outcome variable: (1) 12 months prior to the LC (baseline); (2) during the LC (during); and (3) 12 months after the LC (post). The generalized linear model (Proc Genmod in SAS) was used to compare change in outcome measures over time (baseline–during–post) between the QIIP-LC physicians (hereafter reported as QIIP group) and the control group, accounting for clustering within the physician’s practice and controlling for baseline measures.

Process/dichotomous outcome variables (eg, foot exam) were considered complete if documented in patient charts at least once during each time period. For continuous outcome variables (eg, A1c value), the most recent documented value during each time period was used. Missing values were populated with data carried forward from the previous time period. Other continuous outcome variables such as the number of visits and number of HCPs seen were calculated from the data set.

For each time period, the most recently prescribed medications and the total daily dose for each medication was used to construct treatment intensification variables by medication category (oral antihyperglycemic agents, insulin, antihypertensives and statins). Treatment intensification was determined by comparing baseline medication to the medication regimen during the LC or 12 months post-LC.

Results

Baseline and demographic characteristics of physicians and their patients are provided in table 1. Patient demographics were comparatively equal between groups, with the exception of mean age at diagnosis for the subset of patients with A1c ≥7.3% at baseline (mean age QIIP=54.7; control=50.4 years, p=0.01).

Table 1.

Demographics of physicians and their patients with type 2 diabetes mellitus

Physician demographics
	QIIP (N=34)	Control (N=34)
Physicians, N	34	34
Females, %	41.2	44.1
Mean years in practice (SD)	23.1 (10.51)	21.2 (9.07)
Rural, %	52.9	52.9
Family health team model of care, %	82.4	82.4

Patient demographics
	All patients						Patients with A1c ≥7.3% at baseline
	QIIP (N=406)			Control (N=403)			QIIP (N=153)			Control (N=157)
	N	%		N	%		N	%		N	%
Females, %	406	46.6		403	50.9		153	45.8		157	49.0
Smoking status (% patients who smoke)	333	21.3		294	23.5		128	17.2		113	21.2
	N	Mean (SD)		N	Mean (SD)		N	Mean (SD)		N	Mean (SD)
Age (years)*	406	62.3 (11.4)		403	62.9 (12.3)		153	61.9 (11.4)		157	60.5 (12.6)
Age at diabetes diagnosis (years)†	316	56.0 (12.4)		324	55.9 (13.0)		119	54.7 (11.9)		120	50.4 (12.8)
	N	Mean (SD)	Median	N	Mean (SD)	Median	N	Mean (SD)	Median	N	Mean (SD)	Median
Duration of diabetes (years)‡	316	6.2 (6.3)	4.4	324	6.0 (5.6)	4.4	119	6.9 (5.9)	5.1	120	8.4 (6.8)	7.0

*Age calculated from year of birth to start of baseline.

†Age at diabetes diagnosis for patients with A1C ≥7.3% at baseline, significant p value, p=0.01.

‡Duration of type 2 diabetes calculated from year of diagnosis to start of baseline; no significant differences.

QIIP, Quality Improvement and Innovation Partnership.

Comorbidities and diabetes-related complications of patients are presented in table 2. Most patients were diagnosed with at least one comorbidity (QIIP=95.8%; control=95.8%). More patients in the QIIP group were diagnosed with dyslipidemia (p=0.03), with no other significant differences found. QIIP group patients above study target A1c were more likely to be diagnosed with dyslipidemia (p=0.03) and depression (p=0.004). Statistically significant baseline and demographic characteristics were included as covariates during the analyses, with no significant effect on outcomes.

Table 2.

Comorbidities and complications of patients with type 2 diabetes mellitus

	All patients		Patients with A1c≥7.3% at baseline
	QIIP (N=406) (%)	Control (N=403) (%)	QIIP (N=153) (%)	Control (N=157) (%)
Comorbidity	95.8	95.8	98.0	95.5
Diabetes-related complication	54.7	53.9	56.9	58.6
Comorbidity
Hypertension*	68.7	67.3	66.0	70.1
Dyslipidemia†,‡	88.7	81.9	92.2	82.8
Depression§	18.0	13.9	24.8	12.7
Obesity	26.6	30.5	28.8	36.3
Arthritis	21.9	14.1	24.2	15.3
Respiratory disease	12.3	15.9	10.5	10.2
Hypothyroid	6.7	7.2	5.9	8.9
Diabetes-related complications
Cardiovascular disease	31.3	26.6	35.3	29.3
Nephropathy	15.3	14.4	14.4	19.1
Neuropathy	8.1	8.2	11.1	10.2
Retinopathy	5.7	3.5	9.2	4.5
Other eye disease	17.5	21.3	17.0	15.9
Diabetic foot disease	0.5	2.5	0.7	2.6
Amputations	1.2	0.7	3.3	0.6
Skin disease	2.5	0.7	1.3	0
Erectile dysfunction	6.9	8.9	8.5	9.6
Other	28.8	24.3	28.1	19.8

*Definition: condition documented in chart; when adding whether antihypertensive medication prescribed the total=90.5% (732/809), QIIP=92.1% (374/406), control=88.8% (358/403).

†Significant for the subset of patients above study target A1c, p=0.004.

‡Definition: condition documented in chart and/or antihypertensive/lipid-lowering medication prescribed.

§Significant for all patients and the subset of patients above study target A1c, p=0.03.

QIIP, Quality Improvement and Innovation Partnership.

Primary outcome results

Mean A1c was significantly lower in the QIIP group during the 12-month program (p=0.01); however, these improvements were not sustained post-LC (p=0.10, table 3). No difference between groups was observed for foot exams (p=0.15, table 4).

Table 3.

Glycemic, hypertension and lipid outcomes and management

		Baseline 12 months prior to LC		During 12 months during LC		Post 12 months after LC		p Value over time
		Mean (SD)	Median	Mean (SD)	Median	Mean (SD)	Median
Glycemic outcomes and management, patients with A1c ≥7.3%(QIIP: N=153; control: N=157)
A1c value, %	QIIP	8.5 (1.25)	8.1	8.1 (1.32)	7.8	8.2 (1.62)	7.9	0.10
	Control	8.5 (1.22)	8.0	8.4 (1.51)	8.0	8.4 (1.58)	8.1
		%		%		%		p Value*
A1c at CPG target (≤7.0%)	QIIP	No patients at target		22.2		26.1		0.75
	Control			15.3		16.6
Prescribed an OHA	QIIP	84.3		88.2		90.2		0.99
	Control	82.8		87.9		87.3
Prescribed insulin	QIIP	28.8		39.2		47.1		0.24
	Control	35.0		42.7		49.7
Hypertension outcomes and management, all patients(QIIP: N=406; control: N=403)
		N	Mean (SD)	N	Mean (SD)	N	Mean (SD)	p Value over time
Systolic BP value, mm Hg	QIIP	344	129.8 (16.24)	394	131.0 (17.17)	400	130.7 (16.56)	0.72
	Control	354	132.0 (17.50)	387	130.7 (17.88)	399	131.7 (16.27)
Diastolic BP value, mm Hg	QIIP	344	74.0 (9.49)	394	73.6 (9.82)	400	73.0 (9.36)	0.92
	Control	354	74.9 (10.81)	387	73.5 (10.73)	399	73.7 (10.50)
		N	%	N	%	N	%	p Value over time
BP at CPG target (≤130/80)	QIIP	344	53.8	394	52.3	400	52.8	0.50
	Control	354	48.0	387	50.9	399	46.6
Prescribed an AHTN	QIIP	406	79.1	406	87.0	406	89.2	0.22
	Control	403	79.4	403	84.9	403	86.3
Lipid outcomes and management, all patients(QIIP: N=406; control: N=403)
		N	Mean (SD)	N	Mean (SD)	N	Mean (SD)	p Value over time
LDL cholesterol value, mmol/L	QIIP	284	2.2 (0.80)	369	2.1 (0.80)	385	2.0 (0.74)	0.32
	Control	273	2.4 (0.82)	353	2.3 (0.86)	382	2.2 (0.79)
		N	%	N	%	N	%	p Value over time
LDL cholesterol at CPG target (≤2.0)	QIIP	284	43.0	369	52.6	385	59.2	0.03
	Control	273	34.8	353	40.5	382	44.5
		%		%		%		p Value over time
Prescribed a statin	QIIP	67.0		78.6		84.2		0.97
	Control	59.1		72.5		74.2

Intensification of glycemic, hypertension and lipid management†
		Post from baseline (%)	p Value
Intensified glycemic management: (A) added an OHA, (B) increased total daily dose of an OHA, (C) added insulin, AND/OR (D) increased total daily dose of an insulin	QIIP	71.2	0.30
	Control	65.6
Intensified hypertension management: (A) added an antihypertensive AND/OR (B) increased total daily dose of an antihypertensive	QIIP	51.2	0.08
	Control	44.2
Intensified lipid management: (A) added a statin, (B) increased total daily dose of a statin, AND/OR (C) switched statin medication to atorvastatin/rosuvastatin	QIIP	39.2	0.21
	Control	33.8

*Comparing QIIP and control between the 12 months during the LC and 12 months post-LC only.

†Intensification is a post-only variable; management status at post is compared with status at baseline.

A1c, haemoglobin A1c or HbA1c; AHTN, anti-hypertensive medication; BP, blood pressure; CPG, clinical practice guidelines; LC, learning collaborative; LDL, low density lipoprotein; OHA, oral antihyperglycemic; QIIP, Quality Improvement and Innovation Partnership.

Table 4.

Screening for diabetes-related complications (QIIP N=406; control N=403)

		Baseline 12 months prior to LC (%)	During 12 months during LC (%)	Post 12 months after LC (%)	p Value over time
BP	QIIP	84.7	92.1	87.2	0.12
	Control	87.8	88.1	88.6
Lipid profile*	QIIP	72.7	84.0	76.9	0.02
	Control	72.0	73.7	72.7
ACR tested	QIIP	48.0	53.0	48.3	0.08
	Control	40.7	37.7	37.5
eGFR tested	QIIP	57.1	69.5	69.0	0.39
	Control	55.6	64.0	65.0
Serum creatinine	QIIP	73.7	83.5	80.5	0.07
	Control	73.2	76.4	75.9
Foot exam	QIIP	28.6	49.3	48.5	0.15
	Control	24.8	35.0	38.0
Eye exam	QIIP	22.4	37.9	37.2	0.03
	Control	18.1	20.8	27.5
Peripheral neuropathy exam†	QIIP	14.8	29.3	31.3	0.01
	Control	22.8	25.1	26.6
ECG exam	QIIP	20.4	25.9	25.4	0.22
	Control	14.1	14.4	19.1
Waist circumference documented	QIIP	7.1	21.4	25.6	0.23
	Control	8.7	15.9	20.4
BMI documented	QIIP	29.1	46.8	50.3	0.04
	Control	36.7	40.2	42.9

*Lipid profile includes: triglycerides, HDL, LDL, total cholesterol:HDL, total cholesterol.

†Vibration or 10 g monofilament testing.

ACR, albumin:creatinine ratio; BMI, body mass index; BP, blood pressure; eGFR, estimated glomerular filtration rate; LC, learning collaborative; QIIP, Quality Improvement and Innovation Partnership.

Secondary outcome results

No significant changes in other glycemic outcomes and management were observed between groups (in the subset of patients above study target (A1c ≥7.3%)) (table 3). Similarly, no changes in vascular protection from baseline were observed for hypertension outcomes and management (BP management and medication, table 3), while lipid outcomes and management highlighted that more patients in the QIIP group achieving CPG target LDL cholesterol (≤2.0) over time compared with the control group (p=0.03). No other significant differences in lipid management between groups were found. Table 4 presents the screening for diabetes-related complications. Lipid profile testing (p=0.02), eye examinations (p=0.03), peripheral neuropathy exams (p=0.01), and documented BMI (p=0.04) were significantly higher for the QIIP group. Patients in the QIIP group had significantly more visits to HCPs from baseline compared with the control group (p=0.001, table 5). There were no differences between groups for the number of HCP seen (p=0.39) or the number of patients who saw more than one HCP per visit on average (p=0.89) (an indicator of team-based patient care). QIIP group referrals to a specialist decreased from 31.5% (pre) to 28.8% (post), while control group referrals increased slightly from 28.5% (pre) to 30.0% (post); however, these differences were not statistically significant, p=0.08. The proportion of patients who had documented counseling/education for diabetes (weight, diet, exercise, hypoglycemic events, treatment plan, smoking cessation, or self-management goals) over the study period was examined, with no significant differences between groups (p=0.43).

Table 5.

Patient visits to the primary healthcare team, all patients (QIIP N=406; control N=403)

		Baseline 12 months prior to LC		During 12 months during LC		Post 12 months after LC
		Mean (SD)	Median	Mean (SD)	Median	Mean (SD)	Median
Number of HCP visits*	QIIP	6.5 (6.11)	5.0	7.6 (6.44)	6.0	7.2 (7.20)	5.0
	Control	5.8 (4.68)	5.0	5.6 (4.67)	5.0	5.6 (4.82)	5.0
Visit with…		%		%		%
Family physician or resident	QIIP	94.6		96.1		92.6
	Control	94.3		95.3		93.3
Nurse practitioner	QIIP	16.3		17.0		22.4
	Control	6.5		8.2		6.7
Nurse (RN or RPN)	QIIP	23.4		27.3		24.4
	Control	14.4		20.6		21.1
Diabetes nurse educator	QIIP	4.9		15.3		12.3
	Control	7.9		10.7		12.4
Dietitian	QIIP	9.9		14.3		11.8
	Control	7.0		7.2		6.2
Social worker	QIIP	0.7		2.5		3.5
	Control	1.5		3.2		2.7
Pharmacist	QIIP	2.5		5.7		5.4
	Control	0.5		1.5		3.5
Other	QIIP	4.7		7.6		5.7
	Control	2.0		2.5		5.5

*HCP visit defined as the total number of visits to all HCPs, significant p=0.001 over time.

HCP, healthcare provider; LC, learning collaborative; RN, registered nurse; RPN, registered practical nurse.

Discussion

The rigorous design and methodology of the QIIP-LC program evaluation using a stratified random selection of participants and a control group is the most significant and promising effort to date demonstrating the impact of a QI program in PHC. The chart audit component of this evaluation established the success of the QIIP-LC program at significantly increasing a number of important diabetes process measures in the QIIP group, including lipid profile testing, eye examinations, peripheral neuropathy exams and BMI measurement. Furthermore, more patients in the QIIP group achieved CPG target LDL cholesterol (≤2.0) over time compared with the control group. No statistical differences in key diabetes outcome measures were noted, including A1c, LDL or BP values. It also appears there were no significant differences in how medications for these conditions were prescribed, specialist referrals, or chart reported diabetes counseling, education or self-management goals, even though patients of QIIP-LC participants had significantly more HCP visits.

Our findings are consistent with previous QI evaluation literature demonstrating improvements in diabetes clinical process measures, while the impact on important diabetes outcomes remains to be seen. Some recent studies showed no improvement in A1c, LDL, and BP, while other results vary depending on the length of the postperiod.^16–23 Similarly, improvement in the intensification of diabetes medications has been demonstrated elsewhere; however, no improvements were observed in our evaluation.16 18–20 24

It is important to note that patients from both QIIP and control physicians were well-controlled for BP and lipid management in comparison with recent findings from the national Diabetes Mellitus Status in Canada survey.²⁵ This survey reported significant treatment gaps in global vascular protection, with only 36% of patients with BP at CPG target (≤130/80/80 mm Hg) and 57% of patients with LDL cholesterol at CPG target (≤2.0 mmol/L). QIIP and control physicians had more patients at BP target post-LC (52.8% and 46.6%, respectively), with more QIIP patients at LDL target post-LC (59.2%). The relatively well-controlled patient population for both QIIP and control physicians potentially explains the lack of effect of the QIIP-LC on BP and lipid management simply for the reason that physicians experienced a ceiling effect, with the majority of their patients already in good control. These findings are supported by recent findings in Ontario²⁶ and the USA²⁷ demonstrating a significant reduction in cardiovascular disease in patients with diabetes. Similarly, trends in decreasing mortality rates for patients with diabetes have also been noted in Ontario, Canada, and the UK.²⁸ Reducing vascular risk for patients with diabetes is critical, especially when the aim is to reduce the overall risk of either a primary or secondary cardiovascular event. In this regard, the findings of this evaluation are a positive step in the right direction for patients with diabetes.

An interesting finding from this evaluation was the significantly higher number of HCP visits in the QIIP group. On one hand, this may suggest that efforts to use a team-based approach to care may have had some success, a finding consistent with the perceptions of some QICs and PHC participants; on the other hand, one could also speculate that an increase in healthcare utilization is resource intensive without demonstrating significant clinical improvements. Future research is necessary to gain a more nuanced understanding of healthcare utilization, including PHC team formation, approach and care delivery. Numerous authors have posited about the effectiveness of FHTs and interprofessional team-based care in Ontario for patients with diabetes and chronic disease,^29–32 suggesting that more time may be required for teams to form and learn to work together, and thus see significant differences in care such as diabetes clinical outcome measures.^29–31 A longitudinal research approach, with more time between team formation, QI initiatives and evaluation, may be beneficial for understanding the true impact of team-based care and QI programs like QIIP.

Strengths and limitations

The QIIP-LC program evaluation was designed to address the growing consensus in the literature for more rigorous study designs of QI interventions.^33–38 This evaluation expands the literature on QI evaluation methodology with its use of a control group, a rare study design in research and evaluation of QI programs. To maximize the effectiveness of the control and minimize the risk of contamination, control physicians were recruited using a pragmatic priority approach selecting controls ranked according to distance. Furthermore, collection of clinical chart audit data by independent audit rather than self-report heightened the strength of the findings, and the stratified random selection of participants, pre–post, multimeasure design added strength to this rigorous methodology. QIIP PHC teams were challenged to focus on more than one QI area at a time in a busy clinic setting,¹¹ and it is possible that diabetes was not the focus of teams assessed by chart audit for T2DM outcomes. QIIP had a number of program targets (T2DM, CRC screening, advanced access and team functioning), and it would be beneficial for future evaluations to have the capacity to assess the degree to which participants focused on each target area of the intervention.

Complex QI initiatives are challenging to evaluate, and the QIIP-LC program was no exception. The retrospective nature of the evaluation may have limited the controlled design, and use of the last observation carried forward technique to impute missing values in the dataset may have resulted in a biased estimate of the intervention, or underestimated the variability of the results. Future analyses would benefit from statistical techniques designed to understand the magnitude of this bias and complement existing methods for dealing with missing data. Furthermore, participation bias existed on two levels. Clinical teams volunteered to participate in the program suggesting their interest in improving their practice. This bias was amplified by virtue of informed consent to participate in the evaluation. Participants who refused may have done so based on their lack of interest in QI interventions or their perceived lack of change or improvements. The impact of the QIIP-LC program may have been diluted by a number of simultaneous Ontario MOHLTC provincial initiatives and mandates targeting improvements in diabetes management in primary care.^39–42 Design and implementation of QI programs would benefit from complimentary, rather than overlapping initiatives to optimize the impact of these programs. Finally, physicians and their respective teams were encouraged through the QIIP-LC to develop sustainability plans to integrate successful concepts into the practice and to share lessons learnt in the program to colleagues both within and external to the collaborative. Sustainability and spread of lessons learnt were key program components of the QIIP-LC program, and improvements in both QIIP and control groups could be an indication that changes initiated by QIIP-LC participants were integrated into the whole practice organization and thus, control practices within the same FHT may have adopted these new learnings.

Conclusion

The QIIP-LC program evaluation, including stratified random selection of participants and inclusion of control groups, is one of the most rigorous and promising efforts to date evaluating the impact of a QI program in PHC. While QIIP improved some diabetes process measures, no improvements in clinical outcome measures were noted. With resources in PHC already strained by the rising economic and public health burden of diabetes, this study highlights the importance of formalized evaluation of QI initiatives to provide an evidence base to inform future program planning and scale-up.