Effectiveness of Quality Incentive Payments in General Practice (EQuIP-GP) cluster randomized trial: impact on patient-reported experience

Abstract

Background

Relational continuity, ‘a therapeutic relationship between a patient and provider/s that spans health care events’, has been associated with improved patient outcomes.

Objectives

To evaluate whether an intervention incorporating patient enrolment and a funding model for higher-risk patients influenced patient-reported experience measures, particularly relational continuity.

Methods

Cluster-randomized controlled trial over 12 months (1 August 2018–31 July 2019). Participating patients within intervention practices were offered enrolment with a preferred general practitioner, a minimum of 3 longer appointments, and review within 7 days of hospital admission or emergency department attendance. Intervention practices received incentives for longer consultations (dependent on reducing unnecessary prescriptions and tests), early post-hospital follow-up, and hospitalization reductions. The primary outcome was patient-reported relational continuity, measured by the Primary Care Assessment Tool Short Form.

Results

A total of 774 patients, aged 18–65 years with a chronic illness or aged over 65 years, from 34 general practices in metropolitan, regional, and rural Australia across 3 states participated. Response rates for questionnaires were >90%. From a maximum of 4.0, mean baseline scores for relational continuity were 3.38 (SE 0.05) and 3.42 (SE 0.05) in control and intervention arms, respectively, with no significant between-group differences in changes pre-post trial. There were no significant changes in other patient-focussed measures.

Conclusion

Patient-reported relational continuity was high at baseline and not influenced by the intervention, signalling the need for caution with policies incorporating patient enrolment and financial incentives. Further research is required targeting at-risk patient groups with low baseline engagement with primary care.

Key messages.

• There is scant evidence concerning interventions to improve relational continuity.

• This trial tested patient enrolment and practice incentives to improve continuity.

• The intervention did not improve patient-reported relational continuity of care.

• Research is needed in patient groups with low engagement with general practice.

Background

An optimal model of funding for primary care, which supports improved population health, patient, and provider satisfaction and achieves a reduction in overall health care expenditure,¹ is the subject of intense policy interest, research, reports, and recommendations.^2,3 In particular, there are concerns in Australia that current funding mechanisms, which are based primarily on fee-for-service, do not adequately meet the needs of people living with chronic or complex conditions or multimorbidity.² In response to the need for robust evidence to inform health policy, the Royal Australian College of General Practitioners (RACGP), in conjunction with the Australian Government Department of Health, supported 2 ‘Quality in General Practice’ trials, to test alternative primary care funding mechanisms. A focus was on supporting quality of care for older persons and adults with chronic illness. The RACGP had a particular interest in exploring whether funding mechanisms could enhance relational continuity of care. Relational continuity represents one of the 3 core components of continuity of care (the others being management and informational continuity).⁴ Relational continuity represents “a therapeutic relationship between a patient and one or more providers that spans various healthcare events and results in accumulated knowledge of the patient and care consistent with the patient’s needs”.⁵ Valued by patients,⁶ relational continuity in primary care is associated with reduced hospitalizations,⁷ lower referral rates,⁸ reduced mortality,⁹ and lower overall health care costs.¹⁰ The literature also supports an association between longer consultations and enhanced preventive care, reduced prescribing rates and higher patient enablement.¹¹ In addition, there is some evidence that timely follow-up after hospital discharge is associated with reduced readmissions and mortality, particularly for people with cardiac failure or chronic obstructive pulmonary disease (COPD).¹² Thus, the RACGP wished to test models that encouraged relational continuity of care, longer consultations, and structured post-hospital discharge follow-up and examine the outcomes of patient experience, resource utilization, and costs. There are very few randomized control trials (RCTs) in the literature of funding models to encourage these care processes.

In the EQuIP-GP trial, we tested an intervention incorporating patient enrolment and relevant practice-level financial incentives. The incentives were intended to complement existing arrangements for payments for clinical services in Australia. The payment model was designed following the net benefit correspondence theorem¹³ to create continuous quality of care improvement incentives relative to expected downstream cost savings from improved quality and within existing overall health system budgets. The study protocol and financial incentives model have been previously described.¹⁴ We hypothesized that relational continuity for participating patients would be encouraged by core components of the intervention: enrolment with a preferred primary care physician (general practitioner [GP] in Australia); access to longer consultations with that GP; and timely follow-up after hospitalization. This paper reports the primary trial outcome of patient-reported relational continuity for adult participants. Patient-reported measures of relational continuity are argued to better correspond with the mechanisms that may contribute to its favourable outcomes (e.g. the GP having personal knowledge of the patient) than concentration of care measures (e.g. the proportion of visits with the patient’s usual GP).⁹ We also report secondary outcomes of patient-reported access, coordination and comprehensiveness of care, and self-assessed health. The incentive implementation and health system resource outcomes will be reported separately.

Methods

Design

The design was a pragmatic 2-arm cluster RCT, with 1:1 intervention/control allocation. Cluster randomization was at the practice level to reflect the practice-level intervention. The lead university received ethics approval for the study from the University of Wollongong Human Research Ethics Committee (2017/417), with subsequent approval by Monash University and the University of Tasmania. The trial was registered on 23 January 2018 on the Australian New Zealand Clinical Trials Registry: ACTRN12618000105246.

Participants and setting

The trial setting was the practice networks of the 3 collaborating university departments of general practice (family medicine), based in regional and rural NSW, metropolitan Victoria, and regional and rural Tasmania, Australia.

The eligibility criteria for practices were as follows:

• delivering generalist primary medical care and employing at least one full-time equivalent GP;

• being in business for at least 1 year and not intending to close for a further 2 years;

• being able to generate patient encounter data through Medical Director or Best Practice clinical software (the major software systems in Australian general practice);

• consenting to the use of National Prescribing Service MedicineWise MedicineInsight clinical data extraction software; and;

• not registered as participants or potential participants in the Australian Government Health Care Homes trial.¹⁵

The eligibility criteria for adult participants were as follows:

• active patients (attended practice 3 or more times in the last 2 years);

• age 18–65 years with COPD, diabetes, angina (or ischaemic heart disease), cardiac failure or asthma, or age over 65 years;

• able to read and write in English;

• no significant cognitive deficit; and

• not in significant distress.

Intervention

The active trial ran from 1 August 2018 to 31 July 2019. The intervention was at the practice level and comprised, first, patient enrolment with a preferred GP within the practice. As at the time of writing, Australia does not have a formal process for enrolment of patients with general practices or providers within practices. The trial provided for a ‘de-facto’ enrolment of patients with a nominated GP provider within the practice. The practice would seek to arrange appointments for the participant patient with that GP where practicable. Intervention practices agreed to guarantee enrolled patients access to a minimum of 3 longer GP appointments, and review within 7 days of admission to an emergency department or hospital over the study period. Incentives were paid to intervention practices for consultation lengths exceeding 15min (dependent on reducing potentially unnecessary prescriptions and tests),¹⁴ early post-hospital follow-up, and reducing hospitalization rates. See Table 1 for a summary of the incentive structure.

Table 1.

Incentive structure for intervention practices in Australian EQuIP-GP Trial (2018–2019).

What practices were asked to do	What practices would be paid
Provide 3 longer consults (over 15min) per enrolled patient and reduce unnecessary prescriptions, pathology, and imaging. Reductions measured across all prescriptions, plus across specified pathology and imaging tests.	Paid for every extra minute above 15min, calculated on the mean consultation time across the cohort of enrolled patients. The rate of pay per minute is adjusted according to the overall proportion of reduction in scripts, pathology, and imaging. Capped at $250 per patient. Rate of service use reduction
	5%	10%		15%		20%		25%
	Payment per extra minute
	60c	$1.20		$1.80		$2.40		$3
See minimum of 70% enrolled patients within 1 week of hospital discharge.	Paid on sliding scale according to percentage of patients seen within one week of discharge. Seen within 1 week
	70%		80%		90%		100%
	Payment per patient
	$0		$30		$60		$90
Reduce hospitalizations by up to 40% for enrolled patients.	Paid on sliding scale according to reduction in hospitalizations achieved. Rate of hospitalization reduction
	10%		20%		30%		40%
	Payment per patient
	$50		$100		$150		$200

An intervention facilitator (IF) was employed by each of the 3 collaborating universities to assist intervention practices in understanding and implementing the funding model into usual practice. Each intervention practice received 3 scheduled IF visits over the first 3 months of the trial. Control practices provided usual care. Both intervention and control practices were provided access to links to quality improvement education materials.

Control

Practices randomized to the control group provided treatment as usual and did not receive the financial incentives.

Outcome measures

The primary outcome was the difference between intervention and control groups in the change pre-post trial for the mean score of the relational continuity sub-scale of the Primary Care Assessment Tool Short Form (PCAT-S),¹⁶ at the level of the individual participant. Questions for the relational continuity sub-scale (titled “ongoing care” in the PCAT-S) included “When you go to your Principal Care Provider (PCP), are you taken care of by the same doctor or nurse each time?”; “If you have a question, can you call and talk to the doctor or nurse who knows you best?”; “Does your PCP know you very well as a person, rather than as someone with a medical problem?”; and “Does your PCP know what problems are most important to you?” The term “Principal Care Provider” was defined earlier in the PCAT-S questionnaire as “a doctor or place that you usually go if you are sick or need advice about your health and/or a doctor or place that knows you best as a person and/or a doctor or place that is most responsible for your health care.”

Secondary outcome measures included between-group differences in change in the mean scores for accessibility, coordination, and comprehensiveness of care using the relevant sub-scale of the PCAT-S. Each PCAT-S sub-scale consists of four 4-point Likert-type response items, with each sub-scale scored using the mean of item scores, and a maximum sub-scale score of 4.0.¹⁷ The relational continuity sub-scale of the PCAT-S has 3 items that have been shown to have good discriminatory ability in measuring provider accumulated knowledge of the patient, and one item with adequate discriminatory ability for concentration of care (proportion of time the same doctor is seen).¹⁸ Self-rated health was measured using the Visual Analogue Scale of the EQ-5D-5L health-related quality of life questionnaire (EQ-VAS),¹⁹ scored from 0 to 100. This provided a brief, validated self-reported health-status measure to minimize responder burden in an extensive questionnaire.¹⁹ Scores for the EQ-VAS are positively correlated with the EQ-5D-5L index scores, with less ceiling effect, making it more suitable for a population health measure than the index scores.¹⁹ EQ-VAS scores are positively correlated with both the physical and mental summary scores of the widely used Medical Outcomes Study Short Form 12-item Health Survey (SF-12), but faster to complete.²⁰ Patient participants were surveyed at their entry into the trial and then at trial completion, administered via online or paper-based self-completion or by telephone interview.

The standard deviation of the primary outcome, the relational continuity sub-scale, has previously been reported as 0.70, when assessed in primary care patients in Canada.²¹ For power calculations for the primary outcome, we assumed an intra-cluster correlation coefficient of 0.02 (interquartile range 0.01–0.04),²² loss of 4 practices, and 25% attrition of patients from the remaining practices. Thus, we aimed to recruit 1,080 adult participants across 36 practices, with an end-of-trial target of 32 practices and 720 adults, to detect a change of 0.20 in the mean score of the PCAT relational continuity sub-scale with 98% power.

Recruitment

Recruitment of practices occurred between April and August 2018. Eligible patients of consenting GPs within randomized practices were then recruited to the study between May and December 2018. To identify potentially eligible patients, the research team assisted practice staff to conduct electronic health record searches for patients aged 18–65 years with a specified chronic condition or patients aged over 65 years, seen by participating GPs retrospectively from the search date until the target number of invitations was met. Participating GPs screened the invitation lists to exclude patients they considered unable to sufficiently understand English to participate or with significant cognitive impairment or distress. Practices then posted out an invitation, information, and consent pack to 60 eligible patients in each patient group (120 in total). In 8 practices, where recruitment was low, a second tranche of invitations was distributed to a further sample of 120 adults. It was also permissible for practices to opportunistically recruit eligible patients.

Randomization

Practices were randomized following consent, using dynamic randomization by minimization. We used block randomization stratified within the 3 states by the Australian Standard Geographical Classification Remoteness Area (ASGC-RA: RA1—Major Cities of Australia, RA2—Inner Regional Australia; RA3—Outer Regional Australia).²³ NSW recruitment was stratified by RA1 and RA2, and Tasmanian recruitment was stratified by RA2 and RA3, while Victorian geographic stratification was not necessary as it was restricted to urban RA1 practices. We stratified practices by Index of Relative Socioeconomic Disadvantage (IRSD),²⁴ at each practice’s Local Government Area (LGA) matching within blocks on whether in the top 50% IRSD deciles or not. Additionally, we stratified by practice size, dichotomized to above and below Australian median practice size (≤5 or ≥6 GPs).²⁵ Randomization was conducted by the trial statistician (MJB). The project officers in each state entered a code for each consented practice into a cloud-based database in the order that the practice consents were received and notified the statistician by email. The statistician then entered the randomization sequence into the database. The state project officers notified the general practices of their group allocation. The statistician remained blinded to the intervention allocation until after the primary analyses were performed.

Statistical analysis

Mean baseline PCAT sub-scale and EQ-VAS results were compared with results at the completion of the trial. To account for the potential effects of clustering, we used hierarchical linear models with random main effects specified at the cluster level. Analysis was conducted on an intention-to-treat basis and participants who provided data at one time point only were included. Analysis was conducted using the mixed procedure in STATA (Version 16.1 StataCorp LLC, College Station, TX).

Results

Ultimately, of the 40 practices that were randomized, 34 participated in patient recruitment. These 34 practices recruited 774 patient participants after posting a total of 5,040 invitations. One intervention practice withdrew prior to the intervention commencing. Sixteen intervention practices with 364 participants, and 17 control practices with 371 participants (total n = 735) completed the trial, resulting in a lost to follow-up rate of 5%. The flow diagram of practice and participant recruitment, with losses and reasons, is presented in Fig. 1.

Fig. 1.

Recruitment flow diagram for intervention and control practices for Australian EQuIP-GP Trial (2018–2019).

The participating practice and practitioner characteristics, with comparison against the nationally representative Bettering the Evaluation and Care of Health (BEACH) sample,²⁵ are presented in Table 2.

Table 2.

General practitioner and practice characteristics of participants versus nationally representative characteristics BEACH 2015–2016.

	EQuIP-GP sample	Intervention	Control	BEACH 2015–2016
General practitioners n (% of column total)∗
Females	33 (40.7)	17 (40.5)	16 (41.0)	433 (44.9)
Males	48 (59.3)	25 (59.5)	23 (59.0)	532 (55.1)
Total	81 (100.0)	42 (100.0)	39 (100.0)	965 (100.0)
General practitioner years in practice n (% column total)#
<2 years	2 (2.6)	1 (2.6)	1 (2.6)	8 (0.8)
2–5 years	8 (10.5)	2 (5.3)	6 (15.8)	118 (12.3)
6–10 years	11 (14.5)	3 (7.9)	8 (21.0)	140 (14.6)
11–19 years	13 (17.1)	8 (21.0)	5 (13.2)	145 (15.2)
20+ years	42 (55.3)	24 (63.2)	18 (47.4)	546 (57.1)
Total	76 (100.0)	38 (100.0)	38 (100.0)	957 (100.0)
Remoteness Area Classification of practices (% of column total)
1 Major cities	17 (51.5)	9 (56.3)	8 (47.1)	661 (68.6)
2 Inner regional	10 (30.3)	4 (25.0)	6 (35.3)	215 (22.3)
3 Outer regional	6 (18.2)	3 (18.7)	3 (17.6)	72 (7.5)
4 Remote	0 (0.0)	0 (0.0)	0 (0.0)	12 (1.2)
5 Very remote	0 (0.0)	0 (0.0)	0 (0.0)	4 (0.4)
Total	33 (100.0)	16 (100.0)	17 (100.0)	964 (100.0)
Size of practice—number of individual general practitioners (% of column total)
Solo	1 (3.0)	1 (6.3)	0 (0.0)	77 (8.3)
2–4	7 (21.2)	2 (12.4)	5 (29.4)	226 (24.3)
5–9	16 (48.5)	11 (68.7)	5 (29.4)	360 (38.6)
10–14	7 (21.2)	1 (6.3)	6 (35.3)	167 (17.9)
15+	2 (6.1)	1 (6.3)	1 (5.9)	102 (10.9)
Total	33 (100.0)	16 (100.0)	17 (100.0)	932 (100.0)

Missing data n = 6.

Missing data n = 11.

The patient participant characteristics are displayed in Table 3.

Table 3.

Description of demographic characteristics of Australian EQuIP-GP trial 774 adult participants (2018–2019).

	Control arm	Intervention arm
	n (% of control totals)	n (% of intervention totals)
Adult participants recruited
Sex
Female adults	213 (55.9)	244 (62.1)
Male adults	168 (44.1)	149 (37.9)
Age
18–65 years old	143 (37.5)	164 (41.7)
Over 65 years	238 (62.5)	229 (58.3)
Total adults recruited	381 (100.0)	393 (100.0)
Adult participants after withdrawal
Sex
Female adults	209 (56.3)	230 (63.2)
Male adults	162 (43.7)	134 (36.8)
Age
18–65 years old	139 (37.5)	158 (43.4)
Over 65 years	232 (62.5)	206 (56.6)
Total adults after withdrawals	371 (100.0)	364 (100.0)

At baseline, 93.3% (n = 722) of the originally enrolled participants completed questionnaires, with 90.4% (n = 700) completing post-intervention questionnaires. Participants returned mean scores of 3.38 (control) and 3.42 (intervention) for relational continuity at baseline, with no significant change between groups over time. Mean scores for coordination were 3.43 (control) and 3.38 (intervention) at baseline, with no significant differences between groups over time. Mean scores for accessibility and comprehensiveness of care were between 2.74 and 2.98 at baseline, with no significant change between arms across the trial. The mean EQ-VAS scores at baseline were 72.3 for intervention and 72.0 for control groups, again with no significant across-trial changes (see Table 4).

Table 4.

Comparison of mean Primary Care Assessment Tool and EQ-VAS responses at baseline and follow-up between trial arms from the sample of 735 Australian EQuIP-GP trial participants (2018–2019).

PCAT sub-scale/EQ-VAS	Baseline mean score (SE)	Follow-up mean score (SE)	Baseline mean score (SE)	Follow-up mean score (SE)	Modelled mean difference (95% CIa)	Sig. P	ICCb
Trial arm	Intervention	Intervention	Control	Control
Relational continuity	3.42 (0.053)	3.48 (0.054)	3.38 (0.052)	3.41 (0.052)	−0.02 (−0.13, 0.09)	0.734	0.18
Accessibility	2.91 (0.052)	2.97 (0.053)	2.98 (0.051)	2.95 (0.051)	−0.11 (−0.24, 0.02)	0.089	0.06
Coordination	3.38 (0.046)	3.41 (0.047)	3.43 (0.046)	3.37 (0.046)	−0.08 (−0.22, 0.06)	0.277	0.03
Comprehensive care	2.74 (0.056)	2.79 (0.056)	2.74 (0.055)	2.81 (0.055)	0.04 (−0.06, 0.14)	0.450	0.07
EQ-VAS	72.32 (1.53)	74.33 (1.54)	72.02 (1.50)	74.04 (1.51)	0.02 (−2.38, 2.41)	0.990	0.05

confidence interval.

Intracluster correlation coefficient.

Discussion

This trial was designed to test an intervention that was intended to promote relational continuity of care. We posited that relational continuity for participating patients would be encouraged by enrolment with a preferred GP, access to longer consultations with that GP, and timely follow-up after hospitalization. At face value, these care process components would be expected to provide conditions that would promote patient-reported relational continuity, as measured in our study. In addition, given that policy efforts to improve relational continuity without attention to implementation have been previously criticized as having limited potential to change practice, we provided facilitation support to practices to assist in process change.²⁶ Nonetheless, in the short term, we were not able to demonstrate any improvements in patient-reported relational continuity, other patient-reported experience measures, or self-rated health. Despite the positive effects seen across observational studies, the literature concerning outcomes of interventions to enhance continuity of care in general practice is very limited,^9,27 and this study makes an important contribution to that literature internationally.

There are several potential reasons for the lack of improvement in patient-reported measures in our study. There may have been insufficient support, or duration given our 8- to 12-month intervention period, for practice-level processes to change. These had been cited as putative reasons for a lack of improvement in measured continuity of care, or care process measures, in the first 9 months following the national requirement for a ‘named GP’ for patients aged over 75 years in the UK.²⁶ However, potentially more important is the suggestion from our data that the patients in our study were already perceiving high levels of relational continuity and coordination of care. Baseline mean scores in intervention and control groups were around 3.4 from a maximum of 4.0 in each of these domains. Thus, there may have been limited scope for improvement. While gratifying, this is perhaps also not surprising. In order to avoid enroling patients who were transient, practices only sent invitations to ‘active’ patients of their practices. In addition, older patients and patients with chronic illness were targeted for recruitment; groups known to most value interpersonal continuity of care.⁶ In Australia, patients have freedom of choice of GP. Previous Australian surveys have suggested that 90% of patients, aged 60 years and over, considered that in managing a chronic or complex condition, it was important to have a regular GP who knew them and their medical problems well.²⁸ In turn, GPs value relational continuity.²⁹ Thus, the cohort and their practices had a high probability of having self-organized relational continuity at baseline. Supporting our ‘self-organisation’ hypothesis, we note our PCAT-S scores for relational continuity and coordination were very similar to those obtained using the longer form PCAT in Canadian primary care.^21,30 We also note that our intervention made no difference to access or comprehensiveness of care scores or self-reported health. In keeping with previous research, our results supported the observation that primary care may prioritize organizing continuity over accessibility for patients.³⁰

Observational studies consistently demonstrate an association between relational continuity with a primary care provider and improved outcomes, particularly for older patients and those with chronic conditions.^7,31 As enhancing relational continuity is a potentially low-cost, health care process intervention, it is understandable that it is targeted as a policy to improve patient outcomes and reduce costs.³ Our randomized trial data suggest that caution needs to be applied in extrapolating findings from observational studies to expected outcomes from incentivising patient enrolment. Our pragmatic trial recruited a readily accessible cohort of at-risk patients, in whom the intervention had no observable effect. In seeking to improve relational continuity, and in turn health outcomes, we recommend researchers and policymakers turn attention to less readily accessible but also high-risk groups, e.g., complex patients with no regular GP or those marginalized due to language, cultural, or socioeconomic barriers.²⁶ We recommend prospective trial designs, given the divergence of our findings from what may have been expected at face value from the intervention.

Limitations

The findings of the study should be interpreted in light of its limitations. While the study had excellent coverage across geographic and socioeconomic strata, there is potential for bias in practice recruitment as recruitment was based around existing university practice networks. Similarly, while patient recruitment was designed to reduce the risk of selection bias, the nature of patient enrolment biased the sample towards patients already engaged with their practices. This may be reflected in the higher proportion of females in the sample. Except for a chart audit to provide a baseline for hospitalization rates, it was not possible at the time of the study to collect baseline length of consultation or post-hospital follow-up data reasonably and accurately from the practice records. Due to time and resource constraints, we were only able to provide practices with their performance data and incentives at the end of trial, which may have reduced incentives’ impact on behaviour change. In addition, the timeframes of the trial resulted in limited overall observation times for participants. However, within these limitations, given the broad comparability of the practice sample to representative Australian data,¹⁹ the results can be reasonably generalized to Australian general practices and their patients over the age of 18 years with chronic conditions, or aged over 65 years.

Conclusion

The intervention was not associated with significant improvements in patient-reported experience measures or self-rated health. A plausible explanation is that GPs and their patients had already self-organized mutually valued relational continuity of care. We urge caution in providing incentives to promote an aspect of health care which Australian general practices appear to be already providing. Future planned analyses from this trial include mixed method evaluation of the intervention implementation and use of linked Medicare, Pharmaceutical Benefits, and hospitalization data for health service usage and health economic evaluations. Further research using robust prospective trial methods is required, targeting at-risk patients experiencing discontinuity of care.

Funding

The research team gratefully acknowledges funding received for this project from the Royal Australian College of General Practitioners and the Australian Government Department of Health.

Conflict of interest

Prof Nick Zwar, Prof Grant Russell, A/Prof Jan Radford, and Prof Danielle Mazza have received honoraria from the RACGP for expert committee roles. The other authors have no conflicts to declare.

Data Availability

The data underlying this article will be shared on reasonable request to the corresponding author.