Impact of a patient-specific co-designed COPD care scorecard on COPD care quality: a quasi-experimental study

Abstract

Background:

The evidence that sharing mass care quality data with health service users improves care is weak.

Aims:

We hypothesised that providing patients with individualised care quality data would drive improvements to the care received by those patients.

Methods:

Together with patients who had chronic obstructive pulmonary disease (COPD), we co-designed a quality score card mapping indicators derived from National Institute for Clinical Excellence (NICE) quality standards against matched data taken from their general practice clinical records. All 640 COPD patients from 10 practices had improvements in these indicators before and 3 months after the intervention compared with 595 COPD patients in 10 control practices.

Results:

Significant improvements in referral to pulmonary rehabilitation (P=0.03) and confirmation of diagnosis with spirometry (P=0.001) were seen in the intervention compared with the control practice population (P<0.001). Increases in the provision of self-management plans were seen in both the groups. No improvement was seen in other indicators.

Conclusions:

Although the study is not able to prove a direct cause and effect, there is sufficient evidence presented to warrant the larger-scale evaluation of co-designed, personalised, quality score cards for COPD patients used as a tool to enhance care quality.

Introduction

Primary care clinicians and their patients both want to achieve good clinical care. Audit data from a range of countries, however, suggest that primary care management of people with chronic obstructive pulmonary disease (COPD) is variable and often falls short of that recommended in evidence-based guidelines.^1–4 Such gaps are common across a range of long-term conditions, and a number of improvement initiatives have been employed to support primary care clinicians change practice. Ranging from direct physician payments^5,6 to education and training initiatives, benefits to patients are disappointingly limited.^7,8 An alternative strategy has been to make available data on care quality to the public in order to provide ‘patient choice’ of care provider.^9–11 Although the concept of consumer choice is rational, the evidence of effectiveness is negligible in secondary care^10,12 and lacking in primary care. It may be that large-scale data sets and their display format lack personal relevance to patients who, for the most part, just want good care from their own doctor.

We determined to test whether personalised care quality data presented in a format co-designed by primary care staff with patients would bring about improvements in care to a group of patients with COPD. A COPD care quality scorecard was populated with individualised data drawn from general practice clinical records. Patients were advised to use the scorecards to negotiate with primary care teams for improved care measured against the National Institute for Clinical Excellence (NICE) quality standards for COPD.¹³

Materials and methods

This study used a quasi-experimental design, i.e., a pragmatic non-randomised controlled study. All COPD patients from 10 intervention practices had care quality compared at the beginning and end of the study with all COPD patients from 10 control practices. The study took place in Redbridge, an outer north east London borough. The Redbridge Primary Care Trust (PCT) team used automated data uploads via software (Health Analytics) installed in clinical information systems across all 48 general practices in the borough to collect data on the quality of care of patients Read coded on all practice lists with a diagnosis of COPD. General practitioners (GPs) activated the upload by refreshing the Health Analytics icon installed on the PC desktop. This programme has the capability to identify Read-coded events from the individual patient record, which can then be mapped to the relevant quality standards. The quality standards were defined as those that are outlined in the NICE quality standards document.¹³ The research team selected from this list of 13 recommendations a smaller number to form the manageable content of a scorecard. The following criteria were used to select the standards to be included in the scorecard:

• Standards relevant to primary care general practice,

• Standards judged to have potentially high impact on patient outcomes,

• Standards for which relevant Read codes were available to map to the standard,

• Standards for which previous PCT wide analysis had demonstrated variability and underperformance in general practice within the PCT.

Six quality indicators were selected: post-bronchodilator spirometry recorded to confirm diagnosis; smoking cessation advice given to smokers; annual review carried out within 18 months; patients in receipt of a self-management plan; inhaler technique checked and recorded within 18 months; and patients with Medical Research Council dyspnoea score of 3 or greater referred for pulmonary rehabilitation.

Practices were recruited via email sent to all 48 practices requesting voluntary participation in the study. The first ten who expressed an interest were recruited to the intervention group. Ten further practices in a similar geographical area and with similar list sizes were actively recruited as a control group. This method of recruitment was chosen as pragmatic because of the time-limited research funds and the time lag in engaging >10 practices.

The second stage was to develop a visually attractive and meaningful scorecard that could be used to inform patients of the level of care recommended by NICE and how their care recorded on the primary care clinical systems compared with this. Central to the approach was co-creation of the scorecard by professionals and patients. This work was undertaken jointly with a representative from the Innovation Unit. A number of small workshops were set up involving Redbridge GPs and practice nurses, and 40 patients with COPD nominated by the intervention practices.

Several iterations of the scorecard were produced and then the draft version was tested with 10 additional patients in their own homes. These one-to-one discussions were used to pilot understanding, appearance and relevance of content. As a result of testing, an explanatory accompanying letter was produced that advised patients to take the scorecard with them to their next COPD review with either their GP or practice nurse.

The agreed scorecard consisted of a four-page A4-sized document. The front page contained the patient and GP details with a short explanation. The two-page middle section contained the six-item quality scorecard (Figure 1). For each item there was a brief explanation of what the item was, e.g., spirometry, why it was important to the care of patients with COPD, and the NICE standard itself. Alongside each standard was an indicator of the compliance with that standard using a traffic light system. The data used to assign the level of compliance was derived from the clinical record of that patient held by the GP surgery at which the patient was registered—green indicating full compliance, amber indicating partial compliance and red indicating the absence of confirmation that a standard had been reached. The amber coding would be used if there was evidence that an intervention was planned but had not yet completed; e.g., a patient was referred for advice to stop smoking but was still smoking at the time of the data upload, or a standard had been met in the past but had time-elapsed, e.g., annual review. The fourth, back, page listed the local medical services and contact numbers for the provision of urgent or emergency care for COPD patients in Redbridge.

Figure 1.

Pages 2 and 3 of the personalised scorecard using traffic light system to indicate compliance with key NICE quality standards, accompanying explanation of the standard and its importance to the patient.

An accompanying letter explained that the patients were invited to take the scorecards to their next interaction with their GP or practice nurse and to use it as a means of discussing the care they were receiving and how that care might be improved. A helpline number, direct to the research team, was given with the instruction that patients could call up with any queries or concerns that they might have on receiving the scorecard. The scorecards and letters were then placed in stamped addressed envelopes by practice staff and posted to the relevant patients.

This specific intervention took place alongside a broader COPD quality-improvement programme across three local boroughs, which included the distribution to general practices in the area of a standardised management plan that encouraged the prescription of rescue packs of recommended antibiotics and prednisolone to appropriate patients, with specific instructions on the indications for their use.

Data analyses

Data on the six quality indicators for all COPD patients in the intervention and control practices were extracted using the Health Analytics system and checked manually from the GP-held patient records, at baseline and again 3 months after the scorecards were sent to patients. A log was kept of patient calls to the scorecard helpline. Raw counts and percentages, not adjusting for the cluster effects of GP practices, are shown in Table 1.

Table 1. Change in compliance with NICE quality indicators for patients before and after intervention for comparing the intervention with the control practices.

Quality indicator	Control		Intervention
	Baseline (CB)	Post intervention (CP)	Baseline (IB)	Post intervention (IP)
Post-BD spirometry confirmed diagnosis	11.1 (84/755)	6.1 (49/804)	11.6 (69/595)	12.8 (84/655)
Annual review performed	69.0 (521/755)	62.3 (501/804)	62.9 (374/595)	60.2 (394/655)
Self-management plan received	14.6 (110/755)	18.2 (146/804)	6.1 (36/595)	11.6 (76/655)
MRC 3+ referred for rehabilitation	2.0 (15/755)	4.6 (37/804)	1.7 (10/595)	7.8 (51/655)
Patients quit smoking	21.9 (165/755)	20.0 (161/804)	23.7 (141/595)	23.5 (154/655)

Aggregate raw percentages and counts (numerator/denominator) ignoring the cluster effects of GP practices.

Abbreviations: BD, bronchodilator; MRC, Medical Research Council.

A statistical analysis before and after the intervention was made by comparing the intervention with the control practices. Each of the six quality indicators were recorded as binary responses, and generalised linear regression models were fitted with categorical predictors representing the period (baseline and post intervention) and the cohort (control and intervention groups), and also the interaction between them. The individual GP practices were used in the model to adjust for the effects of clustering. If an interaction was detected, it indicated that the difference between the baseline and post-intervention outcomes differed for the control and the intervention groups. This was tested using a likelihood ratio test, with P-values shown in Table 2. Ethical permission was withheld to collect patient-identifiable data that allowed baseline and post-intervention outcomes to be linked, which means that it is likely that the precision of the before-and-after comparison was reduced. The analyses were carried out using the computer program R.¹⁴

Table 2. Change in compliance with NICE quality indicators for patients before and after intervention for comparing the intervention with the control practices.

Quality indicator	Control			Intervention			Difference in differences: Intervention—control	P-value
	Baseline (CB)	Post intervention (CP)	Difference (CP)−(CB)	Baseline (IB)	Post intervention (IP)	Difference (IP)−(IB)
Post-BD spirometry confirmed diagnosis	6.7	3.3	−3.4	11.3	12.6	1.3	4.7	0.001
Annual review performed	65.2	57.9	−7.3	63.1	60.5	−2.6	4.7	0.22
Self-management plan received	4.5	6.2	1.7	4.8	9.7	4.9	3.2	0.12
MRC 3+ referred for rehabilitation	0.9	2.2	1.3	1.2	8.6	7.4	6.1	0.03
Patients quit smoking	21.8	20.1	−1.7	22.0	22.0	0.0	1.7	0.58

Generalised linear clustered model estimates, allowing for the cluster effects of GP practices, back transformed (from logits) to percentages.

Abbreviations: BD, bronchodilator; MRC, Medical Research Council.

Ethics permission for this study was granted by the Redbridge ethics committee (11/NE/0326).

Results

The 10 intervention practices had a mean registered population of 7,573 (range 4,364–13,049), with a total of 640 registered COPD patients (median 52 per practice). The 10 control practices had a mean of 5,490 registered patients (range 4,027–12,883) with 595 registered COPD patients (median 70).

Baseline and 3-month post-intervention data for the six quality indicators included in the patient scorecard are given in Table 1. One indicator (inhaler technique checked) was discarded from the analysis when it became apparent that the recording of this event was so poor, in terms of both quantity and quality, to make meaningful data collection impossible.

Two indicators in the intervention practices, those of diagnosis confirmed by spirometry and referral for pulmonary rehabilitation if the MRC score was 3 or more, demonstrated increased compliance compared with the control practices at the 3-month point (Table 2). Both the practice groups demonstrated a statistically significant improvement in the receipt of self-management plans (Table 1), but there was no difference in the magnitude of the improvement between the two groups (Table 2).

Approximately 30 patients rang the helpline. Enquiries included patients who were unaware that they had COPD, concerns about the accuracy of information provided, and confusion as to the purpose of the scorecard. Positive comments included interest in the indicators themselves where patients had not previously known what standard of care they should expect, and gratitude for having the details of the COPD support services made known to them when previously not known.

Discussion

Main findings

This study provides new insights into the use of health-care quality data in engaging patients in the improvement of their own care and quality-improvement initiatives, and on the subsequent effect this might have on the quality of care that patients receive. Working with professionals and patients, we have co-created a patient quality scorecard for people with COPD, which has been evaluated for impact against selected NICE quality standards. Patients in the 10 practices used as the intervention group had statistically significantly greater improvements in the care received in the two areas of ‘confirmation of diagnosis using post-bronchodilator spirometry’, and ‘referral for pulmonary rehabilitation’, when compared with patients within the control practices. Patients in both the intervention and the control groups demonstrated significant improvements in ‘the use of self-management plans’. There were no real changes in smoking prevalence rates among people with COPD, and no change in the proportion of patients who had undergone an annual review of their condition. This may have been a reflection of the relatively short follow-up period of the study (3 months) or of the relatively good compliance with this measure at baseline.

Strengths and limitations of this study

Although the findings suggest that there may be a causal relationship between the introduction of the patient scorecard and care improvements, there are significant limitations to this study. We found the data recording for inhaler technique to be non-standardised and poorly recorded, and it became clear that we were unable to extract high-quality information about this process. We were unable to obtain linked patient-identifiable data, a limitation imposed by the ethics committee, and we are therefore unable to match the practice population data at the beginning and end of the study so that we cannot be certain that we are comparing the same populations. It is also the case that practices in the intervention group volunteered to participate, which may have demonstrated a bias towards more motivated primary care teams or those with an interest in COPD compared with the control practices. This choice of study design was simply pragmatic owing to the limited funding to employ researchers and conditions around the time scale of the project. It may be therefore that practice-level involvement in the study itself was sufficient to bring about quality improvements in COPD care. We did not have the resource to record how many patients did use the scorecards to engage with their primary care clinicians. In the same manner, the time period for the follow-up was limited by funding and the attached conditions, which meant that some potential changes in care may not have been detected within the 3-month follow-up period. Finally (as with so much of UK National Health Service (NHS) evaluation), this study took place at a time of a significant change in the NHS, including a larger COPD quality-improvement project across Redbridge.

The strengths of the study are the original nature of this work, which addresses an area of vital importance but with minimal existing evidence. The control group design reduces the impact of external changes but cannot control totally for effect.

Interpretation of findings in relation to previously published work

Previous interventions to improve the quality of COPD patient care delivered by primary care have focussed on the ‘push’ elements, targeting health-care providers at either the individual or the organisational level. Financial incentives such as the UK Quality Outcome Framework¹⁵ and educational interventions are good examples. Although some improvements in care have been demonstrated, there are significant concerns about the value for money⁶ and the effectiveness of these approaches.^7,8,16,17

Alongside such initiatives have been an explosion in the release of health-care quality data into the public domain with the intention of driving change using patient and public ‘pull’. The evidence as exists suggests that patient use of such data is to make provider choices rather than to seek out specific improvements in an individual's own care.⁹ Evidence that public release of quality data improves care is lacking.¹⁸ We explored the use of a quality-improvement tool to empower patients to engage more actively in improving the quality of their own care.

Most patient-related condition-specific information is designed either for the general education of a patient (e.g., an information leaflet about COPD) or for a particular element of self-management (e.g., an action plan for exacerbations of COPD). We are unaware of any previous work that has attempted to engage patients in their care by co-designing a tool to inform them personally about the quality of their current care against national standards, thereby encouraging patients to use this information to discuss improved care with their primary care team.

Research evaluations of the use of health-care quality data have explored a number of provisions, but none of the type reported in this study. The closest that we have found is a hypothetical model for a personalised ‘dashboard’ designed for use by a secondary-care clinician but personalised to the needs of an individual patient.¹⁹ This is in stark contrast to the trend to personalise information outside of the public sector, e.g., social media websites and internet shopping providers, and suggests that there is an opportunity within the health sector to explore these established and successful approaches. We addressed the quality improvement and information agendas by providing bespoke information about each individual patient’s care to the patient him/herself in order to better equip patients to ensure that their needs are met. It is known that activating patient engagement in their health can produce better outcomes for those patients,¹¹ and this may be a relatively simple and a direct way of achieving activation. Currently, the levers to engage patients are relatively unexplored, and authorities in the field have called for more research in this area.²⁰ In this current study, the data are relevant to each individual patient, and they provide the potential for individual patients to make choices less about who provides their care and more about the standard of care that they should expect. This information allows them to become more empowered towards negotiating better care with their primary provider. The co-design of the tool with both patients and primary care staff maximises the potential for mutual acceptance of this concept.

Implications for future research, policy and practice

Our next steps include a detailed analysis of the patient user experience and that of the health-care teams involved in the project. It is likely that further modifications to the tool are required to improve its relevance to a wider population. Translation into the first languages of our increasingly diverse population is also necessary, although the use of pictorial representation and a reduction in text should facilitate its use. Interest has been expressed in using an adapted tool within the England and Wales National COPD audit.

Conclusions

In summary, we have shown that the use of a scorecard, co-created by patients and professionals, which informs patients of the standard of care that they should receive and gaps between this standard and the care that they actually receive, is associated with statistically significant improvements in some dimensions of care (although not others) for COPD. We believe that providing patients with information about the quality of the care that they do and should receive must be a positive factor in developing mutually beneficial partnership working between patients and their health-care providers. These data provide the justification for larger-scale studies to better understand the benefit that co-designed scorecards can drive.

The authors declare no conflicts of interest.