Abstract
Background
Primary care databases contain cardiovascular disease risk factor data, but practical tools are required to improve identification of at-risk patients.
Aim
To test the effects of a system of electronic reminders (the ‘e-Nudge’) on cardiovascular events and the adequacy of data for cardiovascular risk estimation.
Design of study
Randomised controlled trial.
Setting
Nineteen general practices in the West Midlands, UK.
Method
The e-Nudge identifies four groups of patients aged over 50 years on the basis of estimated cardiovascular risk and adequacy of risk factor data in general practice computers. Screen messages highlight individuals at raised risk and prompt users to complete risk profiles where necessary. The proportion of the study population in the four groups was measured, as well as the rate of cardiovascular events in each arm after 2 years.
Results
Over 38 000 patients' electronic records were randomised. The intervention led to an increase in the proportion of patients with sufficient data who were identifiably at risk, with a difference of 1.94% compared to the control group (95% confidence interval [CI] = 1.38 to 2.50, P<0.001). A corresponding reduction occurred in the proportion potentially at risk but requiring further data for a risk estimation (difference = –3.68%, 95% CI = –4.53 to –2.84, P<0.001). No significant difference was observed in the incidence of cardiovascular events (rate ratio = 0.96, 95% CI = 0.85 to 1.10, P = 0.59).
Conclusion
Automated electronic reminders using routinely collected primary care data can improve the adequacy of cardiovascular risk factor information during everyday practice and increase the visibility of the at-risk population.
Keywords: cardiovascular diseases, primary prevention, reminder systems, risk assessment, informatics
INTRODUCTION
Effective primary prevention of cardiovascular disease (CVD) requires health professionals to recognise individuals at risk. Relevant information (such as blood pressure and serum cholesterol measurements) is often recorded in electronic form in UK general practice, making possible automated calculation of risk and the use of on-screen reminders.
Current UK guidelines recommend the identification of those whose risk of a first cardiovascular event is ≥20% over 10 years.1,2 In some cases the risk profile of an individual may already be complete, but in others important information may be missing. Practical tools are required to identify those at risk and also those most likely to benefit from further data collection.
Since 2004, UK GPs have been remunerated through a ‘payment by performance’ system: the Quality and Outcomes Framework (QOF).3 Targets are set for the recording of health data such as smoking status and for the achievement of clinical outcomes such as blood pressure control. Disease registers are maintained for a number of long-term conditions including diabetes, coronary heart disease, and stroke/transient ischaemic attack. The data are almost exclusively recorded in electronic form as this is normally required for payment. This system covers the majority of chronic disease areas but CVD risk estimation was not included until 2009. Electronic health records are normally accessed during consultations and computer screen messages have become part of everyday practice, alerting clinicians to unmet QOF targets for data adequacy or control of clinical parameters. These messages typically appear on opening the notes with the patient present and may therefore influence decisions at the point of care.
It was hypothesised that an automated system of electronic reminders operating within the environment of routine care can usefully support CVD prevention in UK general practice. This article reports the results of a randomised controlled trial that aimed to test the effects of a software tool (the ‘e-Nudge’) on clinical outcomes and data quality related to CVD.
METHOD
The trial took place in the West Midlands of the UK between 2006 and 2008. The protocol has been previously described in detail, including the definitions used for programming the e-Nudge algorithm (see Box 1 for group definitions).4
Box 1. Groups identified by e-Nudge software.
Group A
Patients without known cardiovascular disease or diabetes who are 50–74 years old, and whose most recent risk factor values give an estimated risk level ≥20% over 10 years.
Group B
Patients without known cardiovascular disease or diabetes, who are 50–74 years old, and whose risk profile is incomplete — more information is required to perform a risk estimate — but whose risk would be >20% over 10 years if ‘assumed’ values of the missing factors are used.a
Group C
Patients with cardiovascular disease but not diabetes, who have not had a blood glucose measurement in the past 3 years.
Group D
Patients who are not known to have cardiovascular disease or diabetes, are over 75 years old, and who have persistently elevated blood pressure based on the three most recent consecutive readings.b
aAssumed values for missing variables were derived from the Health Survey for England 2003 as the median values for the 50–75 year age group for total serum cholesterol (males 5.7 mmol/l, females 6.2 mmol/l), high-density lipoprotein cholesterol (males 1.4 mmol/l, females 1.7 mmol/l), and systolic blood pressure (males 135 mmHg, females 132 mmHg). bSystolic blood pressure ≥160mmHg or diastolic blood pressure ≥100mmHg for each of the last three measurements.
The e-Nudge software tool
The e-Nudge searches electronic health records for cardiovascular risk factor data and builds lists of patients identified in each of four groups A–D (Box 1). Cardiovascular risk is estimated using the Framingham risk equation applied to the most recent risk factor measurements.5 The lists are updated every 24 hours to take account of new information. When the records of a patient are accessed, a screen alert message appears if the patient is currently in one of the groups. This happens during a consultation with the patient, or when notes are opened for any other clinical or administrative reason. The message explains why the patient has been identified, and, in the case of patients with insufficient information for a risk estimate, states which data are required. The user is not obliged to respond to the alert but the message continues to appear on opening the notes until the patient's record is updated in such a way that the patient is no longer in one of the groups (for example, by recording a blood pressure measurement if this was missing).
How this fits in
Cardiovascular risk factor data are recorded electronically in UK general practice, but identifying high-risk individuals requires the use of risk algorithms. A cardiovascular risk assessment tool applied to a practice database can identify people at raised risk and those most likely to benefit from further data collection. When linked to screen alert messages, this tool increased the visibility of the high-risk population and the adequacy of data to support risk estimation, but did not reduce cardiovascular event rates significantly over a 2 year period.
The intervention
The e-Nudge was installed in all participating practices. An email was sent to the practice manager (or nominated administrative staff member) every 8 weeks to remind them that lists of intervention patients in the various groups were available for viewing in their practice, but no action was mandatory for the purposes of the study. For those patients randomised in the intervention arm, the software produces screen reminders as described above. For those patients in the control arm, the e-Nudge recorded if they were in any of the groups but did not generate reminders. Access to the group lists was only possible for the intervention patients. Figure 1 gives an example of a Group A alert message appearing on the screen.
Figure 1.
Example of a group A message appearing during consultation.
Practice participants
Eligible practices were those running EMIS (Egton Medical Information Systems) LV software, as this was required to install the e-Nudge system. No other eligibility criteria were applied. EMIS supply clinical and administrative software to nearly 60% of UK practices, and 80% of their systems use the LV version. The majority of EMIS LV practices in Coventry and Warwickshire (41 out of 71 available practices) were invited to take part, and all that were willing were accepted into the trial. The 19 participant practices were based in diverse settings including rural, suburban, and inner-city environments. The practice list sizes varied from <2000 to >14 000 patients, and from single-handed practitioners to large group practices with more than six partners. Coronary heart disease indirectly standardised mortality ratios (for 2003–2005) ranged from 74 to 110 in the districts served by the practices.
Study population
The records of all individuals aged over 50 years and registered with participating practices were included in the trial. This age range was chosen to ensure a study group at relatively high cardiovascular risk.
Outcomes and sample size
The annual cardiovascular event rate was the primary outcome used for the power calculation. Assuming a cardiovascular event rate of 1260 events per 100 000 person-years (all ages) in the control arm,6 and a 10% lower event rate in the intervention arm (rate ratio of intervention to control of 0.9), a total sample of about 70 000 patients followed up for 2 years gives 80% power for a test at level 5% (two-sided) allowing for 15% withdrawal.7 This calculation was based on all-age event rates as it was not possible to find an event rate specifically for the over 50s at the time the trial was designed. The intervention was applied to the over 50s population and the outcome was measured in this age group only. A higher event rate was expected in this age group.
Randomisation and allocation concealment
Randomisation was at the level of the individual patient record. The e-Nudge software automatically randomised registered patients within each practice to intervention and control arms, depending on whether the last digit of the 10-digit NHS number was odd or even. This number is a unique identifier allocated to all individuals registered with the NHS and is generated using an algorithm that takes no account of age, socioeconomic group, or any other factor relevant to cardiovascular risk. The tenth digit is calculated according to the Modulus 11 algorithm,8 and serves as a ‘check digit’ to confirm the number's validity.
New patients registering with a practice during the study were randomised as soon as the NHS number was available in the record. Throughout the trial, users of the e-Nudge were kept unaware of the odd/even rule, but if an alert appeared on opening a record it would be evident that the patient was in the intervention arm. It was made clear to users at the outset that patients who did not trigger alerts were not necessarily at low cardiovascular risk, as they might simply be in the control arm.
Outcome assessment
The number and proportion of patients identified in each of the groups were automatically extracted by the e-Nudge and stored in a file in the practice's main computer server every 8 weeks. The mean values from the final three data captures during the 2-year period were derived for each group. The practice data were aggregated. The annual incidence of cardiovascular events was obtained from each practice's database. Following a search for relevant events (Box 2), each individual record identified was examined using a predetermined protocol to confirm that the event had occurred within the trial period, and, where more than one event was recorded, that these were separate events.
Box 2. Definition of a cardiovascular event used in the trial.
A new diagnosis of cardiovascular disease (that is, entry onto the Coronary Heart Disease [CHD] Register or Stroke/Transient Ischaemic Attack Register)
A new stroke or transient ischaemic attack (whether or not already on the Stroke/Transient Ischaemic Attack Register)
A new myocardial infarction (whether or not already on the CHD Register)
Sudden death from cardiovascular disease
Quality assurance
Recording of cardiovascular outcomes is prone to several sources of error, recognised in the trial protocol.4 Not all cardiovascular events result in a new coded entry into a primary care record, and sometimes a single event is recorded more than once using different codes or entry dates. When a patient dies, the need to record the final event electronically is no longer a priority for clinical care, although it is usual practice to do so. For these reasons, the study examined every electronic record identified in the outcome searches. A small substudy was also carried out in four practices to check whether any sudden cardiovascular deaths had been missed. For this substudy, extra code groups were included in the searches to increase the retrieval of cases: ‘Death administration’, ‘On examination — dead’, and all of their lower-level codes.
Changes to the trial protocol
In the original protocol, patients with existing CVD or diabetes whose blood pressure or serum cholesterol were outside the QOF target were to be identified. However, screen alert messages were introduced to all EMIS systems to support the QOF just before the start of the trial. This group was therefore withdrawn from the trial.
The e-Nudge was initially designed also to identify individuals with possible undiagnosed diabetes based on previous raised blood glucose measurements. A number of such individuals were identified among the baseline data following installation of the e-Nudge software. This led to a nationwide survey to demonstrate that such patients are identifiable across the UK,9 and the result was the introduction of a new software module to all EMIS systems nationally to support early diabetes detection.10 This group was therefore withdrawn from the e-Nudge within the first 6 months of the trial.
During practice visits, one researcher discussed practical issues surrounding the usability of the software which were noted and acted on. After approximately 9 months, the wording of the screen alerts was shortened in response to practice feedback but the information they contained remained the same.
Statistical analysis and intention to treat
Analysis was carried out using STATA 10 software. For the cardiovascular event rates, the rate ratio with 95% confidence interval (CI) and two-tailed P-value were computed using standard likelihood inference techniques for Poisson counts.11 The group proportions were compared using χ2-tests, and two-tailed P-values are reported. Data were analysed from all patients whether or not their computer record had been accessed by primary care staff during the trial.
One of the practices withdrew from the study after less than 6 months, but consented to their data being included in the analysis. However, the automatically captured group data were no longer available from this practice after the software was removed, so only the cardiovascular event rate data were used as part of the final analysis. In one practice, a failure of data capture occurred at baseline and the earliest data available at this site were extracted after the intervention had been in place for 25 days.
RESULTS
Nineteen practices were recruited, with a combined list size (all ages) of approximately 121 000, of which 38 147 were in the over 50-year age group at baseline. A total of 77 208 person-years of follow-up over 2 years (38 382 for intervention participants and 38 826 for control participants) were estimated (Figure 2). Recruitment began in June 2006 and was completed in September 2006. The intervention ran for 24 months in each practice. The trial ended in September 2008. Recruitment into the study had to be closed before the required number of patients over 50 years could be achieved, due to resource constraints.
Figure 2.
The CONSORT flowchart, August 2005.
Proportions of patients in groups A, B, C, and D
The overall proportion of the over-50-year trial population identified in each arm at the end of the trial is given for the four e-Nudge target groups (A–D) in Table 1.
Table 1.
Group proportions in the baseline and outcome populations by trial arm.
| % (n) | ||||
|---|---|---|---|---|
| Group A | Group B | Group C | Group D | |
| Intervention | ||||
| Baseline (n = 18 912) | 5.94 (1124) | 26.9 (5079) | 2.78 (525) | 0.43 (81) |
| Outcome (n = 18 021) | 8.91 (1606) | 19.4 (3502) | 0.55 (99) | 0.33 (59) |
| Change, % | 2.97 | –7.48 | –2.23 | –0.10 |
| Control | ||||
| Baseline (n = 19 235) | 5.91 (1137) | 25.9 (4990) | 2.70 (519) | 0.46 (89) |
| Outcome (n = 18 071) | 6.97 (1260) | 23.1 (4177) | 1.01 (183) | 0.31 (56) |
| Change, % | 1.06 | –2.83 | –1.69 | –0.15 |
| Intervention–control difference at outcome (95% CI) | 1.94 (1.38 to 2.50), P<0.001 | –3.68 (–4.53 to –2.84), P<0.001 | –0.46 (–0.64 to –0.28), P<0.001 | 0.02 (–0.10 to 0.13), P = 0.768 |
A significant increase in the number identified in group A (those whose raised risk was identifiable due to adequate risk factor data) occurred with a corresponding reduction in group B (those with incomplete profiles but potentially at risk). A reduction was also seen in group C (those with CVD but no recent blood glucose measurement). While there were background improvements in the control population during the study, the improvements in the intervention arm were significantly greater for these three groups. However, no significant change was seen in intervention patients aged over 75 years with persistently raised blood pressure (group D).
Practice-level changes in group proportions
Considerable variation was seen between practices in the impact of the e-Nudge on group proportions. For example, the median absolute reduction from baseline in the group B intervention population was 5.1%, with a range of 2.5–20.4%.
Cardiovascular event rates
The cardiovascular event rates in the intervention and control arms are given in Table 2. The rate ratio is 0.96 (95% CI = 0.85 to 1.10). This difference is not statistically significant (two-tailed P = 0.59). Altogether, 2121 individual records were examined in the search for new events during the 2 years of the trial. A total of 930 events occurred in the trial population. In year 1, 492 events occurred in 454 individuals (21 experienced two events, seven experienced three, and one experienced four events). In year 2, 438 events occurred in 412 individuals (19 experienced two, two experienced three, and one experienced four events). The annual searches were run separately and it is not known how many individuals affected in year 1 were also affected in year 2.
Table 2.
Cardiovascular event rates in the intervention and control arms of the trial.
| Events | Patient years of follow-up | Rate/100 000 population/year | |||
|---|---|---|---|---|---|
| Arm | Year 1 | Year 2 | Total | ||
| Intervention | 235 | 219 | 454 | 38 382 | 1183 |
| Control | 257 | 219 | 476 | 38 826 | 1226 |
| Overall | 492 | 438 | 930 | 77 208 | 1205 |
Quality assurance substudy
As described in the Method section, a small substudy was undertaken to assess the rate of fatal unrecorded events. In addition to the 73 events already confirmed in these four practices through the standard searches, a further 34 records were then identified (that included deaths from all causes). When these extra notes were examined, two definite ‘missed’ cardiovascular deaths were identified. A further two possible missed events were found, but it was not clear from the information available whether these satisfied the definition of a cardiovascular event used in this study. Depending on whether two or four missed events are assumed, this suggests that either 2.6% or 5.2% of events are missed by the standard searches. This error should apply equally to both arms of the trial.
DISCUSSION
Summary of main findings
The e-Nudge changed practitioner behaviour to improve the adequacy of risk factor information and increase the visibility of the at-risk population, but did not reduce CVD events significantly. This confirms the prediction made in a preliminary discussion paper that the most immediate benefit of this approach would be a progressive improvement in data quality.12
The greatest impact was seen on the group potentially at risk but in need of more information to complete their risk profiles. The actions taken to collect and record these data reduced this population and increased the numbers whose raised risk was identifiable. Identification of at-risk patients is the first step in primary CVD prevention and this study has demonstrated that automated reminders can facilitate this process in the routine environment of general practice.
An impact was also seen on the group with CVD but no recent blood glucose measurement, but not on the older patients with persistently raised blood pressure. This finding reinforces the conclusion that processes of care are easier to influence than harder clinical outcomes using such an intervention.
Strengths and limitations of the study
This study involved a diverse sample of UK practices displaying a range of list sizes and cardiovascular mortality in the background population. An impact on CVD event rates became unlikely when the original group with CVD or diabetes was withdrawn from the trial, as this group is at higher risk of events than the primary prevention group. Recruitment for the trial was less than required for 80% power and the reduction in event rates in the intervention arm was not significant, raising the possibility of a type-2 error. The authors intend to measure this event rate for the year following the withdrawal of the e-Nudge to see whether the difference has increased or diminished. A trial of this type using individual (rather than practice cluster) randomisation raises the risk of contamination of the intervention into the control population, but as there were no reminders appearing for control patients, this was not considered an important problem. The study design eliminated the risk of unequal practice characteristics between the two arms that may result from cluster randomisation, and inflation of the required sample size due to clustering was avoided.
Comparison with existing literature
Electronic reminders have been shown to improve practitioner behaviour in a number of areas including prescribing,13 but adoption may vary between both individual clinicians and types of reminder.14 In some cases the use of performance measures may be important.15 Within the UK, screen reminders are in common use to assist in achieving QOF targets, but they are potentially of value in many other healthcare contexts.16
Implications for future research and clinical practice
The e-Nudge is one of a small number of software tools so far developed that is capable of supporting the UK's cardiovascular risk assessment programme.17 It is the only one to have been subjected to a controlled trial. This study has demonstrated that automated electronic reminders can raise the visibility of the at-risk population for cardiovascular disease. Improving the adequacy of risk profile information is an essential first step in the effective targeting of preventive interventions to higher-risk individuals. Further work is needed to identify the most effective format of alert messages operating within this environment, to maximise the utility of such tools as part of routine care.
Acknowledgments
We would like to thank all of the practice teams taking part in the trial, Rachel Potter for assisting with data collection, Shaun O'Hanlon for advice on e-Nudge development, and Enid Hennessy, Sandra Eldridge, Nigel Stallard, and Ranjit Lall for statistical advice.
Funding body
The trial was funded by a Department of Health PhD Studentship from Warwick Medical School.
Ethical approval
Ethical approval was obtained from Warwickshire Local Research Ethics Committee, reference 05/Q2803/85.
Competing interests
David Stables is Medical Director of Egton Medical Information Systems (EMIS), the company that programmed the trial software and provided the necessary clinical software platform for the trial. There are no other competing interests.
Trial registration
The trial was registered for an International Standard Randomised Controlled Trial Number: 64828380.
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
REFERENCES
- 1.British Cardiac Society, British Hypertension Society, Diabetes UK et al. JBS 2: Joint British Societies' guidelines on prevention of cardiovascular disease in clinical practice. Heart. 2005;91(suppl 5):v1–v52. doi: 10.1136/hrt.2005.079988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.National Institute for Health and Clinical Excellence. Lipid modification: cardiovascular risk assessment and the modification of blood lipids for the primary and secondary prevention of cardiovascular disease. Clinical Guideline 67. London: NICE; 2008. [PubMed] [Google Scholar]
- 3.Department of Health. Quality and Outcomes Framework. http://www.dh.gov.uk/en/Healthcare/Primarycare/Primarycarecontracting/QOF/index.htm (accessed 3 Nov 2009)
- 4.Holt TA, Thorogood M, Griffiths F, Munday S. Protocol for the ‘e-Nudge trial’: a randomised controlled trial of electronic feedback to reduce the cardiovascular risk of individuals in general practice [ISRCTN64828380] Trials. 2006;7:11. doi: 10.1186/1745-6215-7-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Anderson KM, Wilson PWF, Odell PM, et al. An updated coronary risk profile. A statement for health professionals. Circulation. 1991;83(1):356–362. doi: 10.1161/01.cir.83.1.356. [DOI] [PubMed] [Google Scholar]
- 6.British Heart Foundation. Coronary heart disease statistics: 2004 Edition. London: British Heart Foundation; 2004. [Google Scholar]
- 7.Ng HKT, Tang M-L. Testing the equality of two Poisson means using the rate ratio. Stat Med. 2005;24(6):955–965. doi: 10.1002/sim.1949. [DOI] [PubMed] [Google Scholar]
- 8.NHS Connecting for Health. NHS number. http://www.datadictionary.nhs.uk/data_dictionary/data_field_notes/n/nhs_number_de.asp (accessed 27 Jan 2010)
- 9.Holt TA, Stables D, Hippisley-Cox J, et al. Identifying undiagnosed diabetes: cross-sectional survey of 3.6 million patients' electronic records. Br J Gen Pract. 2008;58(548):192–196. doi: 10.3399/bjgp08X277302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Holt TA. Detection of undiagnosed diabetes using UK general practice data. Br J Diab Vasc Dis. 2008;8(6):291–294. [Google Scholar]
- 11.Clayton D, Hills M. Statistical models in epidemiology. Oxford: Oxford University Press; 1993. [Google Scholar]
- 12.Holt TA, Ohno-Machado L. A nationwide adaptive prediction tool for coronary heart disease prevention. Br J Gen Pract. 2003;53(496):866–870. [PMC free article] [PubMed] [Google Scholar]
- 13.Bennett JW, Glasziou PP, Sim I. Review: computerised reminders and feedback can improve provider medication management. Evid Based Med. 2003;8(6):190. [Google Scholar]
- 14.Agrawal A. Mayo-Smith MF: Adherence to computerized clinical reminders in a large healthcare delivery network. Medinfo. 2004;11(1):111–114. [PubMed] [Google Scholar]
- 15.Fung CH, Woods JN, Asch SM, et al. Variation in implementation and use of computerized clinical reminders in an integrated healthcare system. Am J Manag Care. 2004;10(11 Pt 2):878–885. [PubMed] [Google Scholar]
- 16.Shojania Kaveh G, Jennings A, et al. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009:119–129. doi: 10.1002/14651858.CD001096.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.University of Leicester and the National Screening Committee. The handbook of vascular risk assessment, risk reduction and risk management. Leicester: University of Leicester and the National Screening Committee; 2008. Comparison of software available for a population based CVD risk assessment strategy; pp. 071–074. [Google Scholar]