Abstract
Background
Relatively little is known about prescribing errors in general practice, or the factors associated with error.
Aim
To determine the prevalence and nature of prescribing and monitoring errors in general practices in England.
Design and setting
Retrospective case-note review of unique medication items prescribed over a 12-month period to a 2% random sample of patients. Fifteen general practices across three primary care trusts in England.
Method
A total of 6048 unique prescription items prescribed over the previous 12 months for 1777 patients were examined. The data were analysed by mixed effects logistic regression. The main outcome measures were prevalence of prescribing and monitoring errors, and severity of errors, using validated definitions.
Results
Prescribing and/or monitoring errors were detected in 4.9% (296/6048) of all prescription items (95% confidence interval [CI] = 4.4% to 5.5%). The vast majority of errors were of mild to moderate severity, with 0.2% (11/6048) of items having a severe error. After adjusting for covariates, patient-related factors associated with an increased risk of prescribing and/or monitoring errors were: age <15 years (odds ratio [OR] = 1.87, 95% CI = 1.19 to 2.94, P = 0.006) or >64 years (OR = 1.68, 95% CI = 1.04 to 2.73, P = 0.035), and higher numbers of unique medication items prescribed (OR = 1.16, 95% CI = 1.12 to 1.19, P<0.001).
Conclusion
Prescribing and monitoring errors are common in English general practice, although severe errors are unusual. Many factors increase the risk of error. Having identified the most common and important errors, and the factors associated with these, strategies to prevent future errors should be developed, based on the study findings.
Keywords: general practice, medication errors, prevalence, primary health care
INTRODUCTION
The prescription of medication is the most common form of medical treatment in primary care and over 900 million items are dispensed in the community in England each year.1 Prescribing is therefore an essential skill for GPs. For every prescribing decision, the potential for benefit needs to be balanced against the risk of harm. The prescriber must use clinical knowledge to apply bodies of evidence, rules, and guidance to a prescribing decision, while also taking into account the patient’s view. The challenge of prescribing has increased, owing to the increased complexity of medical care, and the treatment of older and more severely ill patients.2 Errors can occur in this process. In primary care, published error rates per prescription item vary from less than 1%,3 to over 40%,4 the latter being a Swedish study where failure to document the indication for a drug was considered an error. Such variation in error rates is likely to be significantly affected by the definition of error and the rigour with which error detection is undertaken. Nevertheless, prescribing errors in primary care are a preventable source of harm, with a systematic review showing that they account for around 3.7% of hospital admissions.5
Relatively little is known about prescribing errors in general practice, or the factors associated with error. In one study, prescriptions were screened by community pharmacists for prescribing errors; prescribing errors were identified in 7.5% of prescribed items.6 A study in care homes showed that 39% of 256 residents had one or more prescribing errors, with 8.3% of prescription items (or intended prescription items) affected.7 According to the UK’s National Patient Safety Agency, 26% of general practice incidents reported to the National Reporting and Learning System relate to medication,8 and 25% of adverse incidents that resulted in litigation claims in general practice were a result of medication errors, suggesting consequences not only for patient safety but also for practitioners.9
Given the paucity of research in this field, a large-scale study was conducted, with the aim of investigating the prevalence and nature of prescribing and monitoring errors in general practice in England, along with the factors associated with errors.
How this fits in
Prescribing errors are known to be a problem in general practice, but estimates of prevalence vary widely and little is known about the frequency of different types of error or the factors associated with error. In this study, prescribing or monitoring errors were detected in one in 20 prescription items and the vast majority were of mild to moderate severity. A number of factors were associated with prescribing and/or monitoring errors; these included age, sex, number of unique prescriptions per patient, and prescriptions from specific therapeutic groups. Having identified the most common and important errors, and the factors associated with these, it will be possible to devise effective strategies for preventing errors in the future.
METHOD
Setting and selection of general practices
Three English primary care trusts (PCTs) with differing characteristics (inner-city London, urban, and suburban/rural) were approached to act as sites for recruitment of general practices.
Between May and August 2010, all 97 general practices from these PCTs were invited to take part. Thirty replied and 20 expressed an interest in taking part. A total of fifteen were then purposefully selected to obtain a wide spread of different types of practice.
Sample size calculation
On the basis of previous studies, the best estimate for the prescribing error rate in primary care was around 8%.6,7 It was calculated that if a similar prevalence of prescribing errors were to be found in the present study, 3750 prescription items needed to be reviewed, in order to report this with a two-sided 95% confidence interval (CI) and accuracy of ±1.6%. This allowed for the clustered design and assumed a total of 15 practices, an intra-class correlation coefficient of 0.01 (based on the authors’ previous studies),10 and an average cluster size of 250 prescription items reviewed per practice.
The records of a 2% random sample of patients within each general practice were investigated to comfortably exceed the target of 3750 prescription items and thus increase the precision of the estimate of the error rate.
Definitions
The definition of a prescribing error used was one originally developed with hospital-based practitioners:
“A ... prescribing error occurs when, as a result of a prescribing decision or prescription writing process, there is an unintentional, significant ... reduction in the probability of treatment being timely and effective or ... increase in the risk of harm when compared to generally accepted practice.”11
This definition is accompanied by a list of examples of what should and should not be included as an error.
A monitoring error was defined, based on the consensus definition of Alldred et al, together with a list of medications that need blood-test monitoring and its frequency:
“A monitoring error occurs when a prescribed medicine is not monitored in the way which would be considered acceptable in routine general practice. It includes the absence of tests being carried out at the [required] frequency ..., with tolerance of ±50%. ... If a patient refused to give consent for a test, then this would not constitute an error.”12
For the purposes of this study, a list of medications requiring blood-test monitoring was used, along with recommended monitoring intervals (Box 1).
Box 1. Medicines requiring blood test monitoring
The following medications were reviewed to determine whether blood-test monitoring had been done before or soon after the initiation of therapy (for medication that had recently been started), or as part of ongoing maintenance for patients on longer term therapy.
| Monitoring before or soon after the initiation of therapy | |
| Drug/drug group | Monitoring requirements |
| Angiotensin-converting enzyme inhibitor/ | Renal function and electrolytes before starting therapy, repeated 2 weeks after initiation |
| Angiotension II receptor antagonists | Renal function and electrolytes before starting therapy, repeated 2 weeks after initiation |
| Digoxin | Renal function and electrolytes before starting therapy |
| Diuretics | Renal function and electrolytes before starting therapy |
| Glitazones | Liver function tests before starting therapy |
| Statins | Liver function tests before starting therapy |
| Monitoring of longer-term therapy | |
| Drug/drug group | Monitoring requirements |
| Angiotensin-converting enzyme inhibitor/ | 12-monthly renal function and electrolytes |
| Angiotension II receptor antagonists | 12-monthly renal function and electrolytes |
| Amiodarone | 6-monthly thyroid function tests |
| 6-monthly liver function tests | |
| Azathioprine | 3-monthly full blood count |
| Carbimazole | 3-monthly thyroid function tests (6-monthly if patient been stabilised for over 1 year) |
| Digoxin | Digoxin level if toxicity or lack of efficacy suspected. |
| Diuretics | 12-monthly renal function and electrolytes |
| Glitazone | 12-monthly liver function tests |
| Levothyroxine | 12-monthly thyroid function tests |
| Lithium | 3-monthly lithium concentration |
| 12-monthly thyroid function tests | |
| Methotrexate | 3-monthly full blood count |
| 3-monthly liver function tests | |
| 6-monthly renal function and electrolytes | |
| Sulfasalazine | Full blood count 3-monthly in first year |
| Liver function tests 3-monthly in first year | |
| Full blood count 6-monthly in second year | |
| Liver function tests 6-monthly in second year | |
| No further monitoring if stable | |
| Statin | Liver function tests within 3 months, and at 12 months, of starting treatment |
| Theophylline | Theophylline level if toxicity suspected |
| Valproate | 3-monthly liver function tests for first 6 months |
| Warfarin | 12-weekly International Normalized Ratio |
Identification and classification of prescribing and monitoring errors
Data were collected by four pharmacists trained by the research team to identify potential errors from GP records. Data were collected between August 2010 and April 2011. The pharmacists undertook a retrospective review of unique prescription items issued to patients in the 12 months prior to the date of data collection (for any prescription that was repeated during the study period, only the latest of these was examined). The pharmacists identified any potential prescribing or monitoring errors, having taken account of detailed information in patients’ medical records relating to patient characteristics, comorbidities, other medications, allergies, and the need for monitoring.
The details of all potential errors were discussed by a panel (a GP, a clinical pharmacologist, and three pharmacists), to decide whether they fitted the error definition, taking into account practice in primary care, and, if so, how the error should be classified in terms of error type. In the vast majority of cases, the pharmacists who had collected the data provided sufficient information for the panel to make a judgement. Where this was not the case, they were asked to provide further details, and to return to the general practice if needed. The panel maintained a cumulative list of classification decisions to ensure consistency throughout the study.
It became apparent that there were some problems that did not fit within the error definitions, but nevertheless represented less than ideal practice. Therefore, another category of problem, ‘suboptimal prescribing’, was created. The pharmacists also identified prescriptions associated with legal issues, but that did not fall into the definition of error; these were given a separate category of ‘legal problem’. Each potential error was therefore classified as a prescribing error, a monitoring error, suboptimal prescribing, legal problem, or no problem.
Error severity
The severity of each error, identified using a reliable validated 0–10 scale (0 = no risk of harm; 10 = death),13 was assessed by two GPs, two pharmacists, and one clinical pharmacologist. According to this method, the mean score across all five judges was used as the severity score, where errors with scores of <3 were considered minor, those between 3 and 7 moderate, and scores >7 severe.
Data relating to the characteristics of participating general practices
The general practices provided descriptive data, including list size (number of patients); age–sex breakdown of the practice population; number, sex, and types of GPs (and other prescribers) in the practice; whether the practice was a GP training practice, or a dispensing practice; and the clinical computer system used in the practice. Publicly available sources of data (Table 1) were used to obtain deprivation scores and Quality and Outcomes Framework points for the practices.
Table 1.
Comparison of characteristics of general practices involved in the study with National figures for England
| Characteristic | Mean (SD) across GP practices studied | Mean national figurea |
|---|---|---|
| Practice list size | 5916 (3014) | 6610a |
| Number of GPs | 5 (2.3) | 4.73b |
| Deprivation using IMD 2007 score | 30.4 (18.2) | 21.7c |
| QOF medicines management points per practice, % | 99.2% (2.0) | 97.2%d |
| QOF total points per practice, % | 92.5% (6.8) | 93.7%d |
IMD = Index of Multiple Deprivation. QOF = Quality and Outcomes Framework. SD = standard deviation.
Figures for England for 2010, obtained from the Health and Social Care Information Centre [http://www.hscic.gov.uk/searchcatalogue?productid=10382&topics=2%fPrimary+care+services%2fGeneral+practice%2fGeneral+Practice+workforce&sort=Most+recent&size=10&page=1#top] data available in Microsoft Excel Workbook with file name ‘NHS Staff — 2002–2012, General Practice Bulletin Tables.
Figures for England for 2010 calculated from data obtained from the Health and Social Care Information Centre: total GPs in England: 39 409; total general practices: 8324 ([http://www.hscic.gov.uk/searchcatalogue?productid=10382&topics=2%fPrimary+care+services%2fGeneral+practice%2fGeneral+Practice+workforce&sort=Most+recent&size=10&page=1#top] data available in Microsoft Excel Workbook with file name ‘NHS Staff — 2002–2012, General Practice Bulletin Tables.
Mean IMD score for English lower layer super output areas (available from http://webarchive.nationalarchives.gov.uk/20100410180038/http://communities.gov.uk/communities/neighbourhoodrenewal/deprivation/deprivation07/).
2010 Quality and Outcomes Framework figures for England (available from: http://www.ic.nhs.uk/qof).
Data analysis
Data were entered onto a Microsoft® Access® 2007 database, and every item was checked by another member of the research team. Any data-entry errors were corrected and an audit trail was kept.
Categorical variables were described using frequencies and percentages, and means and standard deviations (SDs) or medians and interquartile ranges (IQRs) were used to summarise continuous variables, depending on their distribution.
The prevalence of prescribing and monitoring errors was estimated and 95% CIs reported. Multivariable analyses, modelling the relationships between the risk of error and selected predictor variables and a priori confounders, were performed at the patient level and at the prescription level respectively. The primary outcome measure was binary, indicating the presence of one or more prescribing or monitoring errors, and modelled using logistic regression. To estimate the relationship between prescribing and monitoring errors and relevant covariates, six logistic regression models were built as follows: (1) a univariate model using patient and general practice characteristics; (2) a multivariable model with all patient and general practice characteristics; (3) a univariate model using prescription and prescriber characteristics; (4) a univariate model using prescription and prescriber characteristics, adjusting for patient clustering; (5) a multivariable model with all prescription and prescriber characteristics; and (6) a multivariable model with all prescription and prescriber characteristics, adjusting for patient clustering. To take into account potential clustering, in models 4 and 6, patients were treated as random effects in a multilevel model to adjust for correlation. The significance of the variables in the models was assessed using the Wald χ2 test and by determination of odds ratios (ORs) with associated 95% CIs. Statistical significance for all analyses was set at <0.05 (two-sided). All the statistical analyses were performed using STATA (version 11.2), and the xtmelogit command used to take into account possible clustering (patient).
RESULTS
General practices characteristics
The mean list size of the 15 general practices was 5916 (SD = 3014, range = 1600 to 12 000); 10 (67%) were involved in GP training, two (13%) were dispensing, and 12 (80%) were in an urban area. The practices used a range of different clinical computer systems.
Compared with figures for England, the study practices were reasonably representative in terms of list size and overall scores in the Quality and Outcomes Framework (Table 1), but appeared more deprived.
Medical records reviewed and patient characteristics
The study involved examination of the records of 1777 patients. All practices had computerised records and this was the main source of information; paper-based records (such as hospital correspondence) were used if these provided additional information. Data were available for all patients. These patients had a mean age of 39.5 years (SD = 22.6 years), and 49.7% (884/1777) were female. The age distribution of patients was similar to that of the English population for 2010,14 with 16.7% (297/1777) aged 0–14 years and 15.9% (283/1777) aged ≥65 years. Of the 1777 patients, 67.5% (1200/1777) had at least one prescription during the 12-month retrospective review of their records.
The pharmacists reviewed 6048 unique prescription items. Of these, 48.4% (2929/6048) were acute prescriptions, 51.6% (3119/6048) were repeat prescriptions, and 12.7% (770/6048) were items considered to require blood-test monitoring. A clear majority of the 6048 prescriptions (80.3% [4859/6048]) were issued by GP partners, 12.9% (779/6048) by salaried GPs, 3.1% (185/6048) by locum GPs, 2.2% (133/6048) by GPs in training, and 1.0% (60/6048) by non-medical prescribers; for 0.5% (33/6048) the type of prescriber was not identified.
Prevalence of errors and other problems
From the 6048 prescription items reviewed, there was a prevalence of:
4.1% (247/6048; 95% CI = 3.6% to 4.6%) prescribing errors;
0.9% (55/6048; 95% CI = 0.7% to 1.1%) monitoring errors;
7.1% (427/6048; 95% CI = 6.4% to 7.7%) suboptimal prescribing;
0.1% (8/6048; 95% CI = 0.06% to 0.3%) legal problems.
Restricting analysis to the 770 prescription items that required blood-test monitoring showed a prevalence of monitoring errors of 6.9% (53/770; 95% CI = 5.2% to 8.9%).
The prevalence of prescription items with at least one prescribing and/or monitoring error was 4.9% (296/6048; 95% CI = 4.4% to 5.5%), with a small number of prescriptions containing more than one error. There were 247 prescribing errors and the most common types were incomplete information on the prescription; dose/strength error; and incorrect timing of doses (Table 2). There were 55 monitoring errors and the most common type was failure to request monitoring.
Table 2.
Distribution of different types of prescribing errors
| Types of prescribing error | Frequency | All errors, % |
|---|---|---|
| Incomplete information on prescription | 74 | 30.0 |
|
| ||
| Dose/strength error | 44 | 17.8 |
|
| ||
| Timing error | 26 | 10.5 |
|
| ||
| Frequency error | 20 | 8.1 |
|
| ||
| Omission of concomitant treatment | 19 | 7.7 |
|
| ||
| Unnecessary drug | 12 | 4.9 |
|
| ||
| Contraindication error | 12 | 4.9 |
|
| ||
| Incorrect drug | 10 | 4.0 |
|
| ||
| Duplication | 9 | 3.6 |
|
| ||
| Interaction error | 9 | 3.6 |
|
| ||
| Allergy error | 3 | 1.2 |
|
| ||
| Inadequate documentation in medical records | 3 | 1.2 |
|
| ||
| Quantity error | 3 | 1.2 |
|
| ||
| Formulation error | 2 | 0.8 |
|
| ||
| Generic/brand name error | 1 | 0.4 |
|
| ||
| Total | 247 | 100.0 |
|
| ||
| Type of monitoring error | ||
| Monitoring not requested | 38 | 69.1 |
| Requested but not done | 12 | 21.8 |
| Results not available | 5 | 9.1 |
|
| ||
| Total | 55 | 100.0 |
For the 12-month data-collection period, of the patients who received at least one medication, 18.7% (224/1200; 95% CI = 16.5% to 21.0%) had at least one prescribing and/or monitoring error. For patients aged ≥75 years who had received at least one medication, the prevalence was 41.9% (54/129; 95% CI = 33.2% to 50.9%). For patients who had received five or more medications over the data-collection period, the prevalence of error was 32.3% (152/471; 95% CI = 28.1% to 36.7%), and for patients who had received 10 or more drugs, the prevalence was 48.8% (84/172; 95% CI = 41.2% to 56.6%).
Error severity
Of the 302 prescribing and monitoring errors, 42.4% (128/302) errors had scores of <3, and were thus deemed to be minor; 54.0% (163/302) had scores of 3–7 and were thus moderate; 3.6% (11/302) had scores >7 and were thus severe. The median severity score was 3.3 (IQR = 2.2 to 4.4, minimum 0.7, maximum 8.6). The 55 monitoring errors had a median score of 3.8; the 247 prescribing errors had a lower median score of 3.0. Examples of errors of different severity are shown in Box 2. On the basis of the data collected in this study, no patient was hospitalised or died as a result of the errors detected.
Factors associated with prescribing or monitoring errors
Several significant associations were found in the patient-level multivariable analysis (Table 3). For each additional unique medication item that the patient had received over the course of the 12-month data-collection period, there was a significant 16% increased risk of error (OR = 1.16, 95% CI =1.12% to 1.19%, P<0.001), and women were less likely than men to have a prescribing or monitoring error (OR = 0.66, 95% CI = 0.48 to 0.92, P = 0.013). Compared to those aged 15–64 years, patients who were <15 years had an 87% excess risk of error (OR = 1.87, 95% CI = 1.19 to 2.94, P = 0.006); those aged 65–74 years had a 68% excess risk (OR = 1.68, 95% CI = 1.04 to 2.73, P = 0.035), and those ≥75 years had a 94% excess risk (OR = 1.94, 95% CI = 1.19 to 3.19, P = 0.008). There was a significant 44% risk reduction in errors in practices with more than 10 000 patients (OR = 0.56, 95% CI = 0.31 to 0.99, P = 0.047). Furthermore, non-training practices had borderline significant 39% excess risk of errors (OR = 1.39, 95% CI = 0.97 to 2.01, P = 0.075) compared to training practices.
Table 3.
Patient-level model of risks associated with prescribing or monitoring errors (n = 1777)
| Factor | Univariate models (model 1) | Multivariable models (model 2) | ||||
|---|---|---|---|---|---|---|
|
|
|
|||||
| Odds ratio | 95% CI | P-value | Odds ratio | 95% CI | P-value | |
| Sex | ||||||
| Malea | 1.00 | 1.00 | ||||
| Female | 0.76 | 0.56 to 1.02 | 0.064 | 0.66 | 0.48 to 0.92 | 0.013 |
|
| ||||||
| Number of drugs | 1.17 | 1.13 to 1.20 | <0.001 | 1.16 | 1.12 to 1.19 | <0.001 |
|
| ||||||
| Age category, years | ||||||
| 15–64a | 1.00 | 1.00 | ||||
| 0–14 | 1.53 | 0.99 to 2.35 | 0.053 | 1.87 | 1.19 to 2.94 | 0.006 |
| 65–74 | 2.69 | 1.73 to 4.20 | <0.001 | 1.68 | 1.04 to 2.73 | 0.035 |
| ≥75 | 4.26 | 2.81 to 6.47 | <0.001 | 1.94 | 1.19 to 3.19 | 0.008 |
|
| ||||||
| Dispensing practice | ||||||
| Non dispensinga | 1.00 | 1.00 | ||||
| Dispensing practice | 1.05 | 0.73 to 1.51 | 0.781 | 0.70 | 0.26 to 1.88 | 0.476 |
|
| ||||||
| Training practice | ||||||
| Traininga | 1.00 | 1.00 | ||||
| Not a training practice | 1.33 | 0.98 to 1.81 | 0.065 | 1.39 | 0.97 to 2.01 | 0.075 |
|
| ||||||
| Practice size | ||||||
| 5000–10 000a | 1.00 | 1.00 | ||||
| <5000 patients | 0.83 | 0.58 to 1.17 | 0.281 | 0.88 | 0.58 to 1.34 | 0.553 |
| >10 000 patients | 0.74 | 0.49 to 1.11 | 0.147 | 0.56 | 0.31 to 0.99 | 0.047 |
|
| ||||||
| Location | ||||||
| Urbana | 1.00 | 1.00 | ||||
| Rural practice | 1.03 | 0.74 to 1.44 | 0.849 | 1.06 | 0.43 to 2.58 | 0.905 |
Reference group.
The findings from the prescription-level model are shown in Table 4 (univariate modelling) and Table 5 (multivariable modelling). Two multivariable models are shown in Table 5 and these relate to models 5 and 6 described in the methods. Model 5 included all relevant variables, but did not adjust for clustering by patient. Model 6 did adjust for clustering by patient, but it was not possible to include ‘formulation’ and ‘British National Formulary chapter’, owing to model non-convergence. Both models showed that prescribing a drug on the ‘monitoring list’ (see Box 1 for examples) was associated with a more than threefold increased risk of error (for model 6, OR = 3.63, 95% CI = 2.65 to 4.98, P<0.001). Neither model showed a significant association between the type of prescriber or type of computer system and prevalence of error. Model 5 showed that prescriptions from the following British National Formulary chapters were significantly associated with excess risk of prescribing or monitoring errors compared to the gastrointestinal reference category: cardiovascular system (OR = 2.37, 95% CI = 1.03 to 5.45, P = 0.042), infections (OR = 2.67, 95% CI = 1.17 to 6.11, P = 0.02), malignant disease and immunosuppression (OR = 6.77, 95% CI = 1.71 to 26.84, P = 0.006), musculoskeletal (OR = 6.97, 95% CI = 3.06 to 15.88, P<0.001), eye (OR = 4.92, 95% CI = 1.12 to 21.62), ear nose and oropharynx (OR = 4.60, 95% CI = 1.29 to 16.42, P = 0.019), and skin (OR = 5.78, 95% CI = 2.04 to 16.36, P = 0.001).
Table 4.
Prescription-level univariate models of risks associated with prescribing or monitoring errors: unadjusted and adjusted for patient clustering
| Factor | Univariate unadjusted modela (model 3) | Univariate adjusted modelb (model 4) | ||||
|---|---|---|---|---|---|---|
|
|
|
|||||
| Odds ratio | 95% CI | P-value | Odds ratio | 95% CI | P-value | |
| Medication on monitoring list | 3.32 | 2.56 to 4.30 | <0.001 | 3.57 | 2.68 to 4.75 | <0.001 |
|
| ||||||
| Repeat prescriptionc | 1.40 | 1.10 to 1.78 | 0.006 | 1.39 | 1.08 to 1.79 | 0.011 |
|
| ||||||
| Formulationd | ||||||
| Eye/ear drops | 2.01 | 1.21 to 3.33 | 0.007 | 2.08 | 1.20 to 3.59 | 0.009 |
| Inhalers | 0.41 | 0.21 to 0.81 | 0.010 | 0.40 | 0.20 to 0.79 | 0.009 |
| Injections | 0.51 | 0.24 to 1.10 | 0.447 | 0.51 | 0.23 to 1.11 | 0.091 |
| Liquid oral | 0.82 | 0.49 to 1.38 | 0.087 | 0.80 | 0.46 to 1.38 | 0.414 |
| Rectal | 0.80 | 0.11 to 5.95 | 0.827 | 0.97 | 0.12 to 7.72 | 0.973 |
| Topical | 0.98 | 0.70 to 1.37 | 0.905 | 1.00 | 0.70 to 1.43 | 0.992 |
|
| ||||||
| Prescriber typee | ||||||
| Salaried GP | 0.62 | 0.41 to 0.95 | 0.026 | 0.64 | 0.4 1 to 1.00 | 0.051 |
| Locum GP | 1.07 | 0.56 to 2.05 | 0.835 | 1.12 | 0.55 to 2.27 | 0.760 |
| Training GP | 1.36 | 0.68 to 2.71 | 0.380 | 1.34 | 0.63 to 2.85 | 0.441 |
| Non-medical prescriber | 0.99 | 0.31 to 3.17 | 0.982 | 0.87 | 0.25 to 3.06 | 0.832 |
| Other/unknown | 1.94 | 0.59 to 6.41 | 0.277 | 1.78 | 0.50 to 6.43 | 0.358 |
|
| ||||||
| British National Formulary chapterf | ||||||
| Cardiovascular system | 5.53 | 2.66 to 11.49 | <0.001 | |||
| Respiratory system | 1.58 | 0.65 to 3.84 | 0.315 | |||
| CNS | 2.06 | 0.94 to 4.49 | 0.07 | |||
| Infections | 2.37 | 1.07 to 5.24 | 0.034 | |||
| Endocrine | 2.87 | 1.23 to 6.73 | 0.015 | |||
| Obstetrics, gynaecology, urinary tract disorders | 1.09 | 0.33 to 3.66 | 0.887 | |||
| Malignant disease and immunosuppression | 14.00 | 3.84 to 51.07 | <0.001 | |||
| Nutrition and blood | 1.17 | 0.35 to 3.92 | 0.803 | |||
| Musculoskeletal | 6.38 | 2.87 to 14.21 | <0.001 | |||
| Eye | 6.61 | 2.74 to 15.92 | <0.001 | |||
| Ear nose and oropharynx | 3.50 | 1.29 to 9.50 | 0.014 | |||
| Skin | 3.52 | 1.63 to 7.59 | 0.001 | |||
| Immunology and vaccines | 2.18 | 0.74 to 6.37 | 0.155 | |||
|
| ||||||
| Computer systemg | ||||||
| EMIS PCS | 1.15 | 0.87 to 1.53 | 0.322 | 1.17 | 0.83 to 1.64 | 0.371 |
| Isoft Premiere | 0.67 | 0.45 to 0.98 | 0.040 | 0.64 | 0.40 to 1.00 | 0.051 |
| TPP | 1.13 | 0.79 to 1.63 | 0.507 | 1.06 | 0.68 to 1.65 | 0.812 |
CNS = central nervous system.
Model unadjusted for clustering by patient.
Model adjusted for clustering by patient.
Acute prescriptions.
Solid oral medication (pessaries category omitted).
GP partner.
Gastrointestinal (anaesthesia chapter omitted).
EMIS LV.
Table 5.
Prescription-level multivariable models of risks associated with prescribing or monitoring errors: unadjusted and adjusted for patient clustering
| Factor | Multivariable unadjusted model (model 5)a | Multivariable adjusted model (model 6)b | ||||
|---|---|---|---|---|---|---|
|
|
|
|||||
| Odds ratio | 95% CI | P-value | Odds ratio | 95% CI | P-value | |
| Medication on monitoring list | 3.18 | 2.05 to 4.94 | <0.001 | 3.63 | 2.65 to 4.98 | <0.001 |
|
| ||||||
| Repeat prescriptionc | 1.25 | 0.91 to 1.72 | 0.168 | 0.98 | 0.73 to 1.31 | 0.903 |
|
| ||||||
| Formulationd | ||||||
| Eye/ear drops | 1.43 | 0.39 to 5.23 | 0.586 | |||
| Inhalers | 0.51 | 0.21 to 1.27 | 0.149 | |||
| Injections | 0.41 | 0.06 to 3.07 | 0.387 | |||
| Liquid oral | 1.08 | 0.60 to 1.95 | 0.794 | |||
| Rectal | 1.93 | 0.24 to 15.42 | 0.535 | |||
| Topical | 0.66 | 0.31 to 1.42 | 0.287 | |||
|
| ||||||
| Prescriber typee | ||||||
| Salaried GP | 0.71 | 0.46 to 1.11 | 0.13 | 0.76 | 0.47 to 1.22 | 0.258 |
| Locum GP | 1.16 | 0.60 to 2.28 | 0.655 | 1.23 | 0.60 to 2.54 | 0.576 |
| Training GP | 1.42 | 0.67 to 3.01 | 0.353 | 1.66 | 0.74 to 3.72 | 0.218 |
| Non-medical prescriber | 1.55 | 0.47 to 5.13 | 0.469 | 1.09 | 0.30 to 3.91 | 0.898 |
| Other/unknown | 1.78 | 0.52 to 6.14 | 0.358 | 1.97 | 0.54 to 7.23 | 0.305 |
|
| ||||||
| British National Formulary chapterf | ||||||
| Cardiovascular system | 2.37 | 1.03 to 5.45 | 0.042 | |||
| Respiratory system | 2.26 | 0.83 to 6.17 | 0.11 | |||
| CNS | 2.09 | 0.95 to 4.63 | 0.068 | |||
| Infections | 2.67 | 1.17 to 6.11 | 0.02 | |||
| Endocrine | 1.91 | 0.78 to 4.72 | 0.159 | |||
| Obstetrics, gynaecology, urinary tract disorders | 1.41 | 0.41 to 4.80 | 0.584 | |||
| Malignant disease and immunosuppression | 6.77 | 1.71 to 2 6.84 | 0.006 | |||
| Nutrition and blood | 1.22 | 0.36 to 4.15 | 0.752 | |||
| Musculoskeletal | 6.97 | 3.06 to 15.88 | <0.001 | |||
| Eye | 4.92 | 1.12 to 21.62 | 0.035 | |||
| Ear nose and oropharynx | 4.60 | 1.29 to 16.42 | 0.019 | |||
| Skin | 5.78 | 2.04 to 16.36 | 0.001 | |||
| Immunology and vaccines | 5.91 | 0.60 to 58.00 | 0.127 | |||
|
| ||||||
| Computer systemg | ||||||
| EMIS PCS | 1.06 | 0.79 to 1.42 | 0.706 | 1.13 | 0.80 to 1.60 | 0.501 |
| Isoft Premiere | 0.68 | 0.46 to 1.01 | 0.055 | 0.64 | 0.40 to 1.01 | 0.058 |
| TPP | 1.05 | 0.71 to 1.56 | 0.803 | 0.94 | 0.58 to 1.52 | 0.81 |
CNS = central nervous system.
Model unadjusted for clustering by patient.
Model adjusted for clustering by patient but formulation and chapter not included in the model due to non-convergence of model.
Acute prescriptions.
Solid oral medication (pessaries category omitted).
GP partner.
Gastrointestinal (anaesthesia chapter omitted).
EMIS LV.
DISCUSSION
Summary
The prevalence of prescriptions with prescribing or monitoring errors found in the study was 4.9%. Around one in six of the population studied were exposed to a prescribing or monitoring error if they had been prescribed any medication during the 12-month data-collection period. Given that the study used a sample of practices and patients that was reasonably representative of England, it is believed that these figures give an estimate of the likely prevalence of prescribing and monitoring errors across the country. The vast majority of the errors were judged to be of minor or moderate severity, with one in 550 prescriptions associated with a severe error.
Strengths and limitations
The 15 general practices included in the study were reasonably representative of English general practices in terms of list size, number of GPs per practice, age profile, and points achieved in the Quality and Outcomes Framework, although they were more deprived. The sampling strategy enabled GP training practices and dispensing practices to be included. Compared with overall figures for England, the practices were more deprived.
To avoid any problems with sampling bias at the practice level, a random sample of patients was obtained from each practice. However, it is possible that the recruited practices had relatively high levels of interest in prescribing compared with other practices, and a greater openness to external scrutiny of potential prescribing errors, which could have caused the study to underestimate the true rate of prescribing errors. The pharmacists who collected data for the study all had clinical experience and were provided with training in identifying errors. Nevertheless, it is possible that they varied in their ability to detect potential prescribing and monitoring errors. A validated approach to the definition of error was used in the study, and a multidisciplinary panel was used to decide whether an error was present or not. A consistent approach was also used throughout the study to the application of the error definitions used. Nevertheless, it is possible that a different panel would have come to different judgements, and this highlights one of the challenges of consistent application of error definitions across studies. The present study focused on errors associated with prescribing and blood test monitoring. With more resources, it would have been helpful to have investigated errors associated with other aspects of medicines management in primary care, as done recently in English care homes.6
The multivariable analysis undertaken allowed associations between the prevalence of prescribing and monitoring errors and a range of factors to be explored, while taking account of potential confounders. Statistical significance for all analyses was set at <0.05 (two-sided) and some may regard this as relatively weak. Nevertheless, many of the findings were highly statistically significant.
Comparison with existing literature
This study was larger than previous UK studies, and used robust methods and definitions. The error rates are in line with those reported previously,3–6 but the large and representative sample makes the estimates more precise. The most recent of previous studies found a prevalence of prescribing error of 8.3% and monitoring error of 14.7%,7 albeit it in residential and nursing home patients.7 In contrast, the prevalence of prescribing errors in UK hospitals was 8.9%,15 and in a US hospital was 6.2%.16 Internationally, prescribing errors affect a median of 7% (IQR = 2–14) of medication orders in hospitals.17
The prevalence of prescribing or monitoring errors was particularly high in certain groups of patients: older people, children, men, and those on multiple medications. The high prevalence in older people has been noted elsewhere,6 and is of particular concern because of greater susceptibility of older people to experiencing medication-related harm. The high prevalence in children highlights the need to ensure that correct doses are prescribed with clear dosage instructions, but despite concerns from another study, off-label and unlicensed prescribing was not found to be an important cause of error.18 The finding that patients taking multiple medications have increased risk of error (an association that went beyond the additive effect of each medicine) is of concern, especially as those taking 10 or more medicines had an almost 50% risk of an error during a 12-month period. These findings support those of other studies,19 and highlight the need to find more effective ways of managing polypharmacy.20 The lower risk of error in women is a surprising finding, and is in contrast to findings suggesting that women are at greater risk of adverse drug events.21 It is possible, however, that GPs are more cautious when prescribing for women, hence the lower risk of error.
Drugs associated with a higher risk of error included those requiring a monitoring test, musculoskeletal drugs (most commonly non-steroidal antiinflammatory drugs), and those used for malignant disease and immunosuppression (although the sample was small in this last group). This highlights the need for GPs to pay particular attention to correct prescribing and monitoring of medications associated with the highest risk of error (and associated adverse events). Some of the most serious errors related to the prescription of warfarin without access to blood-test monitoring (Box 2). Given the dangers associated with this drug, it is essential that prescribers have access to the latest international normalised ratio before prescribing, and adhere to local and national policies.22 In addition, this study identified two cases in which drugs were prescribed for patients with a computer-recorded allergy to the drug. Given the design of GP computer systems, it is almost certain that these prescribers would have overridden a hazard alert. Therefore, there is a need to make it more difficult for prescribers to override alerts for the most serious hazards.23
Box 2. Examples of errors judged to be of different grades of severity
Minor errors
1-year old girl prescribed amoxicillin 125 mg/5 ml suspension twice during the same consultation. One was for 2.5 ml three times daily for 1 week, and the other for 5 ml three times daily for 1 week.
Topical betamethasone 0.1% prescribed in an adult patient. No directions given relating to the frequency of application.
Betamethasone cream 0.1% prescribed ‘to be applied sparingly for 1 week’ for a 5-year-old child. No frequency of use specified.
29-year-old patient prescribed co-amoxiclav tablets 21 × 500 mg/125 mg for sinusitis. Dose and frequency not specified on prescription.
Moderate errors
64-year old patient prescribed ibuprofen 400 mg, one tablet to be taken three times daily, after a road accident. No concomitant medication was prescribed for gastric protection. Patient also on aspirin for peripheral vascular disease.
Indometacin 50 mg prescribed with dosage instructions ‘as directed’, with no instructions on frequency or maximum daily intake. Patient diagnosed with alcoholic cirrhosis of the liver.
Patient prescribed levothyroxine 25 μg, one tablet to be taken every day. Thyroid function tests were requested but not done.
Severe errors
62-year-old patient with documented allergy to penicillin; prescribed a course of oral flucloxacillin.
Aciclovir 200 mg prescribed, one tablet to be taken five times a day for widespread cold sores, to a patient with a documented severe allergic reaction to aciclovir.
Older patients on warfarin. Last documented international normalised ratios were more than 2 years previously (n = 7 errors).
93-year-old patient prescribed warfarin. Last documented international normalised ratio was more than 1 year previously. Patient did not attend three consecutive anticoagulant appointments, but warfarin prescription continued.
In addition to the points made above, there are several other approaches by which prescribing and monitoring errors could be reduced in general practices, based on the findings of this study. Given that the most common type of error detected in the study was ‘incomplete dosage instructions’, it is important for GPs to take the time needed to provide clear and complete instructions, particularly for high-risk medicines. To help with this, the latest GP computer systems are able to provide ‘order sentences’ (dosage instructions) that are appropriate for each medication prescribed. Greater use of this functionality by GPs could substantially reduce error rates. Given that monitoring errors are another common category of error, it is important for general practices to ensure that they have robust systems in place to make sure that essential blood-test monitoring is done. GP computer systems can help with identifying patients who need monitoring, and this can also be done at prescription reviews. These reviews can also act as an opportunity to identify and correct errors, but the present study showed that repeat prescriptions were no less likely to contain an error than acute prescriptions. This suggests that prescription reviews may not be as effective as they could be, especially in patients taking multiple medications, and in higher-risk complex patients. It may be helpful to consider different models for conducting these reviews. For example, involving pharmacists and geriatricians in the review of older patients on multiple medications may help reduce unnecessary polypharmacy,20 and also reduce errors.
Pharmacists can also help GPs to reduce prescribing errors by identifying patients at risk (based on searches of the GP computer system), and facilitating appropriate management of any problems identified.24 This approach, would be suitable for tackling some of the errors observed in this study, particularly monitoring errors. Overall, there is considerable scope for GP computer systems to help reduce many of the prescribing errors identified in this study. In addition to points made already, the most advanced systems can help provide advice on correct dosage, based upon a patient’s age, weight, and/or renal function. They can also provide alerts for contraindicated prescribing. A key challenge, however, is to avoid over-alerting prescribers, and to design systems to minimise the risk of accidentally overriding alerts for potentially fatal errors.
It should be noted that the study reported in this paper was part of a larger investigation funded by the General Medical Council (GMC) in the UK, and further details are available from the GMC website.25
Implications for future research
Further research is now needed to develop and test interventions to reduce the prevalence of prescribing and monitoring errors in general practices. It is already known that computerised clinical decision support can help reduce prescribing errors in hospitals,26 but there is a need for more studies in primary care, particularly to evaluate the more sophisticated systems in current use and to determine whether there are safety benefits to reducing the hazard-alert burden on prescribers. Further research is also needed into whether the prevalence of prescribing and monitoring errors in general practice can be reduced by education, training, audit, and practical support, as demonstrated previously.24 It is also important to evaluate the effectiveness of systems-based approaches to reducing prescribing and monitoring errors whereby processes used in general practices such as repeat prescribing, medication review, and monitoring are optimised. Finally, it would be helpful to do further research to determine the extent to which the prevalence of prescribing and monitoring errors can be reduced by more extensive involvement of clinical pharmacists in promoting high-quality, safe prescribing in general practice. For example, a model could be developed in which pharmacists work with GPs to address the most important factors associated with error.
Prescribing or monitoring errors occurred in one in 20 prescription items, and most of these were judged to be of mild to moderate severity; one in 550 prescription items contained a severe error. The risks of error were higher in children, older people, male patients, and those on multiple medications. Several groups of drugs were associated with higher risks of error, including those requiring blood-test monitoring. Having identified the most common and important errors, and the factors associated with these, strategies to prevent future errors should be developed, based on the study findings.
Acknowledgments
The Centre for Medication Safety and Service Quality is affiliated with the Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust, which is funded by the National Institute of Health Research. We thank: Brian Hancock for designing the Microsoft Access database used in the study; Simon Dunton, Philip Lee, Colin MacFarlane, Cornelia Schnelle, and Dulcie Watanabe from the BNF and RPS Publishing for providing a database of all the drugs and preparations used in the BNF; the PCTs and related bodies who helped set up and run the project, including Dr Doug Black, Beth Hird, Rachel Illingworth, Sanjeet Johal, Richard Jones, Dr Barhey Manraj, Jonathon Mason, Cathy Quinn, and Amanda Sullivan; the general practices that took part in the study; the pharmacists who collected data for the study, Tess Dawoud, Gillian Gookey, Roshan Jayaseelan, and Uzma Rashid); Clare Randall for providing administrative support and formatted the report; April McCambridge for assisting with referencing; Liesbeth Dermont and Bo Ye for assistance with coordinating assessment of the severity of the errors identified; Ndeshi Salema for thoroughly checking every entry in the database against the original data collection forms; and Gill Gookey, Chris Love, Alex MacDonald, and Rupert Payne for conducting assessments of the severity of the errors identified.
Funding
The study was funded by the GMC. The funders had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the article for publication. The researchers are independent from the funders.
Ethical approval
Approval was obtained from Nottingham Research Ethics Committee 1 (Reference 10/H0403/29; 27 April 2010). NHS Research and Development approval was obtained from participating PCTs. The ethics committee agreed that patients did not have to give informed consent. No personal identifiable data were recorded.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
REFERENCES
- 1.NHS Information Centre . Prescriptions dispensed in the community: England, statistics for 2000 to 2010. Leeds: The Health and Social Care information Centre; 2011. Prescribing and primary care services. https://catalogue.ic.nhs.uk/publications/prescribing/primary/pres-disp-com-eng-2001-11/pres-disp-com-eng-2001-11-rep.pdf (accessed 5 Jun 2013). [Google Scholar]
- 2.Maxwell S, Walley T, Ferner RE. Using drugs safely. BMJ. 2002;324(7343):930–931. doi: 10.1136/bmj.324.7343.930. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Quinlan P, Ashcroft DM, Blenkinsopp A. Medication errors: a baseline survey of interventions recorded during the dispensing process in community pharmacies. Int J Pharm Pract. 2002;10(S1):R67. [Google Scholar]
- 4.Claesson C, Burman K, Nilsson J, Vinge E. Prescription errors detected by Swedish pharmacists. Int J Pharm Pract. 1995;3:151–156. [Google Scholar]
- 5.Howard RL, Avery AJ, Slavenburg S, et al. Which drugs cause preventable admissions to hospital? A systematic review. Br J Clin Pharmacol. 2007;63(2):136–147. doi: 10.1111/j.1365-2125.2006.02698.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Shah SNH, Aslam M, Avery AJ. A survey of prescription errors in general practice. Pharm J. 2001;267:860–862. [Google Scholar]
- 7.Barber ND, Alldred DP, Raynor DK, et al. Care homes’ use of medicines study: prevalence, causes and potential harm of medication errors in care homes for older people. Qual Saf Health Care. 2009;18(5):341–346. doi: 10.1136/qshc.2009.034231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.National Patient Safety Agency . Patient safety incident reports in the NHS: national reporting and learning system quarterly data summary. London: National Patient Safety Agency; 2008. [Google Scholar]
- 9.National Clinical Assessment Service . ‘NCAS CASEWORK’. The first eight years. UK: National Clinical Assessment Service; 2009. [Google Scholar]
- 10.Avery AJ, Rodgers S, Cantrill JA, et al. Protocol for the PINCER trial: a cluster randomised trial comparing the effectiveness of a pharmacist-led IT-based intervention with simple feedback in reducing rates of clinically important errors in medicines management in general practices. Trials. 2009;10:28. doi: 10.1186/1745-6215-10-28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dean B, Barber N, Schachter M. What is a prescribing error? Qual Health Care. 2000;9(4):232–237. doi: 10.1136/qhc.9.4.232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Alldred DP, Standage C, Zermansky AG, et al. Development and validation of criteria to identify medication-monitoring errors in care home residents. Int J Pharm Pract. 2008;16:317–323. [Google Scholar]
- 13.Dean BS, Barber ND. A validated, reliable method of scoring the severity of medication errors. Am J Health Syst Pharm. 1999;56(1):57–62. doi: 10.1093/ajhp/56.1.57. [DOI] [PubMed] [Google Scholar]
- 14.NHS Information Centre . Attribution dataset GP-registered populations — 2010. Leeds: NHS Information Centre; http://www.hscic.gov.uk/searchcatalogue?productid=4285&q=title%3a%22Attribution+data+set+GP-registered+populations%22&sort=Relevance&size=10&page=1#top (accessed 5 Jun 2013). [Google Scholar]
- 15.Dornan T, Ashcroft D, Heathfield H, et al. An in depth investigation into causes of prescribing errors by foundation trainees in relation to their medical education. EQUIP Study 2009. London: General Medical Council; [Google Scholar]
- 16.Bobb A, Gleason K, Husch M, et al. The epidemiology of prescribing errors: the potential impact of computerized prescriber order entry. Arch Intern Med. 2004;164(7):785–792. doi: 10.1001/archinte.164.7.785. [DOI] [PubMed] [Google Scholar]
- 17.Lewis PJ, Dornan T, Taylor D, et al. Prevalence, incidence and nature of prescribing errors in hospital inpatients: a systematic review. Drug Saf. 2009;32(5):379–389. doi: 10.2165/00002018-200932050-00002. [DOI] [PubMed] [Google Scholar]
- 18.McIntyre J, Conroy S, Avery A, et al. Unlicensed and off label prescribing of drugs in general practice. Arch Dis Child. 2000;83(6):498–501. doi: 10.1136/adc.83.6.498. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Guthrie B, McCowan C, Davey P, et al. High risk prescribing in primary care patients particularly for honourable to adverse drug events: cross sectional population database analysis in Scottish general practice. BMJ. 2011;342:d3514. doi: 10.1136/bmj.d3514. [DOI] [PubMed] [Google Scholar]
- 20.The Model of Care Polypharmacy Working Group, Quality and Efficiency Support Team Scottish Government Health, and Social Care Directorates . Polypharmacy guidance. October 2012. Edinburgh: NHS Scotland and the Scottish Government; 2012. http://www.central.knowledge.scot.nhs.uk/upload/Polypharmacy%20full%20guidance%20v2.pdf (accessed 5 Jun 2013). [Google Scholar]
- 21.Zopf Y, Rabe C, Neubert A, et al. Women encounter ADRs more often than do men. Eur J Clin Pharmacol. 2008;64(10):999–1004. doi: 10.1007/s00228-008-0494-6. [DOI] [PubMed] [Google Scholar]
- 22.National Patient Safety Agency . Actions that can make anticoagulant therapy safer: patient safety alert 18 (Ref: NPSA/2007/18) London: National Patient Safety Agency; 2007. http://www.npsa.nhs.uk/EasySiteWeb/GatewayLink.aspx?alId=10797 (accessed 5 Jun 2013). [Google Scholar]
- 23.Phansalkar S, Edworthy J, Hellier E, et al. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc. 2010;17(5):493–501. doi: 10.1136/jamia.2010.005264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Avery AJ, Rodgers S, Cantrill JA, et al. A pharmacist-led information technology intervention for medication errors (PINCER): a multicentre, cluster randomised, controlled trial and cost-effectiveness analysis. Lancet. 2012;379(9823):1310–1319. doi: 10.1016/S0140-6736(11)61817-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Avery T, Barber N, Ghaleb M, et al. Investigating the prevalence and causes of prescribing errors in general practice: the PRACtICe Study. London: GMC; 2012. http://www.gmc-uk.org/about/research/12996.asp (accessed 5 Jun 2013). [Google Scholar]
- 26.Schedlbauer A, Prasad V, Mulvaney C, et al. What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behaviour? J Am Med Inform Assoc. 2009;16(4):531–538. doi: 10.1197/jamia.M2910. [DOI] [PMC free article] [PubMed] [Google Scholar]