Abstract
Background
Electronic health record (EHR) systems are used by clinicians to record patients’ medical information, and support clinical activities such as prescribing. In England, healthcare professionals are advised to ‘prescribe generically’ because generic drugs are usually cheaper than branded alternatives, and have fixed reimbursement costs. ‘Ghost-branded generics’ are a new category of medicines savings, caused by prescribers specifying a manufacturer for a generic product, often resulting in a higher reimbursement price compared with the true generic.
Aim
To describe time trends and practice factors associated with excess medication costs from ghost-branded generic prescribing.
Design and setting
Retrospective cohort study of English GP prescribing data and EHR deployment data.
Method
A retrospective cohort study was conducted, based on data from the OpenPrescribing.net database from May 2013 to May 2019. Total spending on ghost-branded generics across England was calculated, and excess spend on ghost-branded generics calculated as a percentage of all spending on generics for every CCG and general practice in England, for every month in the study period.
Results
There were 31.8 million ghost-branded generic items and £9.5 million excess cost in 2018, compared with 7.45 million ghost-branded generic items and £1.3 million excess cost in 2014. Most excess costs were associated with one EHR, SystmOne, and it was identified that SystmOne offered ghost-branded generic options as the default. After informing the vendor, the authors monitored for subsequent change in costs, and report a rapid decrease in ghost-branded generic expenditure.
Conclusion
A design choice in a commonly used EHR has led to £9.5 million in avoidable excess prescribing costs for the NHS in 1 year. Notifying the vendor led to a change in user interface and a rapid, substantial spend reduction. This finding illustrates that EHR user interface design has a substantial impact on the quality, safety, and cost-effectiveness of clinical practice; this should be a priority for quantitative research.
Keywords: clinical software, cohort studies, drugs prescribing, ghost-branded generics, primary care
INTRODUCTION
Electronic health record (EHR) systems are used by clinicians to record patients’ medical information, and to support clinical activities such as prescribing or test ordering. There is limited literature describing how EHR system design features can impact on clinical practice. However, this typically uses indirect evidence: qualitative research observing or interviewing clinicians; questionnaires interrogating clinicians about their experiences of EHRs; or descriptive analyses of clinicians’ spontaneous reports of errors and safety issues.1–3 There has been little quantitative analysis exploring the impact of different EHR systems on clinical practice, other than small studies evaluating change in practice following the implementation of specific new alerts or defaults.4,5
In England, healthcare professionals are advised by the NHS to ‘prescribe generically’ because generic versions of the drug are usually cheaper than branded alternatives. This advice is simple to remember, generic prescribing is widespread, and one estimate attributes savings of £7.1 billion to the policy over the last 40 years.6 Doctors occasionally deviate from this practice for clinical reasons, and sometimes for financial savings where specific brands (including specific brands of a medication that is available in generic form) are cheaper than the fixed generic price.
In December 2018 the authors uncovered unexpected excess expenditure for primary care prescribing in England’s prescribing data (Box 1), where prescriptions that were apparently prescribed generically had been charged above the standard NHS tariff prices, amounting to an estimated excess cost of more than £9.5 million per annum. The authors discovered they were not ‘true’ generic items and so termed the items responsible ‘ghost-branded generics’. A ghost-branded generic is prescribed and dispensed where a prescriber selects a generic product but specifies the manufacturer, usually inadvertently. In such cases, the pharmacy is reimbursed for this specific manufacturer’s version of the generic, whose price may substantially exceed the standard generic price. However, the NHS prescribing data reports the prescription as if it had been for a true generic.
Box 1.
Ghost-branded generics and how they were discovered
| The price per unit tool on OpenPrescribing is an innovative way to identify large potential cost savings in NHS primary care prescribing.8,9 In short, by writing a prescription slightly differently (for example, tablets versus capsules), a saving can be made. The authors were challenged by the fact that this tool showed that atorvastatin 20 mg,10 prescribed identically, was unexpectedly costing differing amounts. This should not be the case; investigations uncovered a whole new area of reimbursement problems, which the authors termed ghost-branded generics. For prescriptions written in primary care in England, the NHS reimburses community pharmacies for the medicines they purchase. The reimbursement price is set monthly in the NHS Drug Tariff for generic medicines. For branded medicines the price is set by the manufacturer and listed in the NHS Dictionary of Medicines and Devices (DM+D). Medicines are broadly divided into three categories for reimbursement purposes: proprietary, generic, and branded generic. The following is an illustrated example of the contraceptive desogestrel:
Following discussions with the NHS Business Services Authority, the authors discovered that important information about medicines reimbursement is not shared in the prescribing dataset. When GPs are apparently prescribing generically, it is possible that they are specifying which manufacturer of the particular generic the pharmacy should supply, in brackets, after the name of the drug. The pharmacy must supply this manufacturer’s brand, and they are then reimbursed at the specific price provided in DM+D, not the generic Drug Tariff price. So it is treated as a ‘branded generic’ for dispensing and reimbursement purposes, but is coded as a normal generic for publication of GP prescribing patterns. However, it is the authors’ experience that only on very few occasions do GPs intend to specify a brand. After investigating ghost-branded Generics on two of the most common EHRs in England the authors determined that the majority of excess costs were attributable to a single EHR, SystmOne. Further details of the initial investigation can be found on the DataLab blog.11 |
This article describes how the authors discovered that almost all the £9.5 million excess treatment costs were attributable to a design feature in one EHR system used by 34% of England’s practices in 2016;7 and how the impact of an EHR design change made by the vendor was evaluated after alerting them to the issue.
METHOD
Study design
The study was a retrospective cohort study in prescribing data from all English NHS general practices and clinical commissioning groups (CCGs), including cross-sectional and longitudinal analysis.
How this fits in
| Ghost-branded generics were responsible for an excess cost in general practice prescribing of £9.5 million in 2018 and were previously unknown apart from to a few individuals involved in technical details of reimbursing the costs of medicine. A single electronic health record, SystmOne, was responsible for most excess costs because of a single design choice in the user interface defaults. Modification of the EHR user interface has led to substantial reductions in costs associated with ghost-branded generics, illustrating the substantial impact EHRs can have on quality, safety, and cost-effectiveness of clinical practice. Prescribers and practices can monitor their own prescribing of ghost-branded generics online (https://openprescribing.net/measure/ghost_generic_measure/all-england), and can audit and address any concerns in their own prescribing or locally adapted defaults on their EHR. |
Data preparation and sources
Data were extracted from the OpenPrescribing.net database. This imports openly accessible primary care prescribing data from the large monthly files published by the NHS Business Services Authority (see Box 2 for a description of the English NHS organisations mentioned in the article). These data are sourced from community pharmacy claims data and therefore contain all items that have been dispensed. They contain one row for each different medication and dose, in each prescribing organisation in NHS primary care in England, describing the number of items (that is, prescriptions issued) and the total cost each month since mid-2010.12 All available data were extracted for standard general practices, with other organisations such as prisons and hospitals excluded, according to the NHS Digital dataset of practice characteristics. Numbers of patients registered at each practice were obtained from NHS Digital,13 and details on EHR use in each general practice were also obtained from NHS Digital (personal communication, April 2019). The study population was all NHS GP practices in England, as defined as setting = 4 by NHS Digital practice characteristics, May 2013 and May 2019.
Box 2.
NHS organisations in England featured in this article
| Clinical commissioning group (CCG) |
| A collection of GP practices working together to plan, commission, and pay for healthcare services in a local area. There are approximately 200 CCGs in England and they make payments for all prescriptions written by GPs in their membership and bear any excess cost related to ghost-branded generics. |
| Medicines and Healthcare products Regulatory Agency |
| The agency that regulates medicines in the UK. Responsibilities include issuing licences, approving medicines names, and issuing guidance on safety. |
| NHS Business Services Authority |
| An NHS body that calculates the remuneration and reimbursement due to pharmacies across England, and publishes data on prescribing. |
| NHS Digital |
| The national information and technology partner to the health and care system. It publishes various statistics, such as the amount of patients registered at a general practice, and is also responsible for procuring EHRs on behalf of the NHS. |
| NHSX |
| A national organisation responsible for digital transformation and lead policy, implementation, and change related to it. |
| The Phoenix Partnership (TPP) |
| A healthcare technology company, which produces SystmOne, a commonly used EHR across England. |
Identification of ghost-branded generic prescribing
Ghost-branded generics were identified as products where the reimbursement price (net cost) is different from the NHS Drug Tariff price, excluding those medicines that had a known in-month change from the Drug Tariff price, usually due to stock shortages and known as a ‘price concession’ applied.14 Data were excluded for prescriptions dispensed before May 2013 because ‘price concessions’ were not uniformly and automatically paid before this.
Trends and variations in ghost-branded generic prescribing over time
Prescribing of all ghost-branded generic items and excess costs were plotted on a time series graph, and a table of the top 10 chemical substances affected in 2018 was produced. Total spending on ghost-branded generics across England was calculated. Excess spend on ghost-branded generics was then calculated as a percentage of all spending on generics for every CCG and practice in England, for every month in the study period. This takes into account underlying variation in generic prescribing between organisations. The data are presented as deciles for each month. Lastly, excess ghost-branded generics cost was calculated and plotted, broken down by the EHR system used by each practice.
EHR system user interface evaluation
Having identified that the excess costs were attributable to practices using SystmOne,11 a senior pharmacist visited two practices, one using EMIS and one SystmOne, in order to observe the processes necessary to issue a prescription for generic atorvastatin in each EHR system.
Evaluating the impact of a change to the user interface medicines prescribing screen in a commonly used EHR, SystmOne
Having established that the ghost-branded generic was being presented at the top of the ‘picking list’ in SystmOne, the authors notified the EHR vendor, The Phoenix Partnership (TPP), which rapidly implemented a change to the user interface. To assess the impact of this change, prescribing of all ghost-branded generic items and spend was plotted on a time series graph for every practice using the SystmOne EHR, highlighting significant points on the timeline, including a change to the pick list on the medicines prescribing screen. All CCGs were also plotted on a time series graph and CCGs were identified that have reduced their prescribing of ghost-branded generics by 25%.
Software and reproducibility
Data management was performed using Python and Google BigQuery, with analysis carried out using Python. Data, as well as all code for data management and analysis, are archived online on Github.15
Patient and public involvement
The authors’ website (OpenPrescribing.net) is an openly accessible data explorer for all NHS England primary care prescribing data, which receives a large volume of user feedback from professionals, patients, and the public. This feedback is used to refine and prioritise the informatics tools and research activities. Patients were not formally involved in developing this specific study design.
RESULTS
Characteristics of cohort
A total of 8132 standard general practices prescribed at least one ghost-branded generic across the whole study period and were included. In December 2018, the month ghost-branded generics were discovered, 7145 general practices were included spanning 192 CCGs, with 2684 (38%) practices using SystmOne EHR and 4090 (57%) using EMIS, with the remaining 371 (5%) practices using Vision or Microtest. No standard general practices were excluded.
Trends and variation in ghost-branded generic prescribing over time
Figure 1 shows the excess cost of prescribing of ghost-branded generics (Figure 1a) and the total number of ghost-branded generic items (Figure 1b). In 2014 there were 7.45 million ghost-branded generic items and £1.3 million excess cost; in 2018 there were 31.8 million ghost-branded generic items and £9.5 million excess cost. Figure 1c shows that since May 2013 excess costs associated with SystmOne have risen consistently, whereas the costs in practices using EMIS and other EHR systems have remained low (see Supplementary Figure S1 for details of market share of all EHR systems). Table 1 shows the chemical substances that were responsible for the most excess cost associated with ghost-branded generics across England in 2018. Although atorvastatin had the greatest associated excess cost (£1.7 million), several items used in the treatment of mental health conditions also appear.
Figure 1.
Prescribing of ghost-branded generics across all standard general practices in England, from May 2013 to May 2019. (a) Excess cost compared with tariff prices (£); (b) number of items prescribed; (c) total excess cost (£) of ghost-branded generic prescribing grouped by EHR system. EHR = electronic health record. GBG = ghost-branded generic.
Table 1.
Top 10 ghost-branded generic chemical substances ranked in order of excess cost (£) compared with tariff prices, in 2018, across all standard general practices in England combined
| Chemical substance | Number of items | Excess spend |
|---|---|---|
| Atorvastatin | 5 559 934 | £1 694 636 |
| Pregabalin | 159 404 | £665 385 |
| Memantine hydrochloride | 100 178 | £512 713 |
| Montelukast | 241 626 | £298 154 |
| Aripiprazole | 18 605 | £269 940 |
| Losartan potassium | 786 568 | £269 048 |
| Quetiapine | 89 215 | £235 940 |
| Amlodipine | 2 719 024 | £219 620 |
| Trazodone hydrochloride | 26 253 | £213 471 |
| Prednisolone | 103 825 | £180 959 |
Figure 2 illustrates the trends and variation in excess cost of prescribing of ghost-branded generics across English CCGs (Figure 2a) and practices (Figure 2b) between May 2013 and May 2019 as a proportion of their total spend on generic prescribing. In December 2018, the month ghost-branded generics were discovered, the median proportion across all CCGs was 0.13% while the maximum was 1.55% (25th to 75th percentile range = 0.01% to 0.42%). The variation among practices was even more striking, with 60% of practices having close to 0% excess spend, whereas the 99th percentile had a proportion of 2.78%. Among practices using SystmOne (Figure 2c), most practices had some excess spend on ghost-branded generics and at the 99th percentile this accounted for 4% as a proportion of all generic spending compared with practices using EMIS, where the 99th percentile accounted for 0.4%.
Figure 2.
Excess cost of prescribing ghost-branded generics as a proportion of total generic spend, across (a) all CCGs; (b) all general practices; and (c) general practices using SystmOne EHR in England, May 2013 to May 2019. CCG = clinical commissioning group. EHR = electronic health record.
EHR system user interface evaluation
Having identified SystmOne as the EHR used in general practices where most of the excess costs related to ghost-branded generics occurred, user research was conducted on each EHR interface. Specifically, the authors set out to prescribe a generic medicine in EMIS and in SystmOne. In SystmOne, when generating a prescription for atorvastatin, the user is given a ‘picking list’ of medication options that includes ghost-branded generics (that is, the medicine and the manufacturer name) alongside true generics; in EMIS these ghost-branded generics are not presented in the picking list. Furthermore, when a specific formulation is selected in SystmOne, this is stored as a prioritised form of the medication, and is presented at the top of the picking list for all subsequent occasions when the medicine is prescribed, for all patients in the practice. This user interface feature is intended to save time by prompting prescribers to select previously used medicines more easily; however, in this case it reinforces a less desirable choice of medicine.
Evaluating the impact of a change to the user interface medicines prescribing screen in a commonly used EHR, SystmOne
In December 2018 the authors posted concerns about ghost-branded generics informally in a blog post, and notified the SystmOne vendor TPP of their concerns. TPP implemented a change to its user interface in February 2019 to prevent the prescribing of the affected items in the future and provided template searches to identify current repeat prescriptions. Figure 1c shows a substantial decrease in the cost of ghost-branded generics across the NHS in SystmOne practices following these two actions. Fifteen CCGs (8%) were identified that have achieved a very substantial reduction (Figure 3).
Figure 3.
The percentage excess cost of prescribing ghost-branded generics as a percentage of total generic spend, across the 30 highest spending CCGs. Those that have seen >25% reduction have been highlighted. CCG = clinical commissioning group.
DISCUSSION
Summary
Ghost-branded generics cost the NHS an extra £9.5 million in 2018. The authors were able to identify these excess costs using NHS open data; to attribute them to a single design feature of a popular EHR system; and to initiate a change in that EHR user interface by the vendor. This in turn led to a rapid, substantial spend reduction, illustrating the impact of EHR user interface on prescribing behaviour.
Strengths and limitations
All typical practices in England were included, thus minimising the risk of a biased sample. Real prescribing and spending data were used that are sourced from pharmacy claims data used for very substantial payments: CCGs and pharmacies are both therefore highly motivated to ensure these data are accurate. A small number of atypical practice settings such as walk-in centres were excluded because these generally do not issue repeat prescriptions for medicines, and no data on their EHRs are available. The data in this study do not include hospital medicines data, but the authors do not expect the same issue to occur in hospitals, because medicines are procured differently and the use of electronic prescribing and EHRs in secondary care in England is more limited.
One weakness of this study is that ghost-branded generics were identified based on their reimbursement price, which may not be 100% accurate. This is because the NHS Business Services Authority collects, but does not share, the more granular prescribing data needed to identify ghost-branded generics with complete accuracy (based on the presence of a generic name and manufacturer name in the ‘prescription item’ field).
A key strength of this analysis is that entirely open data were used, and all the code and data are shared for critical evaluation, re-use, and reproducibility.
Comparison with existing literature
To the authors’ knowledge this article represents the first research using large-scale national data and qualitative observations to identify a shortcoming in clinical practice caused by an EHR design feature. The authors are aware of only one small study in the Netherlands that previously reported an association between EHR system and inferior performance on one prescribing safety indicator, in only 90 GP practices, with no follow-up to establish the design flaw.16 A 2017 systematic review1 identified 34 relevant studies exploring the role of computerised systems in suboptimal prescribing. However, none used quantitative methods to directly compare practices in different systems: most reported the results of questionnaires interrogating clinicians about their experiences of EHRs; qualitative research observing and/or interviewing clinicians; or quantitative analyses of large databases of clinicians’ spontaneous reports of errors and safety issues. Some studies set out to evaluate the impact of a single specific new change to a computerised prescribing system, typically as a behavioural intervention to increase compliance with a desired choice.17–19 For example, two small studies assessed modifications to the default settings in an EHR that improved generic prescribing rates by 5.4% and 23.1%, respectively, albeit against a backdrop of low generics use in the US.4,5
Implications for research and practice
Three key policy issues have arisen from this study: the importance of evaluating EHR systems; of implementing open standards in health care; and of open data analysis in health care.
Clinicians use EHR systems to store information, retrieve relevant information rapidly when assessing a patient, and to implement specific clinical actions such as ordering a test or prescribing a treatment. Healthcare activity is increasingly computerised, and EHR software is likely to exert a very substantial influence on the way that modern medicine is practised, in the same way that the rapid explosion in the use of social media has changed the ways that people interact socially.20 The authors were therefore surprised to find so little engagement by the clinical academic community in evaluating the impact of EHR design choices. Specifically, the authors are not aware of any previous attempts to use variation in observed prescribing behaviour between different EHR systems to identify, explain, and address the causes of suboptimal prescribing, or indeed any other aspect of clinical practice. They are now pursuing a research programme to evaluate differences in prescribing associated with different EHR systems in English primary care. More broadly, the relative absence of more ambitious work to address wider questions of EHR design is concerning. The EHR is a key technology used in primary care by clinicians, and will become increasingly important in secondary care as the NHS implements EHRs on a widespread basis. Questions of how best to represent, retrieve, and present knowledge about patients to clinicians — and the impact this can have on patient care — should be a key priority for funders and researchers in ‘digital health’.
One of the contributing factors to prescribing ghost-branded generics is likely to be that, when a prescriber is choosing from a pick list, a ghost-branded generic looks like a true generic. This is an example of a ‘look alike sound alike’ error, which has been recognised as a common source of medication error.21,22 The Medicines and Healthcare products Regulatory Agency (see Box 2 for details) has issued a safety alert because of adverse events, including deaths as a result of confusion between similarly named medicines.23 Although the chance of a serious adverse event and fatal outcome are low with ghost-branded generics, there are some medicines, such as category 1 anti-epileptic drugs, where it is important to specify the manufacturer on the prescription for clinical reasons,24 that is, prescribe a ghost-branded generic, so it is necessary to allow this functionality in some limited situations.
In recent years the NHS has pursued a strategy of setting standards to support the uptake and safe use of technology in the NHS, culminating in the launch of NHSX (see Box 2 for details).25 The NHS Dictionary of Medicines and Devices (DM+D) is the mandated standard drug dictionary for all suppliers of systems related to medicines across the NHS.26 Although initially the design choice made by SystmOne that caused ghost-branded generics was thought to be largely unforeseen, in fact implementation guidance27 supporting the DM+D standard makes reference to the potential for confusion among prescribers driven by this issue. If the NHS sets standards, it must also invest in assessing whether these standards have been adhered to, to ensure that system providers make modifications quickly if shortcomings are identified, and to monitor for unintended consequences of any standards set.
The UK government has recognised the importance of sharing NHS data where possible,28 and the publication since 2010 of highly granular NHS primary care prescribing data has facilitated a rich ecosystem of tools, such as the live dashboard created by the authors (OpenPrescribing.net), alongside extensive original research from multiple teams.7,29–35 There is, however, substantial room for improvement. The authors have previously written about breaches by the NHS of best practice around data management and publication.36 In the case of ghost-branded generics, it is a concern that the NHS Business Services Authority manages the DM+D standard and has responsibility for its accuracy, but has not published medicines data compliant with its own standard. Ghost-branded generics could have been identified sooner if it had done so.27 The publication of prescribing data compliant with the mandated NHS standard of DM+D is therefore warranted.
In summary, a design choice in a commonly used EHR has led to an excess cost to the NHS of £9.5 million in 2018 in ghost-branded generics. A live dashboard on the OpenPrescribing.net website has been created to monitor this phenomenon in every practice and CCG in England. The authors recommend further research into EHR design choices, and publication of prescribing data compliant with the mandated NHS standard of DM+D.
Acknowledgments
The authors are grateful to Peter Inglesby and Dave Evans for their contribution to maintaining databases and the OpenPrescribing website; to The Phoenix Partnership, SystmOne, NHS Business Services Authority, and wider NHS colleagues for discussions that have informed this work on ghost-branded generics; and to all the colleagues who have addressed the issue to save unnecessary NHS spending.
Funding
No specific funding was sought for this work. Ben Goldacre’s work on clinical informatics is supported by the National Institute for Health Research (NIHR) Biomedical Research Centre, Oxford; NHS England; the Health Foundation (award reference number: 7599); the NIHR School of Primary Care Research (award reference number: 327); the NIHR Applied Research Collaboration Oxford and Thames Valley; and Health Data Research UK; and policy work is supported by the Mohn Westlake Foundation. Funders had no role in the study design, collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.
Ethical approval
This study uses exclusively open, publicly available data, therefore no ethical approval was required.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
Ben Goldacre has received research funding from the Laura and John Arnold Foundation, the Wellcome Trust, the Oxford Biomedical Research Centre, the NIHR Applied Research Collaboration Oxford and Thames Valley, the NHS NIHR School of Primary Care Research, the Health Foundation, Health Data Research UK, the Mohn Westlake Foundation, and the World Health Organization; he also receives personal income from speaking and writing for lay audiences on the misuse of science. Richard Croker, Alex J Walker, Helen J Curtis, and Seb Bacon are employed on Ben Goldacre’s grants for OpenPrescribing. Brian MacKenna is seconded to the DataLab from NHS England. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR, NHS England, or the Department of Health and Social Care.
Discuss this article
Contribute and read comments about this article: bjgp.org/letters
REFERENCES
- 1.Brown CL, Mulcaster HL, Triffitt KL, et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc. 2017;24(2):432–440. doi: 10.1093/jamia/ocw119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.US Food and Drug Administration Computerized prescriber order entry medication safety (CPOEMS): uncovering and learning from issues and errors. 2015 https://www.fda.gov/media/95234/download (accessed 4 Aug 2020).
- 3.Rizk S, Oguntebi G, Graber ML, Johnston D. Report on the safe use of pick lists in ambulatory care settings: issues and recommended solutions for improved usability in patient selection and medication ordering. 2016 https://www.healthit.gov/sites/default/files/report-on-the-safe-use-of-pick-lists-in-ambulatory-care-settings.pdf (accessed 4 Aug 2020).
- 4.Patel MS, Day S, Small DS, et al. Using default options within the electronic health record to increase the prescribing of generic-equivalent medications: a quasi-experimental study. Ann Intern Med. 2014;161(10 suppl):S44–S52. doi: 10.7326/M13-3001. [DOI] [PubMed] [Google Scholar]
- 5.Patel MS, Day SC, Halpern SD, et al. Generic medication prescription rates after health system-wide redesign of default options within the electronic health record. JAMA Intern Med. 2016;176(6):847–848. doi: 10.1001/jamainternmed.2016.1691. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Appleby J. How much has generic prescribing and dispensing saved the NHS? 2015 https://www.kingsfund.org.uk/blog/2015/07/how-much-has-generic-prescribing-and-dispensing-saved-nhs (accessed 4 Aug 2020).
- 7.Kontopantelis E, Stevens RJ, Helms PJ, et al. Spatial distribution of clinical computer systems in primary care in England in 2016 and implications for primary care electronic medical record databases: a cross-sectional population study. BMJ Open. 2018;8(2):e020738. doi: 10.1136/bmjopen-2017-020738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Walker AJ, Curtis HJ, Croker R, et al. Measuring the impact of an open web-based prescribing data analysis service on clinical practice: cohort study on NHS England data. J Med Internet Res. 2019;21(1):e10929. doi: 10.2196/10929. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Croker R, Walker AJ, Bacon S, et al. New mechanism to identify cost savings in English NHS prescribing: minimising ‘price per unit’, a cross-sectional study. BMJ Open. 2018;8(2):e019643. doi: 10.1136/bmjopen-2017-019643. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.EBM DataLab Top saving opportunities for NHS Wakefield CCG in November 2018. https://openprescribing.net/ccg/03R/0212000B0AAABAB/price_per_unit/?date=2018-11-01 (accessed 4 Aug 2020).
- 11.MacKenna B. Ghost branded generics: why does the cost of generic atorvastatin vary? 2018 https://ebmdatalab.net/ghost-branded-generics-why-does-the-cost-of-generic-atorvastatin-vary%ef%bb%bf (accessed 4 Aug 2020).
- 12.NHS Business Services Authority Information services portal (ISP) https://www.nhsbsa.nhs.uk/information-services-portal-isp (accessed 4 Aug 2020).
- 13.NHS Digital Patients registered at a GP practice. 2019 https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/general-practice-data-hub/patients-registered-at-a-gp-practice (accessed 4 Aug 2020).
- 14.Pharmaceutical Services Negotiating Committee Price concessions. 2020 https://psnc.org.uk/dispensing-supply/supply-chain/generic-shortages (accessed 4 Aug 2020).
- 15.EBM DataLab Ghost branded generics GitHub code repository. https://github.com/ebmdatalab/ghost_branded_generics_paper (accessed 4 Aug 2020).
- 16.Opondo D, Visscher S, Eslami S, et al. Quality of co-prescribing NSAID and gastroprotective medications for elders in the Netherlands and its association with the electronic medical record. PLoS One. 2015;10(6):e0129515. doi: 10.1371/journal.pone.0129515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Halpern SD, Ubel PA, Asch DA. Harnessing the power of default options to improve health care. N Engl J Med. 2007;357(13):1340–1344. doi: 10.1056/NEJMsb071595. [DOI] [PubMed] [Google Scholar]
- 18.Patel MS, Volpp KG, Asch DA. Nudge units to improve the delivery of health care. N Engl J Med. 2018;378(3):214–216. doi: 10.1056/NEJMp1712984. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Vaughn VM, Linder JA. Thoughtless design of the electronic health record drives overuse, but purposeful design can nudge improved patient care. BMJ Qual Saf. 2018;27(8):583–586. doi: 10.1136/bmjqs-2017-007578. [DOI] [PubMed] [Google Scholar]
- 20.Shirky C. Here comes everybody: the power of organizing without organizations. 2008 https://max.book118.com/free_down/05051830642011.pdf (accessed 8 Apr 2020).
- 21.World Health Organization Medication without harm. 2017 https://apps.who.int/iris/handle/10665/255263 (accessed 4 Aug 2020).
- 22.Department of Health and Social Care The report of the Short Life Working Group on reducing medication-related harm. 2018 https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/683430/short-life-working-group-report-on-medication-errors.pdf (accessed 4 Aug 2020).
- 23.Medicines and Healthcare products Regulatory Agency Drug-name confusion: reminder to be vigilant for potential errors. 2018 https://www.gov.uk/drug-safety-update/drug-name-confusion-reminder-to-be-vigilant-for-potential-errors (accessed 4 Aug 2020).
- 24.Medicines and Healthcare products Regulatory Agency Antiepileptic drugs: updated advice on switching between different manufacturers’ products. 2017 https://www.gov.uk/drug-safety-update/antiepileptic-drugs-updated-advice-on-switching-between-different-manufacturers-products (accessed 4 Aug 2020).
- 25.NHS England Interoperability. https://www.england.nhs.uk/digitaltechnology/connecteddigitalsystems/interoperability (accessed 4 Aug 2020).
- 26.NHS Digital SCCI0052: Dictionary of medicines and devices (dm+d) 2018 https://digital.nhs.uk/data-and-information/information-standards/information-standards-and-data-collections-including-extractions/publications-and-notifications/standards-and-collections/scci0052-dictionary-of-medicines-and-devices-dm-d (accessed 4 Aug 2020).
- 27.Health and Social Care Information Centre dm+d implementation guide (primary care) 2015 https://www.nhsbsa.nhs.uk/sites/default/files/2017-02/dmd_Implemention_Guide_%28Primary_Care%29_v1.0.pdf (accessed 4 Aug 2020).
- 28.Office for Life Sciences Life sciences: industrial strategy. 2017 https://www.gov.uk/government/publications/life-sciences-industrial-strategy (accessed 4 Aug 2020).
- 29.Curtis HJ, Croker R, Walker AJ, et al. Opioid prescribing trends and geographical variation in England, 1998–2018: a retrospective database study. Lancet Psychiatry. 2019;6(2):140–150. doi: 10.1016/S2215-0366(18)30471-1. [DOI] [PubMed] [Google Scholar]
- 30.Walker AJ, Curtis HJ, Bacon S, et al. Trends and variation in prescribing of low-priority treatments identified by NHS England: a cross-sectional study and interactive data tool in English primary care. J R Soc Med. 2018;111(6):203–213. doi: 10.1177/0141076818769408. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Curtis HJ, Dennis JM, Shields BM, et al. Time trends and geographical variation in prescribing of drugs for diabetes in England from 1998 to 2017. Diabetes Obes Metab. 2018;20(9):2159–2168. doi: 10.1111/dom.13346. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Curtis HJ, Walker AJ, Goldacre B. Impact of NICE guidance on tamoxifen prescribing in England 2011–2017: an interrupted time series analysis. Br J Cancer. 2018;118(9):1268–1275. doi: 10.1038/s41416-018-0065-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Gonem S, Cumella A, Richardson M. Asthma admission rates and patterns of salbutamol and inhaled corticosteroid prescribing in England from 2013 to 2017. Thorax. 2019 doi: 10.1136/thoraxjnl-2018-212723. [DOI] [PubMed] [Google Scholar]
- 34.Saeed HS, Wright RB, Ghosh SK. Trends in the prescribing of topical nasal agents using an NHS England data base. Clin Otolaryngol. 2018;43(5):1296–1302. doi: 10.1111/coa.13143. [DOI] [PubMed] [Google Scholar]
- 35.Edelstein M, Agbebiyi A, Ashiru-Oredope D, Hopkins S. Trends and patterns in antibiotic prescribing among out-of-hours primary care providers in England, 2010–14. J Antimicrob Chemother. 2017;72(12):3490–3495. doi: 10.1093/jac/dkx323. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Bacon S, Goldacre B. Overcoming barriers to working with National Health Service England’s open data. J Med Internet Res. 2020;22(1):e15603. doi: 10.2196/15603. [DOI] [PMC free article] [PubMed] [Google Scholar]