Development of prescribing-safety indicators for GPs using the RAND Appropriateness Method

Abstract

Background

In the UK, a process of revalidation is being introduced to allow doctors to demonstrate that they meet current professional standards, are up-to-date, and fit to practise. Given the serious risks to patients from hazardous use of medicines it will be appropriate, as part of the revalidation process, to assess the safety of prescribing by GPs.

Aim

To identify a set of potential prescribing-safety indicators forthe purposes of revalidation of individual GPs in the UK.

Design and setting

The RAND Appropriateness Method was used to identify, develop, and obtain agreement on the indicators in UK general practice.

Method

Twelve GPs from across the UK with a wide variety of characteristics assessed indicators for appropriateness of use in revalidation.

Results

Forty-seven safety indicators were considered appropriate for assessing the prescribing safety of individual GPs forthe purposes of revalidation (appropriateness was defined as an overall panel median score of ≥7 (on a 1-9 scale), with no more than three panel members rating the indicator outside the 3-point distribution around the median]. After removing indicators that were variations on the same theme, a final set of 34 indicators was obtained; these cover hazardous prescribing across a range of therapeutic areas, hazardous drug-drug combinations, prescribing with a history of allergy, and inadequate laboratory-test monitoring.

Conclusion

This study identified a set of 34 indicators that were considered, by a panel of 12 GPs, to be appropriate for use in assessing the safety of GP prescribing forthe purposes of revalidation. Violation of any of the 34 indicators indicates a potential patient-safety problem.

INTRODUCTION

Medication prescription is one of the most powerful tools for GPs in the prevention and treatment of disease and the alleviation of symptoms. Nearly800 million prescriptions are dispensed annually in the community in England.¹ However, medication-related adverse events are an important source of patient morbidity, many cases of which could be prevented by the highest-quality prescribing and medicines management.^2-5

Revalidation is a single, integrated process that allows doctors working in the UK to demonstrate that they meet current professional standards, are up-to-date, and fit to practise, by providing supporting information in a revalidation portfolio.⁶ Pilots of revalidation are currently ongoing. For GPs, the Royal College of General Practitioners (RCGP) is proposing standards and methods that must be approved by the General Medical Council — these include prescribing-safety indicators. Given the importance of prescribing by GPs, it is appropriate that potential indicators of the quality and safety of prescribing are considered in any scheme for revalidation.

There have been many attempts over recent years to develop prescribing indicators in the UK,⁷ such as those based on the interrogation of prescriptions issued by GPs (for example, using prescribing analysis and cost tabulation data⁸). Although potentially useful for analysing prescribing patterns, these data are rarely linked to diagnoses and patient characteristics, and so have limitations when assessing quality and safety. Other indicators have required very detailed analysis and assessment of clinical records (for example, the medication appropriateness index⁹), which would not be feasible for the large-scale assessment of all GPs in the UK.

The evidence base for developing quality indicators for safe prescribing is limited; however, systematically combining available evidence with expert professional opinion using a consensus methodology can create quality indicators in areas where it would not otherwise be possible.¹⁰ Moreover, with developments in methods for interrogating electronic medical records, there is now the opportunity in the UK to develop and use sophisticated indicators that can give an assessment of the quality and safety of prescribing by individual GPs.

This study aimed to identify a set of potential prescribing-safety indicators for the purposes of revalidation of individual GPs in the UK, contextualised by the wider implementation of revalidation.

METHOD

Identification of indicators

Potential indicators that described a pattern of prescribing that could be hazardous and may put patients at risk of harm were identified. Potential indicators that were not suitable for assessing the prescribing of individual doctors and where the prospects were low for extracting relevant data from GP clinical computer systems were excluded. The inclusion and exclusion criteria are shown in Box 1.

Box 1. Inclusion and exclusion criteria

Inclusion criterion

1. The indicator describes a pattern of prescribing that is potentially hazardous and may put patients at risk of harm.

Exclusion criteria

1. The indicator describes a pattern of prescribing that could not easily be attributed to a single doctor

2. The indicator describes optimal prescribing for long-term conditions where prescribing decisions are likely to be made overtime (potentially by more than one prescriber), rather than at a single consultation.

3. The indicator describes a pattern of prescribing that is so unusual in UK general practice that the yield is likely to be too low to justify inclusion in the indicators set.

4. Extraction of data required for the indicator (from general practice electronic health records) is unlikely to be feasible.

How this fits in

GPs have a critically important role in patient safety through the judicious prescription, and careful monitoring, of patients' medicines. Nevertheless, many patients are put at risk, and some are harmed, as a result of hazardous prescribing in general practice. Therefore, assessing the safety of prescribing by GPs is an important facet of any system for assessing fitness to practise. One way of doing this is by using prescribing-safety indicators, which have been developed in other areas of medicine. This article describes the development of a set of prescribing-safety indicators for use in general practice, with a specific focus on their use in revalidation in the UK.

Multiple sources of information were used to identify potential indicators;^11-24 these are outlined in Table 1. These sources were identified by one of the researchers with the help of a reference group between August 2008 and March 2009. The indicators developed in this project were derived mostly from existing sources because of the considerable amount of work that has been done on prescribing-safety indicators in other countries and other settings. ^11-24 Over 400 prescribing indicators were reviewed against the inclusion and exclusion criteria and against attributes of good indicators, including importance, validity, and feasibility of data collection using electronic health records.^10,25

Table 1.

Sources of the reviewed prescribing indicators

Source	Indicators reviewed (n)	Indicators from each source included in round one of consensus processa (n)	Source details	Relevance to prescribing-safety indicators for revalidation
Assessing Care of Vulnerable Elders (ACOVE)11,12	217	2	This RAND project11 developed a set of evidence-based, quality-of-care indicators relevant to vulnerable older people using systematic literature reviews, expert opinion, and guidance from expert groups and stakeholders. The indicators have been considered for use in the UK12	Many of these indicators relate to process measures and the care of patients with long-term conditions, ratherthan to prescribing decisions that could easily be attributed to one GP
Beers criteria13	89	20	A set of criteria from the US for assessing potentially inappropriate medication use in people aged ≥65 years. The original list of criteria were published in the 1990s and updated in 200313	These provided the greatest number of potentialindicators, but some of the Beers criteria relate either to drugs not commonly used in the UK or issues that were felt not of high clinical importance
British National Formulary(BNF)14	N/A	4	Four of the indicators came only from the BNF, which was also used to check the validity of many of the other potential indicators	N/A
Draft design specification for NHS IT systems aimed at minimising risk of harm from medications15	N/A	2	This report included a comprehensive literature review on risks of harm from medications in primary care and provides examples of hazardous prescribing	N/A
Medication Appropriateness Index9	N/A	N/A	A method for assessing the appropriateness of medication based on a range of factors such as indication, evidence for effectiveness, and directions and absence of important contraindications. It lends itself best to detailed analysis of prescribing based on clinical judgement, rather than interrogation of electronic medical records	Specific indicators not included
National Patient Safety Agency (NPSA) documents16-18	N/A	N/A	The NPSA16 has produced a number of documents that are relevant to the safety of prescribing in primary care.17 For example, the fourth report from the Patient Safety Observatory18 highlighted medication incidents in the community and at the interface between community and hospital care, as well as suggesting ways in which risks of harm could be reduced. The NPSA has highlighted a number of specific safety issues relevant to primary care, including anticoagulant prescribing, dosing errors with opioid medicines, and the prescribing of methotrexate17	Information from NPSA documents used to help inform the prescribing-safety indicators project
The National Service Framework for Older People(NSF)19 and Medicines for Older People: Implementing Medicines-Related Aspects of the NSF for Older People20	N/A	N/A	These documents raise important issues regarding medicines used in older people	Specific indicators for assessing the quality and safety of prescribing are not explicitly suggested, although many of the issues raised are covered in the approaches taken by Assessing Care of Vulnerable Elders and Beers criteria
PINCER trial indicators21	11	6	A cluster randomised trial took place in the UK between 2005 and 2009 to assess a pharmacistled intervention versus simple feedback in correcting clinically important problems in medicines management in general practices in England. The outcome measures for the trial represent important examples of hazardous prescribing, inadequate monitoring, and potentially hazardous dosage instructions21	Most of these indicators were considered relevant
Preventable drug-related morbidity22 indicators	29	4	Based on identifying preventable morbidity associated with drug use. The University of Manchester has adapted US indicators for use in the UK and, along with the University of Nottingham, has successfully used these indicators of electronic health records in general practices in England22	These indicators focus on a preventable morbidity associated with drug use, but from this information it would be difficult using electronic searches to identify which doctor(s) were responsible for the prescribing (or monitoring failures) that led to the preventable morbidity
Quality and Outcomes Framework (QOF)23	8	0	The QOF contains a number of prescribing- related indicators. Most of these relate to the need to prescribe a particular drug for a particular clinical condition. There are also more general indicators of prescribing, including the need to undertake medication reviews	Indicators relate to the need to prescribe a particular drug fora particular clinical condition, which is considered outside the scope of the current exercise. The more general indicators of prescribing, such as the need to undertake medication reviews, cannot easily be applied to the prescribing of an individual GP in a group practice or assessed through interrogation of the electronic health record
Screening Tool of Older Persons' potentially inappropriate Prescriptions (STOPP) and Screening Tool to Alert doctors to Right Treatment (START)24	65	27	These sets of indicators have been developed to assess the appropriateness of prescribing for older people. The tools have been developed and validated by a team from Cork, Republic of Ireland	Many of these indicators were considered highly relevant to general practice; variations on 27 of them were put to the consensus panel. STOPP indicators not included were not considered, by the research team, to be of high enough clinical importance forthe purposes of revalidation, or it would be very difficult to extract relevant data from GP computer systems

^aAs there was some overlap in indicators between the different sources, after removing duplicates, 50 indicators (and a total 68 indicator statements) were included in round one.

After removing duplicates, 50 indicators were considered suitable for inclusion in round one of the subsequent consensus technique. Of these, 44 came from a least one of the sources highlighted in Table 1; four came only from the British National Formulary (BNF)¹⁴ and two only from the draft design specification for NHS IT systems.¹⁵

Defining the indicators and evidence base

In most cases, indicators were defined according to the wording used in previously published studies and reports but, in some instances, this was altered either to make the indicators more relevant to UK general practice or to give more specific detail regarding the drugs and conditions covered. For 13 of the indicators, one or more variations in the text were produced; this resulted in a total of 68 indicatorstatements for the 50 indicators. As an example, although the prescription of β-blockers is contraindicated in asthma, there is evidence to suggest that in various types of heart disease, such as heart failure, the benefits may outweigh the risks; therefore, a variation on an asthma/β-blocker indicator would be to exclude patients with cardiac conditions where benefits of β-blockers may outweigh the risks.

Electronic searches of the literature were conducted and respected reference sources, such as the BNF,¹⁴ Martindale: The Complete Drug Reference,²⁶ and Stockley's Drug Interactions,²⁷ were drawn on. Indicators were also shown to the team responsible for developing the content of the BNF, who provided additional background evidence. The supporting evidence base was summarised in an evidence-based synopsis for each of the potential indicators.

The reasons why the different types of indicators did not yield many candidate indicators from the sources included are shown in Table 1. For example, many Assessing Care of Vulnerable Elders¹¹ indicators developed in the US, relate to process measures rather than prescribing decisions that can be attributed to one GP; likewise, some of the Beers criteria¹³ and indicators of the Screening Tool of Older Persons' potentially inappropriate Prescriptions (STOPP)²⁴ relate more to the appropriateness of prescribing than safety.

The indicators were presented in the following categories:

• A: cardiovascular and respiratory disease;

• B: central nervous system (including analgesics);

• C: anti-infective agents;

• D: women's health and urinary disorders;

• E: musculoskeletal;

• F: hazardous co-prescriptions, interactions, and allergy; and

• G: laboratory-test monitoring.

Consensus process

The RAND Appropriateness Method was used to select the most appropriate indicators. This method seeks to combine scientific evidence with the collective judgement of experts: a consensus opinion is derived from a group, with individual opinions aggregated into refined aggregated opinion.^25,28,29 A 12-member panel was appointed. Two GPs with known expertise in therapeutics and one with experience of working with the British Medical Association's General Practitioners' Committee were purposely selected. The remaining nine panelists were identified by writing to all RCGP faculties in the UK in March 2009 to ask for volunteers, which resulted in 58 replies. The following factors were taken into account when selecting the remaining panelists:

• professional background (practising GPs);

• employment status (a range different types of GP, including partners, salaried GPs, a retainer-scheme GP, and a locum);

• sex (an equal mix of males and females);

• geography (GPs from each of the three countries of the UK); and

• professional roles (some GPs with experience of working as appraisers/assessors of GPs).

It was estimated that each memberof the panel committed at least 3 days of work to the consensus-building exercise; they were partially reimbursed for their time.

Data analyses

This study adhered to the RAND Appropriateness Method by conducting a two-round consensus process. In round one, which was conducted by email in June 2009, panel members were asked to consider each indicator on its own merits using the summarised evidence for each as well as their own experience as practitioners.

In round two, panelists met for a 1-day face-to-face meeting in July 2009, where, under the chairmanship of two moderators, they discussed each indicator in turn as a group and then re-rated the indicators on individual rating sheets. These round-two rating sheets included feedback from the first round, presenting the frequency distribution of ratings of all panelists across the 9-point scale, the overall panel median rating, and, for each panelist, a reminder of how they had rated each indicator in round one. Although panel members were made aware of their own ratings from the first round, the scores from other panel members were anonymised.

During round two, panelists were also asked to consider whether they wished to propose an alternative wording for each indicator or propose any additional indicators. This resulted in wording changes for 21 indicators and two new ones. Each of the indicators — whether original, revised, or new — was scored by each of the panel members.

In both rounds, panelists were asked to rate each indicator on a 9-point integer scale. A rating of 1 meant that, in their opinion, it would be extremely inappropriate to use the indicator, whereas a rating of 9 meant its use would be extremely appropriate. The overall panel median ratings were as follows:

• 1-3: inappropriate;

• 4-6: equivocal, or unsure of appropriateness; and

• 7-9: appropriate.

Using the consensus technique, there was another aspect to the rating process, namely, the level of agreement within a panel for each scale for each indicator. Agreement signified that 80% of panelists' ratings were within the same 3-point region (that is, 1-3, 4-6, 7-9), as the observed median. To be included in the final recommended set of indicators, therefore, ‘appropriate’ was defined as an overall panel median score of >7, with no more than three panel members rating the indicator outside the 3-point distribution around the median; for example, if the median score was 7, no more than three panel members gave a score <6 or >8. Results are presented for the final (round-two) ratings only.

RESULTS

Ninety-two indicators were considered in round two of the exercise; the full results are shown in Appendix 1. Forty-seven prescribing-safety indicators were considered appropriate for assessing the prescribing safety of individual GPs for the purposes of revalidation (overall panel median rating of 7-9 with agreement); these are shown in bold in Appendix 1). These consisted of 32 originally worded indicators, along with 13 variations on these and two that were suggested by the expert panel during round two. Six indicators were rated inappropriate for assessing the prescribing safety of individual GPs for the purposes of revalidation (overall panel rating of 1-3 with agreement). The remaining indicators were rated as equivocal.

After removing indicators that were variations on the same theme, 34 that were rated as appropriate for use in assessing the prescribing safety of individual GPs for the purposes of revalidation were identified; these are shown in Box 2. Variations on indicators that were either most strongly rated by panelists or were most comprehensive were selected — for example, panelists rated indicators involving benzodiazepines more strongly where these indicators also contained Z-drugs.

Box 2. Indicators rated as valid for assessing the prescribing safety of individual GPs

A: Cardiovascular and respiratory disease

1. Beta-blockerto a patient with asthma (excluding patients who also have a cardiac condition, where the benefits of beta-blockers may outweigh the risks

2. Short-acting nifedipine (excluding patients with Raynaud's disease)

3. Digoxin at a dose >125 μg daily in a patient with renal impairment (for example, CKD3 or worse)

4. Digoxin at a dose of >125 μg daily for a patient with heart failure who is in sinus rhythm

5. Diltiazem orverapamil in a patient with heart failure

6. Aspirin at a dose >75 mg daily for ≥1 month in a patient aged >65 years

7. Long-acting beta-2 agonist inhaler to a patient with asthma who is not also prescribed an inhaled corticosteroid

B: Central nervous system (including analgesics)

8. Aspirin to a child aged ≤16 years

9. Metoclopramide or prochlorperazine in a patient with Parkinson's disease

10. Benzodiazepine or Z-drug for ≥21 days in a patient aged >65 years who is not receiving benzodiazepines orZ-drugs on a long-term basis

11. Initiation of benzodiazepine or Z-drugs basis for ≥21 days in a patient >65 years with depression

C: Anti-infective agents

12. Mefloquine to a patient with a history of convulsions

D: Women's health and urinary disorders

13. Combined hormonal contraceptive to a woman with a history of venous or arterial thromboembolism

14. Oral ortransdermal oestrogens to a woman with a history of breast cancer

15. Oral ortransdermal oestrogen without progesterone in a woman with an intact uterus

16. Combined hormonal contraceptive to a woman aged ≥35 years who is a current smoker

17. Combined hormonal contraceptive to a woman with body mass index ≥40

E: Musculoskeletal

18. NSAID, without co-prescription of an ulcer healing drug, to a patient with a history of peptic ulceration

19. NSAID in a patient with heart failure

20. NSAID in a patient with chronic renal failure (for example, CKD3 orworse)

21. Long-term (>28 days) NSAID (except for ibuprofen ≤1200 mg daily) in a patient aged >65 years

F: Hazardous co-prescriptions, interactions, and allergy

22. Warfarin in combination with an oral NSAID

23. Phosphodiesterase type-5 inhibitor (for example, sildenafil) to a patient who is also receiving a nitrate or nicorandil

24. Clarithromycin or erythromycin to a patient who is also receiving simvastatin, with no evidence that the patient has been advised to stop the simvastatin while taking the antibiotic

25. Potassium salt or potassium sparing diuretic (excluding aldosterone antagonists) to a patient who is also receiving an ACE inhibitor or angiotensin II receptor antagonist

26. Verapamil to a patient who is also receiving a beta-blocker drug

27. Penicillin-containing preparation to a patient with a history of allergy to penicillin

G: Laboratory test monitoring

28. Warfarin to a patient without a record of international normalised ratio having been measured within the previous 12 weeks (excluding patients who self monitor)

29. Amiodarone without a record of liver function being measured in the previous 9 months

30. Amiodarone without a record of thyroid function being measured within the previous 9 months

31. ACE inhibitor or angiotensin II receptor antagonist without a record of renalfunction and electrolytes being measured prior to starting therapy

32. Lithium without a record of a lithium level being measured within the previous 6 months

33. Methotrexate without a record of a full blood count within the previous 3 months

34. Methotrexate without a record of liver function having been measured within the previous 3 months

DISCUSSION

Summary

Thirty-four prescribing-safety indicators were identified as appropriate for use in assessing the prescribing safety of individual GPs for the purposes of revalidation. These cover hazardous prescribing across a range of therapeutic areas, hazardous drug-drug combinations, prescribing with a history of allergy, and inadequate laboratory-test monitoring.

Strengths and limitations

The strengths of the study include the wide range of sources used to identify prescribing-safety indicators as well as the involvement of both a reference group and staff editors from the BNF to validate aspects of the work and the evidence base. Each indicator was accompanied by an evidence-based summary. The panel consisted of GPs with a wide range of characteristics. Although over 400 potential prescribing indicators were reviewed, only those appropriate for use in the assessment of the prescribing safety of individual GPs for the purposes of revalidation were included. This study also adhered to a validated systematic consensus method for developing appropriateness scenarios.²⁹

The indicators were not defined so they could be immediately converted into reliable and unambiguous queries for running on GP computer systems, as with the PINCER trial.²¹ In addition, the indicators were not phrased in such a way as to take account of a doctor's volume of prescribing (whereby instances of hazardous prescribing might be considered as a proportion of overall numbers of prescriptions).

For each of the potential indicators, the supporting evidence base was summarised by undertaking electronic searches of the literature and drew on respected reference sources, such as the BNF,¹⁴ Martindale: The Complete Drug Reference,²⁶ and Stockley's Drug Interactions.²⁷ Although it is possible that more extensive literature reviews would have identified further evidence or other sources of potential prescribing-safety indicators, the involvement of a reference group and consultation with the editors of the BNFmitigate against this.

Comparison with existing literature

It was felt that the modified Beers criteria¹³ and STOPP indicators²⁴ would be of particular relevance to UK general practice, although fewer than half of the indicators from these sources were considered appropriate for use in revalidation. There are a number of reasons for this, such as the inclusion and exclusion criteria utilised, the perceived clinical importance of the indicators, and the focus on indicators for the purposes of revalidation. Nevertheless, the study shows the importance of giving careful consideration to the purpose forand setting in which they are planned to be used.

Implications for research and practice

This study has highlighted a number of issues that have implications for future research as well as the development and application of prescribing-safety indicators. Ongoing work will now determine whether each indicator can be translated into a computer query capable of reliably assessing the prescribing safety of individual GPs; the use of these queries will then be tested on GP computer systems. If implemented, it will be important to determine whether the use of prescribing-safety indicators improves prescribing performance and patient outcomes.

There are other implications that must be noted. The evidence base for most of the indicators is weak. This is, perhaps, unsurprising given that clinical trials focus on demonstrating the efficacy of interventions, rather than their safety; nevertheless, few of the indicators that were identified would fulfil the evidence required for inclusion in, for example, the Quality and Outcomes Framework.^23,30 Building the evidence base would mean conducting further research on the harms associated with potentially hazardous prescribing and inadequate medication monitoring, as well as the extent to which avoiding/correcting hazardous prescribing and inadequate monitoring improves patient safety.

In addition, although several of the sources used took account of the most common and important medication errors and/or preventable adverse drug events in general practice,^15,21,22 future pharmacoepidemiological studies might help to provide even better information for the development of prescribing-safety indicators.

It will be important to consider the ways in which the prescribing-safety indicators might be used. The primary purpose of this study was their use in revalidation, but the indicators could also be used for the purposes of audit, GP appraisal, and in intervention studies similar to the PINCER trial.²¹ Encouraging the use of the indicators for improving the safety of prescribing will be important.

This study identified a set of 34 indicators that were considered, by a panel of 12 GPs, to be appropriate for use in assessing the prescribing safety of GPs forthe purposes of revalidation. Violation of any of these indicators points towards a potential patient-safety problem.

Appendix 1.

Detailed results from second round of RAND Appropriateness Method exercise.

Identifier	Indicator	Overall panel median Consensus
A: Cardiovascular and respiratory disease
A1(a)a	Prescription of a beta-blockerto a patient with asthma	5	Equivocal
A1(b)a	Prescription of a non-cardioselective beta-blocker to a patient with asthma	6	Disagreement
A1(c)a,b	Prescription of a beta-blockerto a patient with asthma (excluding patients who also have a cardiac condition)	8	Agreement
A2	Prescription of a nitrate to a patient with aortic stenosis	3	Agreement
A3	Prescription of short-acting nifedipine in an older person (>65 years) (excluding patients with Raynaud's disease)	6	Disagreement
A3Xb,c	Prescription of short-acting nifedipine (excluding patients with Raynaud's disease)	7	Agreement
A4(a)a	In an older person (>65 years), prescription of digoxin at a dose of >125 μg daily	7	Equivocal
A4(b)a,b	In a patient with renal impairment, prescription of digoxin at a dose > 125 μg daily (for example, CKD 3 or worse)	7	Agreement
A5b	Prescription of digoxin at a dose of > 125 μg daily for a patient with heart failure who is in sinus rhythm	8	Agreement
A6b	Prescription of diltiazem orverapamil in a patient with heart failure	8	Agreement
A7b	In an older patient (>65 years), prescription of aspirin at a dose >75 mg daily	7	Agreement
A7Xb,c	In an older patient (>65 years), prescription of aspirin at a dose >75 mg daily for ≥1 month	8	Agreement
A8	In a patient with heart failure, the prescription of a high-sodium content drug	3	Equivocal
A9(a)a	In an older patient (>65 years) with an arrhythmia, the prescription of a tricyclic antidepressant	4	Equivocal
A9(b)a	In an older patient (>65 years) with a cardiac conductive abnormality, the prescription of a tricyclic antidepressant	4	Equivocal
A10	In a patient with a history of gout, the prescription of a thiazide diuretic	3	Agreement
A11b	The prescription of a long-acting beta-2 agonist inhaler to a patient with asthma who is not also using an inhaled corticosteroid	8	Agreement
B: Central nervous system (including analgesics)
B1(a)a,b	Prescription of aspirin to a child aged ≤16 years	9	Agreement
B1(b)a,b	Prescription of aspirin to a child aged ≤16 years, where the child does not have Kawasaki's disease	9	Agreement
B2	In a patient with epilepsy, the prescription of a drug that may lower the seizure threshold	3	Agreement
B3(a)a,b	In a patient with Parkinson's disease, the prescription of metoclopramide	8	Agreement
B3(b)a,b	In a patient with Parkinson's disease, the prescription of prochlorperazine	8	Agreement
B4(a)a	In an older person (>65 years) the prescription of a benzodiazepine on a long-term basis (that is, >1 month)	4.5	Equivocal
B4(a)Xa,c	In an older person (>65 years) the prescription of a benzodiazepine or Z-drug on a long-term basis (that is, >1 month)	4.5	Disagreement
B4(b)a,b	In an older person (>65 years) who is not receiving benzodiazepines on a long-term basis, the prescription of a benzodiazepine for >21 days	8	Agreement
B4(b)Xa,b,c	In an older person (>65 years) who is not receiving benzodiazepines orZ-drugs on a long-term basis, the prescription of a benzodiazepine or Z-drug for>21 days	8	Agreement
B5(a)a	In an older person (>65 years) with depression, the prescription of a long-term benzodiazepine (that is, >21 days)	5	Disagreement
B5(a)Xc	In an older person (>65 years) with depression, the prescription of a long-term benzodiazepine or Z-drug (that is, >21 days)	6	Disagreement
B5(b)a	In an older person (>65 years) with depression initiation of benzodiazepine on a long-term basis (that is, >21 days)	8	Equivocal
B5(b)Xa,b,c	In an older person (>65 years) with depression initiation of benzodiazepine orZ-drugs on a long-term basis (that is, >21 days)	8	Agreement
B6	In a patient with COPD, the prescription of a benzodiazepine	4	Agreement
B6Xc	In a patient with COPD, the prescription of a benzodiazepine or Z-drug	4	Agreement
B7(a)a	In an older patient (>65 years) with a history of falls, the prescription of either a benzodiazepine, an antipsychotic, or a tricyclic antidepressant	3	Agreement
B7(a)Xa,c	In an older patient (>65 years) with a history of falls, the prescription of a benzodiazepine, a Z-drug, an antipsychotic, or a tricyclic antidepressant	3	Agreement
B7(b)a	In an older patient (>65 years) with a history of falls, the prescription of a benzodiazepine	4	Equivocal
B7(b)Xa,c	In an older patient (>65 years) with a history of falls, the prescription of a benzodiazepine or Z-drug	4	Equivocal
B7(c)a	In an older patient (>65 years) with a history of falls, the prescription of an antipsychotic	3.5	Equivocal
B7(d)a	In an older patient (>65 years) with a history of falls, the prescription of a tricyclic antidepressant	3.5	Equivocal
B8	In an older patient (>65 years) with hyponatraemia/syndrome of inappropriate antidiuretic hormone secretion, the prescription of a selective serotonin reuptake inhibitor	4	Equivocal
B9	In an older person (>65 years) with cognitive impairment, the prescription of an anti-muscarinic drug	3.5	Equivocal
B10	In a patient with angle closure glaucoma, the prescription of an anti-muscarinic drug	4	Equivocal
C: Anti-infective agents
C1	Prescription of a tetracycline to a child aged >12 years	5	Equivocal
C2(a)a	Prescription of mefloquine to a patient with a history of psychiatric disorder (including depression)	6	Disagreement
C2(b)a,b	Prescription of mefloquine to a patient with a history of convulsions	8	Agreement
C2(b)Xa,c	Prescription of mefloquine to a patient with a history of epilepsy	3	Equivocal
C3	In an older patient (>65 years) long-term prescription of nitrofurantoin (that is, ≥28-day prescription)	6	Disagreement
D: Women's health and urinary disorders
D1b	Prescription of a combined hormonal contraceptive to a woman with a history of venous or arterial thromboembolism	9	Agreement
D2(a)a,b	Prescription of oral ortransdermal oestrogens to a woman with a history of breast cancer	9	Agreement
D2(b)a	Prescription of a topical (vaginal) oestrogen to a woman with a history of breast cancer	5	Equivocal
D3b	Prescription of oral ortransdermal oestrogen without progesterone in a woman with an intact uterus	9	Agreement
D4	Prescription of an anti-muscarinic drug to a patient with bladder outflow obstruction	3	Agreement
D5Xa,b,c	Prescription of a combined hormonal contraceptive to a woman aged a35 years who is a current smoker	8.5	Agreement
D6Xb,c	Prescription of a combined hormonal contraceptive to a woman with body mass index of ≥40	8	Agreement
E: Musculoskeletal
E1(a)a,b	Prescription of a non-selective NSAID to a patient with a history of peptic ulceration	8	Agreement
E1(a)Xa,b,c	Prescription of an NSAID to a patient with a history of peptic ulceration	9	Agreement
E1(b)a,b	Prescription of a non-selective NSAID to an older patient (>65 years) with a history of peptic ulceration	8	Agreement
E1(b)Xa,b,c	Prescription of an NSAID to an older patient (>65 years) with a history of peptic ulceration	9	Agreement
E1(c)a,b	Prescription of a non-selective NSAID, without co-prescription of an ulcer healing drug, to a patient with a history of peptic ulceration	8	Agreement
E1(c)Xa,b,c	Prescription of an NSAID, without co-prescription of an ulcer healing drug, to a patient with a history of peptic ulceration	9	Agreement
E2	Prescription of a selective COX-2 inhibitor NSAID to a patient with cardiovascular disease (for example, CHD, cerebrovascular disease, or peripheral arterial disease)	3	Equivocal
E3	In an older patient (>65 years), prescription of an NSAID without co-prescription of an ulcer healing drug (for example, misoprostol or a proton pump inhibitor)	6	Equivocal
E4	In an older patient (>65 years), prescription of a long-term (a28 days) full-dosage, longer half-life non-COX-2-selective NSAID	4	Equivocal
E5b	In an older patient (>65 years) with heart failure, the prescription of an NSAID	8	Agreement
E5Xb,c	In a patient with heart failure, the prescription of an NSAID	8	Agreement
E6b	In an older patient (>65 years) with chronic renal failure, the prescription of an NSAID (for example, CKD 3 or worse)	8	Agreement
E6Xb,c	In a patient with chronic renal failure, the prescription of an NSAID (for example, CKD 3 or worse)	8.5	Agreement
E7b	In an older patient (>65 years) prescription of a long-term (>28 days) NSAID (except for ibuprofen ≤1200 mg daily)	7	Agreement
F: Hazardous co-prescriptions, interactions, and allergy
F1	Prescription of warfarin and aspirin in combination, without co-prescription of an ulcer healing drug	6	Disagreement
F2b	Prescription of warfarin in combination with an oral NSAID	7.5	Agreement
F3b	Prescription of a phosphodiesterase type-5 inhibitor (for example, sildenaf il) to a patient who is also receiving a nitrate or nicorandil	8	Agreement
F4(a)a	Prescription of clarithromycin or erythromycin to a patient who is also receiving simvastatin	4.5	Disagreement
F4(b)a,b	Prescription of clarithromycin or erythromycin to a patient who is also receiving simvastatin, with no evidence that the patient has been advised to stop the simvastatin while taking the antibiotic	7	Agreement
F5(b)a,b	Prescription of a potassium salt or potassium-sparing diuretic (excluding aldosterone antagonists, such as spironolactone) to a patient who is also receiving an ACE inhibitor or angiotensin II receptor antagonist	8	Agreement
F6b	Prescription of verapamil to a patient who is also receiving a beta-blocker drug	8	Agreement
F7	Prescription of quinine to a patient who is also receiving a cardiac glycoside, for example, digoxin	5	Equivocal
F8	Prescription of ciprofloxacin or norfloxacin to a patient who is also receiving theophylline	5	Agreement
F9b	Prescription of a penicillin-containing preparation to a patient with a history of allergy to penicillin	8.5	Agreement
F10	Prescription of warfarin and aspirin in combination	6	Equivocal
G: Laboratory test monitoring
G1b	Prescription of warfarin to a patient without a record of INR having been measured within the previous 12 weeks	7	Agreement
G1Xb,c	Prescription of warfarin to a patient without a record of INR having been measured within the previous 12 weeks (excluding patients who self monitor)	8	Agreement
G2	Prescription of amiodarone without a record of liver function being measured in the previous 6 months	7	Equivocal
G2Xb,c	Prescription of amiodarone without a record of liver function being measured in the previous 9 months	8	Agreement
G3	Prescription of amiodarone without a record of thyroid function being measured within the previous 6 months	7	Equivocal
G3Xb,c	Prescription of amiodarone without a record of thyroid function being measured within the previous 9 months	8	Agreement
G4b	Prescription of an ACE inhibitor without a record of renal function and electrolytes being measured prior to starting therapy	7.5	Agreement
G4Xb,c	Prescription of an ACE inhibitor or angiotensin II receptor antagonist without a record of renal function and electrolytes being measured prior to starting therapy	8	Agreement
G5(a)a	Prescription of lithium without a record of a lithium level being measured within the previous 3 months	6.5	Equivocal
G5(b)a,b	Prescription of lithium without a record of a lithium level being measured within the previous 6 months	8	Agreement
G6b	Prescription of methotrexate without a record of a full blood count within the previous 3 months	8	Agreement
G6Xc	Prescription of methotrexate without a record of a full blood count within the previous 4 months	7	Equivocal
G7b	Prescription of methotrexate without a record of liver function having been measured within the previous 3 months	8	Agreement
G7Xc	Prescription of methotrexate without a record of liver function having been measured within the previous 4 months	6.5	Equivocal

^aIndicators ending in letters ‘(a)’, ‘(b)’, (c), or ‘(d)’ are variations in the wording of indicators used in the first round (and repeated in the second round) of the exercise

^bIndicators highlighted in bold are those rated appropriate for assessing the safety of prescribing of individual GPs for the purposes of revalidation.

^cIndicators ending in ‘X’ are variations on indicators (or new indicators) suggested by the panel and assessed in the second round only. ACE = angiotensin-converting enzyme. ARB = angiotensin receptor blocker. CHD = coronary heart disease. CKD = chronic kidney disease. COPD = chronic obstructive pulmonary disease. INR = international normalised ratio. NSAID = nonsteroidal anti-inflammatory drug.

Funding body

This project was part-funded by the Academy of Medical Royal Colleges.

Ethical approval

Research ethics committee approval was not needed for this study.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss