Skip to main content
PMC home page
Quality in Health Care : QHC logoLink to Quality in Health Care : QHC
. 2000 Dec;9(4):222–231. doi: 10.1136/qhc.9.4.222

New Zealand and United Kingdom experiences with the RAND modified Delphi approach to producing angina and heart failure criteria for quality assessment in general practice

S Buetow 1, G Coster 1
PMCID: PMC1743535  PMID: 11101707

Abstract

Objectives—(1) To describe the development of minimum review criteria for the general practice management in New Zealand (NZ) of two chronic diseases: stable angina and systolic heart failure, and (2) to compare the NZ angina criteria with a set produced in Manchester to assess the extent to which use of the same approach to criteria development yields similar criteria.

Methods—A modified Delphi approach, based on the RAND consensus panel method, was used to produce minimum criteria for reviewing the recorded management of heart failure and angina in NZ general practice. The criteria for angina were compared with those produced in the UK, including assessment of the extent to which each set describes actions that the other panel agrees are necessary to record.

Results—For each condition we report minimum criteria describing actions rated as (a) necessary to record and (b) inappropriate to take but, if taken, necessary to record. Although strong scientific evidence underpins approximately one quarter and one third, respectively, of the final sets of NZ and UK angina criteria for actions necessary to record, the NZ criteria agree strongly with the UK criteria (33 of 39 criteria, 85%) but there is less UK agreement with the NZ angina criteria (28 of 40 criteria, 70%).

Conclusion—Despite the lack of scientific evidence for up to three quarters of angina care in general practice, the RAND based approach to criteria development was used in NZ to reproduce most of the UK angina criteria for actions rated as necessary to record in general practice. It is important to make explicit whether ratings of necessity and appropriateness apply to the recording of actions or to the actions themselves.

(Quality in Health Care 2000;9:222–231)

Key Words: review criteria; quality of care; general practice; angina; systolic heart failure

Full Text

The Full Text of this article is available as a PDF (162.1 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Baker R., Fraser R. C. Development of review criteria: linking guidelines and assessment of quality. BMJ. 1995 Aug 5;311(7001):370–373. doi: 10.1136/bmj.311.7001.370. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Brook R. H., Kosecoff J. B., Park R. E., Chassin M. R., Winslow C. M., Hampton J. R. Diagnosis and treatment of coronary disease: comparison of doctors' attitudes in the USA and the UK. Lancet. 1988 Apr 2;1(8588):750–753. doi: 10.1016/s0140-6736(88)91550-4. [DOI] [PubMed] [Google Scholar]
  3. Buetow S. A., Sibbald B., Cantrill J. A., Halliwell S. Appropriateness in health care: application to prescribing. Soc Sci Med. 1997 Jul;45(2):261–271. doi: 10.1016/s0277-9536(96)00342-5. [DOI] [PubMed] [Google Scholar]
  4. Campbell S. M., Roland M. O., Shekelle P. G., Cantrill J. A., Buetow S. A., Cragg D. K. Development of review criteria for assessing the quality of management of stable angina, adult asthma, and non-insulin dependent diabetes mellitus in general practice. Qual Health Care. 1999 Mar;8(1):6–15. doi: 10.1136/qshc.8.1.6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Cleland J. G., McGowan J., Clark A., Freemantle N. The evidence for beta blockers in heart failure. BMJ. 1999 Mar 27;318(7187):824–825. doi: 10.1136/bmj.318.7187.824. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Donabedian A. The quality of care. How can it be assessed? JAMA. 1988 Sep 23;260(12):1743–1748. doi: 10.1001/jama.260.12.1743. [DOI] [PubMed] [Google Scholar]
  7. Fraser R. C., Khunti K., Baker R., Lakhani M. Effective audit in general practice: a method for systematically developing audit protocols containing evidence-based review criteria. Br J Gen Pract. 1997 Nov;47(424):743–746. [PMC free article] [PubMed] [Google Scholar]
  8. Gapinski J. P., VanRuiswyk J. V., Heudebert G. R., Schectman G. S. Preventing restenosis with fish oils following coronary angioplasty. A meta-analysis. Arch Intern Med. 1993 Jul 12;153(13):1595–1601. [PubMed] [Google Scholar]
  9. Grimshaw J., Russell I. Achieving health gain through clinical guidelines. I: Developing scientifically valid guidelines. Qual Health Care. 1993 Dec;2(4):243–248. doi: 10.1136/qshc.2.4.243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hadorn D. C., Baker D. W., Kamberg C. J., Brooks R. H. Phase II of the AHCPR-sponsored heart failure guideline: translating practice recommendations into review criteria. Jt Comm J Qual Improv. 1996 Apr;22(4):265–276. doi: 10.1016/s1070-3241(16)30230-9. [DOI] [PubMed] [Google Scholar]
  11. Hasdai D., Garratt K. N., Grill D. E., Lerman A., Holmes D. R., Jr Effect of smoking status on the long-term outcome after successful percutaneous coronary revascularization. N Engl J Med. 1997 Mar 13;336(11):755–761. doi: 10.1056/NEJM199703133361103. [DOI] [PubMed] [Google Scholar]
  12. Hicks N. R. Some observations on attempts to measure appropriateness of care. BMJ. 1994 Sep 17;309(6956):730–733. doi: 10.1136/bmj.309.6956.730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Kahan J. P., Bernstein S. J., Leape L. L., Hilborne L. H., Park R. E., Parker L., Kamberg C. J., Brook R. H. Measuring the necessity of medical procedures. Med Care. 1994 Apr;32(4):357–365. doi: 10.1097/00005650-199404000-00004. [DOI] [PubMed] [Google Scholar]
  14. Kahn K. L., Rogers W. H., Rubenstein L. V., Sherwood M. J., Reinisch E. J., Keeler E. B., Draper D., Kosecoff J., Brook R. H. Measuring quality of care with explicit process criteria before and after implementation of the DRG-based prospective payment system. JAMA. 1990 Oct 17;264(15):1969–1973. [PubMed] [Google Scholar]
  15. Kleinman L. C., Kosecoff J., Dubois R. W., Brook R. H. The medical appropriateness of tympanostomy tubes proposed for children younger than 16 years in the United States. JAMA. 1994 Apr 27;271(16):1250–1255. [PubMed] [Google Scholar]
  16. Kosecoff J., Chassin M. R., Fink A., Flynn M. F., McCloskey L., Genovese B. J., Oken C., Solomon D. H., Brook R. H. Obtaining clinical data on the appropriateness of medical care in community practice. JAMA. 1987 Nov 13;258(18):2538–2542. [PubMed] [Google Scholar]
  17. Kravitz R. L., Laouri M., Kahan J. P., Guzy P., Sherman T., Hilborne L., Brook R. H. Validity of criteria used for detecting underuse of coronary revascularization. JAMA. 1995 Aug 23;274(8):632–638. [PubMed] [Google Scholar]
  18. Krumholz H. M., Wang Y., Parent E. M., Mockalis J., Petrillo M., Radford M. J. Quality of care for elderly patients hospitalized with heart failure. Arch Intern Med. 1997 Oct 27;157(19):2242–2247. [PubMed] [Google Scholar]
  19. Naylor C. D. What is appropriate care? N Engl J Med. 1998 Jun 25;338(26):1918–1920. doi: 10.1056/NEJM199806253382612. [DOI] [PubMed] [Google Scholar]
  20. Rose G., Colwell L. Randomised controlled trial of anti-smoking advice: final (20 year) results. J Epidemiol Community Health. 1992 Feb;46(1):75–77. doi: 10.1136/jech.46.1.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Selby J. V., Fireman B. H., Lundstrom R. J., Swain B. E., Truman A. F., Wong C. C., Froelicher E. S., Barron H. V., Hlatky M. A. Variation among hospitals in coronary-angiography practices and outcomes after myocardial infarction in a large health maintenance organization. N Engl J Med. 1996 Dec 19;335(25):1888–1896. doi: 10.1056/NEJM199612193352506. [DOI] [PubMed] [Google Scholar]
  22. Shekelle P. G., Chassin M. R., Park R. E. Assessing the predictive validity of the RAND/UCLA appropriateness method criteria for performing carotid endarterectomy. Int J Technol Assess Health Care. 1998 Fall;14(4):707–727. doi: 10.1017/s0266462300012022. [DOI] [PubMed] [Google Scholar]
  23. Shekelle P. G., Kahan J. P., Bernstein S. J., Leape L. L., Kamberg C. J., Park R. E. The reproducibility of a method to identify the overuse and underuse of medical procedures. N Engl J Med. 1998 Jun 25;338(26):1888–1895. doi: 10.1056/NEJM199806253382607. [DOI] [PubMed] [Google Scholar]
  24. Stephens N. G., Parsons A., Schofield P. M., Kelly F., Cheeseman K., Mitchinson M. J. Randomised controlled trial of vitamin E in patients with coronary disease: Cambridge Heart Antioxidant Study (CHAOS) Lancet. 1996 Mar 23;347(9004):781–786. doi: 10.1016/s0140-6736(96)90866-1. [DOI] [PubMed] [Google Scholar]

Articles from Quality in Health Care : QHC are provided here courtesy of BMJ Publishing Group

RESOURCES