Skip to main content
NCBI home page
Quality & Safety in Health Care logoLink to Quality & Safety in Health Care
. 2005 Aug;14(4):240–245. doi: 10.1136/qshc.2004.013227

An experimental study of determinants of the extent of disagreement within clinical guideline development groups

A Hutchings 1, R Raine 1, C Sanderson 1, N Black 1
PMCID: PMC1744054  PMID: 16076786

Abstract

Objective: To assess the effect of design features and clinical and social cues on the extent of disagreement among participants in a formal consensus development process.

Methods: Factorial design involving 16 groups consisting of 135 general practitioners (GPs) and 42 mental health professionals from England. The groups rated the appropriateness of four mental health interventions for three conditions (chronic back pain, irritable bowel syndrome, and chronic fatigue syndrome) in the context of various clinical and social cues. The groups differed in three design features: provision of a systematic literature review (versus not provided), group composition (mixed versus GP only), and assumptions about the healthcare resources available (realistic versus idealistic). Disagreement was measured using the mean absolute deviation from a group's median rating for a scenario.

Results: None of the design features significantly affected the extent of disagreement within groups (all p>0.3). Disagreement did differ between treatments (closer consensus for cognitive behavioural therapy and behavioural therapy than for brief psychodynamic intervention therapy and antidepressants) and cues (closer consensus for depressed patients and patients willing to try any treatment).

Conclusion: In terms of the extent of disagreement in the groups in this study, formal consensus development was a robust technique in that the results were not dependent on the way it was conducted.

Full Text

The Full Text of this article is available as a PDF (88.0 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Bernstein S. J., Lázaro P., Fitch K., Aguilar M. D., Kahan J. P. Effect of specialty and nationality on panel judgments of the appropriateness of coronary revascularization: a pilot study. Med Care. 2001 May;39(5):513–520. doi: 10.1097/00005650-200105000-00011. [DOI] [PubMed] [Google Scholar]
  2. Brook R. H., Kosecoff J. B., Park R. E., Chassin M. R., Winslow C. M., Hampton J. R. Diagnosis and treatment of coronary disease: comparison of doctors' attitudes in the USA and the UK. Lancet. 1988 Apr 2;1(8588):750–753. doi: 10.1016/s0140-6736(88)91550-4. [DOI] [PubMed] [Google Scholar]
  3. Burgers Jako S., Grol Richard, Klazinga Niek S., Mäkelä Marjukka, Zaat Joost, AGREE Collaboration Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003 Feb;15(1):31–45. doi: 10.1093/intqhc/15.1.31. [DOI] [PubMed] [Google Scholar]
  4. Burnand B., Vader J. P., Froehlich F., Dupriez K., Larequi-Lauber T., Pache I., Dubois R. W., Brook R. H., Gonvers J. J. Reliability of panel-based guidelines for colonoscopy: an international comparison. Gastrointest Endosc. 1998 Feb;47(2):162–166. doi: 10.1016/s0016-5107(98)70350-5. [DOI] [PubMed] [Google Scholar]
  5. Campbell S. M., Hann M., Roland M. O., Quayle J. A., Shekelle P. G. The effect of panel membership and feedback on ratings in a two-round Delphi survey: results of a randomized controlled trial. Med Care. 1999 Sep;37(9):964–968. doi: 10.1097/00005650-199909000-00012. [DOI] [PubMed] [Google Scholar]
  6. Cluzeau F. A., Littlejohns P., Grimshaw J. M., Feder G., Moran S. E. Development and application of a generic methodology to assess the quality of clinical guidelines. Int J Qual Health Care. 1999 Feb;11(1):21–28. doi: 10.1093/intqhc/11.1.21. [DOI] [PubMed] [Google Scholar]
  7. Coulter I. D., Marcus M., Freed J. R. Consistency across panels of ratings of appropriateness of dental care treatment procedures. Community Dent Health. 1998 Jun;15(2):97–104. [PubMed] [Google Scholar]
  8. Herrin J., Etchason J. A., Kahan J. P., Brook R. H., Ballard D. J. Effect of panel composition on physician ratings of appropriateness of abdominal aortic aneurysm surgery: elucidating differences between multispecialty panel results and specialty society recommendations. Health Policy. 1997 Oct;42(1):67–81. doi: 10.1016/s0168-8510(97)00055-9. [DOI] [PubMed] [Google Scholar]
  9. Imamura K., Gair R., McKee M., Black N. Appropriateness of total hip replacement in the United Kingdom. World Hosp Health Serv. 1996;32(2):10–14. [PubMed] [Google Scholar]
  10. Landrum M. B., McNeil B. J., Silva L., Normand S. L. Understanding variability in physician ratings of the appropriateness of coronary angiography after acute myocardial infarction. J Clin Epidemiol. 1999 Apr;52(4):309–319. doi: 10.1016/s0895-4356(98)00166-8. [DOI] [PubMed] [Google Scholar]
  11. Murphy M. K., Black N. A., Lamping D. L., McKee C. M., Sanderson C. F., Askham J., Marteau T. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i-iv, 1-88. [PubMed] [Google Scholar]
  12. Naylor C. D., Basinski A., Baigrie R. S., Goldman B. S., Lomas J. Placing patients in the queue for coronary revascularization: evidence for practice variations from an expert panel process. Am J Public Health. 1990 Oct;80(10):1246–1252. doi: 10.2105/ajph.80.10.1246. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Park R. E., Fink A., Brook R. H., Chassin M. R., Kahn K. L., Merrick N. J., Kosecoff J., Solomon D. H. Physician ratings of appropriate indications for three procedures: theoretical indications vs indications used in practice. Am J Public Health. 1989 Apr;79(4):445–447. doi: 10.2105/ajph.79.4.445. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Raine Rosalind, Haines Andy, Sensky Tom, Hutchings Andrew, Larkin Kirsten, Black Nick. Systematic review of mental health interventions for patients with common somatic symptoms: can research evidence from secondary care be extrapolated to primary care? BMJ. 2002 Nov 9;325(7372):1082–1082. doi: 10.1136/bmj.325.7372.1082. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Raine Rosalind, Sanderson Colin, Hutchings Andrew, Carter Simon, Larkin Kirsten, Black Nick. An experimental study of determinants of group judgments in clinical guideline development. 2004 Jul 31-Aug 6Lancet. 364(9432):429–437. doi: 10.1016/S0140-6736(04)16766-4. [DOI] [PubMed] [Google Scholar]
  16. Richardson F. M. Peer review of medical care. Med Care. 1972 Jan-Feb;10(1):29–39. doi: 10.1097/00005650-197201000-00004. [DOI] [PubMed] [Google Scholar]
  17. Savoie I., Kazanjian A., Bassett K. Do clinical practice guidelines reflect research evidence? J Health Serv Res Policy. 2000 Apr;5(2):76–82. doi: 10.1177/135581960000500204. [DOI] [PubMed] [Google Scholar]
  18. Scott E. A., Black N. When does consensus exist in expert panels? J Public Health Med. 1991 Feb;13(1):35–39. doi: 10.1093/oxfordjournals.pubmed.a042575. [DOI] [PubMed] [Google Scholar]
  19. Shekelle P. G., Kahan J. P., Bernstein S. J., Leape L. L., Kamberg C. J., Park R. E. The reproducibility of a method to identify the overuse and underuse of medical procedures. N Engl J Med. 1998 Jun 25;338(26):1888–1895. doi: 10.1056/NEJM199806253382607. [DOI] [PubMed] [Google Scholar]
  20. Shekelle P. G., Schriger D. L. Evaluating the use of the appropriateness method in the Agency for Health Care Policy and Research Clinical Practice Guideline Development process. Health Serv Res. 1996 Oct;31(4):453–468. [PMC free article] [PubMed] [Google Scholar]
  21. Vader J. P., Burnand B., Froehlich F., Dupriez K., Larequi-Lauber T., Pache I., Dubois R. W., Gonvers J. J., Brook R. H. Appropriateness of upper gastrointestinal endoscopy: comparison of American and Swiss criteria. Int J Qual Health Care. 1997 Apr;9(2):87–92. [PubMed] [Google Scholar]
  22. Vader J. P., Porchet F., Larequi-Lauber T., Dubois R. W., Burnand B. Appropriateness of surgery for sciatica: reliability of guidelines from expert panels. Spine (Phila Pa 1976) 2000 Jul 15;25(14):1831–1836. doi: 10.1097/00007632-200007150-00015. [DOI] [PubMed] [Google Scholar]
  23. Vella K., Goldfrad C., Rowan K., Bion J., Black N. Use of consensus development to establish national research priorities in critical care. BMJ. 2000 Apr 8;320(7240):976–980. doi: 10.1136/bmj.320.7240.976. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Wood J., Freemantle N. Choosing an appropriate unit of analysis in trials of interventions that attempt to influence practice. J Health Serv Res Policy. 1999 Jan;4(1):44–48. doi: 10.1177/135581969900400111. [DOI] [PubMed] [Google Scholar]
  25. Woolf S. H., Grol R., Hutchinson A., Eccles M., Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999 Feb 20;318(7182):527–530. doi: 10.1136/bmj.318.7182.527. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Quality & safety in health care are provided here courtesy of BMJ Publishing Group

RESOURCES